Compare commits

..

50 Commits

Author SHA1 Message Date
Paul Nothaft 99df3e204f chore(stable): release 3.46.10 (#1332)
Build and Push Docker Images / build-backend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-backend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-backend (push) Blocked by required conditions
Build and Push Docker Images / build-frontend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-frontend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-frontend (push) Blocked by required conditions
Build and Push Docker Images / summary (push) Blocked by required conditions
2026-09-07 23:03:37 +02:00
Paul Nothaft 95e3af0800 Merge pull request #1327 from PicPeak/fix/sanitize-html-2.17.7-stable
fix(security): bump sanitize-html to 2.17.7 (stable)
2026-09-07 09:18:32 +02:00
Paul Nothaft 8421b7b668 fix(setup): require Node 22.12 for sanitize-html 2026-09-07 09:00:02 +02:00
Paul Nothaft 0f426ef699 fix(security): bump sanitize-html to 2.17.7
Trivy flags the backend image on two sanitize-html advisories, both
fixed upstream:

- CVE-2026-63670 (fixed 2.17.6): a literal solidus after a raw-text end
  tag (`</textarea/>`) is treated as text by htmlparser2 and re-emitted
  unescaped, so disallowed markup passes when textarea or xmp is in
  allowedTags.
- CVE-2026-84371 (fixed 2.17.7): an SVG SMIL animation whose
  attributeName selects href lets the sibling values/from/to/by
  attributes carry URLs past the scheme policy.

2.17.5 -> 2.17.7, exact pin as before. The new version brings its own
htmlparser2 12 / domhandler 6 / domutils 4 / dom-serializer 3 /
entities 8 tree under node_modules/sanitize-html; nothing else in the
lock moves.

That tree is ESM-only, so the backend now needs unflagged require(esm):
Node 20.19+ or 22.12+. The image is node:22-alpine and CI runs 22, but
engines.node still admitted 22.0-22.11, where require('sanitize-html')
throws ERR_REQUIRE_ESM at startup (publicSiteService loads it during
initialisation). engines is now ^20.19.0 || >=22.12.0 and the native
setup script's Node check enforces the same range instead of accepting
any 22.x. On the supported versions the sanitiser behaves identically
to 2.17.5 on the tracker and newsletter fixtures.

Jest 29's CommonJS registry cannot evaluate ESM either, so every suite
importing a route or service that uses the sanitiser would fail at
import. jest.config.js now maps `sanitize-html` to jest.sanitizeHtml.js,
which hands that one module to Node's real loader via
process.getBuiltinModule('module') — a plain require('module') inside
Jest is Jest's wrapper and returns an empty object for this package.
Verified against a real 2.17.7 install: the sanitiser suites and a
settings route suite pass; without the mapper they fail with "Cannot
use import statement outside a module".
2026-09-06 23:06:45 +02:00
Paul Nothaft be243aafe8 chore(stable): release 3.46.9 (#1283)
Build and Push Docker Images / build-backend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-backend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-backend (push) Blocked by required conditions
Build and Push Docker Images / build-frontend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-frontend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-frontend (push) Blocked by required conditions
Build and Push Docker Images / summary (push) Blocked by required conditions
2026-09-03 22:26:45 +02:00
Paul Nothaft 3f90221f40 Merge pull request #1281 from PicPeak/fix/security-scan-batch-1-stable
fix(security): batch 1 (stable) — zxcvbn DoS, revocation forgery, unlink traversals, stored Content-Type, edge middleware
2026-09-03 12:36:42 +02:00
Paul Nothaft ed0a8e7656 chore(deps): apply non-breaking npm audit fixes
Stable twin of the main commit: backend qs/body-parser, frontend axios,
dompurify, linkify-it and the transitive set npm audit fix resolves without
a major bump. sanitize-html 2.17.7 (ESM-only parser tree, Jest 29 cannot
load it; the advisory needs svg tags no sanitizer config allows) and the
tiptap / react-router majors are left out, as on main.
2026-09-03 12:15:53 +02:00
Paul Nothaft c89ce8e172 docs: say the upload allow-list covers every path, video extensions must be added
Settings help text for Allowed File Types (EN, DE). The reference page
lives in the docs repository (PicPeak/docs#18).

(cherry picked from commit f3b062a3, locale files only)
2026-09-03 12:15:39 +02:00
Paul Nothaft d81cade7cc fix(security): harden four smaller gallery and contract paths, drop the unmounted photo auth middleware
- the customer contract PDF stream applies assertContractPdfPath like the
  admin and public contract routes
- OG previews fall back to the site card for draft, archived and
  deactivated galleries instead of leaking name, date and welcome message
- video Range requests are validated before the 206 is written; a NaN,
  inverted or out-of-file range now answers 416
- share-token comparisons in gallery resolve/info use the constant-time
  helper share-login already used
- middleware/photoAuth.js and the galleryAuth/photoAuth/verifyGalleryAccess
  exports of middleware/auth.js were unreferenced since the static mounts
  went; the auth.js copy had neither slug binding nor issuer pin, so it is
  removed before anyone mounts it

(cherry picked from commit 835312e8e6)
2026-09-03 12:15:21 +02:00
Paul Nothaft 406c638451 fix(security): stop reflecting submitted values in validation errors everywhere, cap credential lengths, close the login timing oracle
safeValidationErrors moves to utils/routeHelpers and replaces every
res.status(400).json({ errors: errors.array() }) in the routes, so no 400
body carries the submitted value any more (setup, customer auth and
customer change-password were still echoing rejected passwords).

Admin login, gallery verify, customer login/register/reset, customer
change-password and setup now cap username/slug at 255 and passwords at
MAX_PASSWORD_LENGTH at the validator, so an oversized value never reaches
the lockout lookup, bcrypt or the failed-attempt log.

Admin and customer login run one bcrypt compare on every path; the unknown
account branch used to return in microseconds against ~100ms for a wrong
password, which enumerated usernames despite the generic message.

(cherry picked from commit 40a8a9882a)
2026-09-03 12:14:42 +02:00
Paul Nothaft b1369068ae fix(security): close three middleware gaps around the API edge
Stable port of the main commit; the admin-preview and maintenance-gate
items do not exist on this branch.

- the general rate limiter skipped anyone holding any verified JWT; a
  gallery token is minted for free on password-less galleries and slideshow
  links, so that was an unlimited budget for every /api route. Only admin
  sessions skip now
- the 50mb JSON limit is scoped to /api/admin and /api/v1; everything else
  gets 2mb, so an unauthenticated body can no longer stall JSON.parse
- the CSRF Content-Type gate accepted multipart from any origin; cross-site
  form posts are now rejected via Sec-Fetch-Site / Origin, with a Host match
  fallback for same-origin installs that leave FRONTEND_URL unset

(cherry picked from commit 839bf4e4, adapted)
2026-09-03 12:14:02 +02:00
Paul Nothaft a8d57f0d69 fix(security): never serve a photo under its stored MIME, and stop trusting the chunked-upload type
chunked-upload/init stored the client-declared mimeType on the photo row and
the gallery, secure-image and protected-image routes echoed it as
Content-Type, so a JPEG/HTML polyglot declared as text/html rendered inline
on the app origin for every guest. The admin photo route already resolved
the type safely (#908 review); that logic now lives in
utils/photoContentType and every serving route uses it.

The chunked path derives the MIME from the filename extension and requires
that extension to be on the admin allow-list, matching what the multipart
path enforces through its multer fileFilter.

(cherry picked from commit 063977d97d)
2026-09-03 12:12:56 +02:00
Paul Nothaft 882101b586 fix(security): contain logo, favicon and PDF-logo unlinks to their upload directories
Settings > Branding persisted logo_url / favicon_url verbatim and on clear
unlinked path.join(storage, url) behind a startsWith('/uploads/logos/')
check, which '..' segments pass. The business-profile PDF logo did the same
behind a /pdf-logo-\d+\./ marker test, and used absolute values as given.
Either let a settings.edit or settings.banking holder delete any file the
process can reach.

Both now resolve through helpers in utils/safePath that only ever name a
flat leaf inside the fixed directory. The /favicon.ico streamer is narrowed
the same way: it contained to the whole uploads/ root, which also holds
signed contracts and transfer files.

(cherry picked from commit 3e46530072)
2026-09-03 12:12:12 +02:00
Paul Nothaft c6d401685f fix(security): verify the signature before writing a token to the revocation list
revokeToken() base64-decoded the payload without checking the signature and
inserted a row keyed on id-iat-type, the same key isTokenRevoked() matches
for real sessions. The logout endpoints are unauthenticated, so anyone could
forge a payload naming another user's id, type and login second and log them
out remotely; a far-future exp also left rows that cleanup never swept.

Expiry is still ignored so logging out an expired session stays idempotent.

(cherry picked from commit 0ca0e4a922)
2026-09-03 12:12:12 +02:00
Paul Nothaft 6481708def fix(security): stop reflecting submitted passwords in validation errors
Codex review round 2. The 400 I added in the previous commit returned
errors.array() verbatim, and express-validator puts the submitted `value` in
each error -- so rejecting an oversized password echoed that password back, and
re-allocated up to the 50mb body limit on an unauthenticated endpoint, partly
undoing the denial-of-service fix this branch exists for.

The same call appeared at seven sites in this file, five of which validate a
password field: /admin/login, /gallery/verify, /gallery/:slug/client-login,
/admin/change-password and /password-strength. Every failed login was returning
the attempted password in its response body, where it reaches proxy logs, error
monitoring and browser tooling. Fixed at all seven rather than only the one the
review pointed at.

Only `value` is dropped. `msg`, `path` and the rest are kept, because the two
shapes express-validator produces are both consumed in the frontend -- AcceptInvite
reads {field, message} from routeHelpers.validateRequest, EventDetails reads
{msg, path} from raw errors.array() -- and switching auth.js to the helper's
shape would have broken the latter for a reason unrelated to security.

1 more test. Backend suite: 2744 passed.

(cherry picked from commit 903e471753)
2026-09-03 12:12:12 +02:00
Paul Nothaft 706d402c1e fix(security): enforce the strength-endpoint validators, and stop the generator spinning
Codex review round 1 on the batch-1 security fixes. One finding is a
regression this branch introduced.

generateSecurePassword retried by recursing on any candidate validatePassword
rejected. The new 128-character cap makes EVERY candidate invalid once a caller
asks for more than that, so `generateSecurePassword({ length: 129 })` went from
returning a password to unbounded recursion and a stack overflow. It now
refuses an impossible length up front, and the retry is a bounded loop rather
than recursion -- every candidate failing is possible for reasons other than
bad luck (a charset that cannot satisfy the configured policy), and that case
deserves an error someone can act on rather than a blown stack. No caller in
the repo passes a length at all; the hazard was in the exported surface.

The route validators were decorative. POST /api/auth/password-strength never
called validationResult(), so the length bound I added only recorded an error
that nothing read: the oversized body still reached zxcvbn and the endpoint
still answered 200. The cap inside validatePassword() was doing all the work.
Errors are now returned as a 400 before the validator runs, which is what the
previous commit claimed.

Also awaited validatePasswordInContext, which is async. Unawaited, `validation`
was a Promise and every field in the response -- valid, score, errors, feedback
-- came back undefined. Pre-existing, in the lines this change already touches,
and it made the endpoint useless for the real-time validation it exists for.

1 more test. Backend suite: 2742 passed. The 23 eslint errors in server.js are
pre-existing and identical on main.

(cherry picked from commit 054cd6f82f)
2026-09-03 12:12:12 +02:00
Paul Nothaft ed08ff84ff fix(security): bound password input before zxcvbn, and drop the legacy media mounts
Two findings from the GHSA-pwx6-5pqc-c5xq scan bundle, both verified against
the code and reproduced before fixing.

**Unauthenticated denial of service via password strength (csf_495d53fa).**
POST /api/auth/password-strength takes `body('password').notEmpty()` with no
upper bound, sits behind express.json({ limit: '50mb' }), and hands the string
to zxcvbn, whose matching is superlinear and runs synchronously on the event
loop. Measured on this codebase, in ms of blocked loop: 128 -> 41, 512 -> 1367,
1000 -> 5097, 5000 -> did not return in two minutes. One unauthenticated
request of about a kilobyte stops the whole process for five seconds; a few
kilobytes stops it indefinitely. The /api/auth rate limit does not help when a
single request is already enough.

The cap lives in validatePassword() so it covers every caller, present and
future; the route validator is defence in depth. 128 keeps the worst case at
the cost of an ordinary request while staying far above any real password --
bcrypt consumes only the first 72 bytes, so length past that adds no entropy to
the stored hash anyway. This is the only unauthenticated reach into zxcvbn:
setup is token-gated and self-closing, and acceptInvite/adminAuth use the
regex-only validator in passwordGenerator.

**The /photos and /thumbnails static mounts (csf_9aa6afe6, csf_559cd5cc,
csf_b14d462e, csf_547d26fa, and the gallery half of csf_34e420af).**
They served the raw originals and thumbnail trees behind photoAuth, which
authorises on a slug match. A static file server cannot apply per-photo rules,
so everything the gallery API decides was absent: allow_downloads, per-category
allow_downloads, watermarking, the resolution cap, reveal-mode windows,
visibility='hidden', download logging, and the customer-assignment re-check
that makes revocation immediate. photoAuth also bcrypt-compares an
x-gallery-password header per request with no limiter -- both rate-limit gates
return early for non-/api paths -- so the mount was an unmetered password
oracle. The filenames needed to drive all of this are handed to every guest in
the photos listing.

Nothing builds these URLs: no reference in frontend/src, none in the email
templates, and the only backend mentions are the /api/admin/photos/... API
routes and a maintenance-mode prefix list. The equivalent authorised routes are
/api/gallery/:slug/photo/:id and /thumbnail/:id. nginx still proxies the two
locations; they now 404, which is the intent.

**The /uploads mount (csf_1fc92f57).** It exposed the whole uploads/ root with
no auth middleware at all, and that root also holds signed contract PDFs
(uploads/contracts/signed) and client transfer files (uploads/transfers/<id>),
reachable by anyone who learned or guessed a filename. Narrowed to the two
public asset trees it exists for; contracts and transfers keep their own
authorised routes.

Removing the mounts leaves src/middleware/photoAuth.js unreferenced by
application code. Left in place deliberately -- deleting it and its tests is a
separate cleanup, and a smaller diff backports more safely.

Backend suite: 2742 passed.

(cherry picked from commit 14cd5eacb3)
2026-09-03 12:12:06 +02:00
Paul Nothaft 7fe80220f1 chore(stable): release 3.46.8 (#1250)
Build and Push Docker Images / build-backend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-backend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-backend (push) Blocked by required conditions
Build and Push Docker Images / build-frontend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-frontend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-frontend (push) Blocked by required conditions
Build and Push Docker Images / summary (push) Blocked by required conditions
2026-09-01 22:29:02 +02:00
Paul Nothaft ccdcdd6116 chore: keep issue screenshots out of the source tree (#1260)
Preventative twin of the main-side cleanup. This branch has no stray images
to remove — it just gets the same guard, so the two branches agree and a
backport cannot carry one across.

Two PR screenshots landed at main's repo root in #1241 and shipped as part of
the source tree. Screenshots belong on a `screenshots/*` branch, which is how
every other UI change here has attached its evidence.

Anchored with a leading slash so docs/ keeps its own images and test-assets/
keeps the fixtures the e2e specs load. Verified no tracked file on this branch
matches the new patterns.

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
2026-09-01 09:34:55 +02:00
Paul Nothaft ccc725f36e fix(upload): let Android guests reach the camera without breaking video (#1248)
* fix(upload): let Android guests reach the camera without breaking video

Stable twin of #1244 (which replaces #1117). The reporter is on 3.46.1, so
this branch is where the bug is actually being hit.

Recent Android versions route an <input> whose accept list is entirely
image/video types to the system photo picker, which has no camera entry —
so a guest standing at the event can only pick an existing photo, not take
one. Including a type that picker can't handle forces the general chooser,
which does offer the camera.

Gated on the Android UA: iOS and desktop pickers behave correctly and would
only gain a selectable PDF that addFiles then rejects. No image-only guard
— #1117 added one that broke video uploads outright on any install
configured for them, and it was redundant anyway, since
extensionsToMimeTypes only emits types it has a mapping for and the
existing allowlist check already rejects a picked PDF.

The premise — that this actually surfaces the camera option on Android — is
taken at the reporter's description level and still needs confirmation on a
device.

Co-authored-by: Zszywany <Zszywany@users.noreply.github.com>

* fix(upload): use android/allowCamera instead of .pdf for the chooser fallback

Same mechanism, better token. Chrome on Android 14/15 sends an input whose
accept list is all media types to the photo picker, which has no camera tile;
adding a value that picker cannot satisfy makes it fall back to the general
chooser, which does offer the camera.

`.pdf` achieves that but advertises PDFs as selectable — pick one and the
existing allowlist check answers "Invalid file type", which is a dead end we
put in front of the guest ourselves. `android/allowCamera` is the token the
workaround converged on: not a real MIME type, matches no file, so it flips
the picker without offering anything.

Neither token ever widened what is accepted — addFiles validates against
extensionsToMimeTypes, which only emits types it has a mapping for — but not
showing the guest a choice that cannot work is worth the one-line change.

Verified in a browser rather than asserted: the real component rendered under
an Android UA emits

  image/jpeg,image/png,image/webp,android/allowCamera

and under a desktop UA

  image/jpeg,image/png,image/webp

with the visible modal identical in both, and the format hint still reading
"JPG, JPEG, PNG, WEBP" — the token does not leak into anything a guest sees.

* fix(upload): keep the camera token off Firefox for Android

External review round. The gate was a bare /Android/i, which Firefox for
Android matches — so it received a token invented to reroute Chromium's photo
picker, a picker it does not use. The doc comment two lines up already said
Firefox behaves correctly; the code did not agree with it.

Inert at best, and at worst it perturbs a chooser that was working. Narrowed
to Android minus Firefox, which is the Chromium-family set the behaviour was
actually observed on (Chrome and Edge, Android 14/15), with a UA test to pin
it.

---------

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
Co-authored-by: Zszywany <Zszywany@users.noreply.github.com>
2026-09-01 09:24:38 +02:00
Paul Nothaft fed99ac03d fix(archives): write a real timestamp on restored photos (#1257)
Brings this branch in line with main, which fixed it in passing.

The archive restore inserted photos with a bare Date for uploaded_at. Inside
jest the sqlite3 binding's type dispatch misses sandbox-created Dates and
stores the literal string "[object Object]", so every restored photo got a
garbage timestamp. Verified on this branch rather than assumed:

  bare Date   -> "[object Object]"
  toISOString -> "2026-09-01T06:48:41.915Z"

Production writes Dates as ms-numbers and is unaffected, which is exactly why
it survives unnoticed — it only corrupts what tests read back, so a future
test asserting on a restored photo's date would have believed it.

The regression test fails against the previous line.

Not touched: the category insert a few lines up has the same shape, but it is
identical on main, so fixing it here alone would re-open the divergence this
commit closes. Worth one small PR against both branches.

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
2026-09-01 08:52:35 +02:00
Paul Nothaft a01731d986 chore: remove .pyc files committed to stable by mistake (#1256)
My doing, in #1247: I built that stable twin in a working tree that still
held untracked bytecode from main's ML sidecar, and a `git add -A` swept 16
.pyc files in alongside the two real ones.

Stable never carried the ignore rule because the sidecar itself is main-only
— which is precisely why nothing stopped it here. Added, so a shared working
tree cannot repeat it.

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
2026-09-01 08:41:37 +02:00
Paul Nothaft 1d9f0b6c64 fix(events): apply the gallery password policy to publish (#1255)
Stable twin of #1253. This branch has only the publish door —
send-gallery-email is #1235, main-only — so the same gap exists here in one
place rather than two.

/publish re-hashes password_hash from a plaintext the admin re-types in the
publish dialog, validated with nothing but express-validator's
isLength({min:6}). So the configured complexity — moderate by default,
meaning 8 characters plus upper, lower and a digit — governed event creation
and password reset while this door accepted 'aaaaaa' and made it the live
gallery password.

Not an escalation: it needs admin auth plus events.edit. It is a policy gap,
the admin UI advertising a complexity level this write path did not enforce.

BEHAVIOUR CHANGE: an API-only consumer publishing with a sub-policy password
now gets 400, with the same body shape event creation returns (error,
details, score, feedback).

3 tests, including that the rejection happens BEFORE the write — the gallery
keeps its old hash and stays a draft — and that a publish carrying no
password at all is untouched.

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
2026-09-01 08:36:41 +02:00
Paul Nothaft 261e243070 fix(archives): take the restored category from the manifest (#1240) (stable) (#1243)
* fix(archives): take the restored category from the manifest (#1240) (stable)

Stable twin of #1240. The reporter hit this on 3.46.7 — stable — with 596
photos restored and 0 categories, so this is the branch the bug was actually
found on.

Stable carries the manifest on both sides already: archiveService selects
`photo_categories.name as category_name` and serialises it, and the restore
route builds manifestByFilename. It just never read the category out of it,
deriving one from the ZIP's first path segment instead. Archives store photos
as they sit on disk, so an event whose photos live in the gallery root
produces a flat zip, no category resolves, and every photo comes back with
category_id null — silently, behind a 200.

Carries the whole of #1240, not a subset: the manifest-first resolution and
the shared resolveCategoryId from Marian's commit, plus the follow-up that
makes the manifest authoritative when it says "no category" — an entry with a
null category_name is a photo that was genuinely uncategorized, and falling
through to the directory contradicted the record being restored from. That
matters because the directory is not a category: entry names are the storage
key minus events/active/{slug}, so a real archive yields `individual/` and
`collages/`, and reading the first segment invents categories with those
names.

Re-verified on stable rather than assumed: all four tests pass here, and three
of them fail against stable's current route, with the legacy no-manifest
fallback passing either way. The two changed files are byte-identical to main.

Co-authored-by: Marian <lippitz.marian@yahoo.de>

* fix(archives): keep the stable twin to stable's schema, and close two category holes

External review on #1243 caught that this twin was ported wrong and that the
category resolver has two holes the main PR shares.

PORTED WRONG. I took main's whole adminArchives.js rather than applying the
category change to stable's, which dragged in main-only face cleanup:
photo_faces and event_people have migrations on main and none on stable, so
every permanent archive deletion would have thrown a missing-table error —
after the ZIP was already unlinked, leaving the event archived with its
archive gone and a 500 back. Rebuilt from stable's file with only the category
change; the diff against stable is now the fix and nothing else.

GLOBAL CATEGORIES WERE CLONED. Seeded categories (Ceremony, Reception) have
event_id NULL, so an event-only lookup missed them and created a second row —
and is_global defaults to TRUE, so that duplicate then appeared in every other
event's category list. The lookup now uses the same visibility rule the photo
routes use (own rows OR global), and anything it does create is explicitly
is_global false.

ORIGINAL-FILENAME ARCHIVES MATCHED NOTHING. With
general_use_original_filenames_for_downloads on at archive time, archiveService
names each ZIP entry after the original filename while the manifest stays keyed
by photos.filename — so the lookup missed every entry and those archives lost
categories exactly as before the fix. The manifest is now indexed by
original_filename as well, without letting it shadow a real filename key.

7 tests, three of them new; each new one fails against the un-fixed route and
the legacy no-manifest fallback passes throughout.

* fix(archives): sanitized original names and deterministic category scope

Round 2 of external review on #1243.

The original_filename index used the raw column, but archiveService runs the
name through sanitizeForZipEntry() before writing the entry — so an original
containing a slash or control byte was emitted under a different name than the
manifest records, and the lookup missed it. Both spellings are indexed now,
using the same helper the writer uses.

Not total, and the comment says so: uniquifyZipNames() appends `_1` when two
photos in one event share an original name, and that suffix cannot be
reconstructed from the manifest. Those fall through to the directory exactly
as they did before this fix — no worse, just not better. Closing it needs the
emitted name recorded at archive time, which is a writer change and a new
archive format.

The category lookup used one OR-query with .first(). An event-scoped category
and a global one may share a name — the category API permits it — so the
engine picked whichever, and a photo could be silently reassigned to the
global row, losing event-local settings like allow_downloads. Two queries now,
event-scoped first: the event's own row is the more specific answer.

9 tests, two new; both fail against the un-fixed route.

* fix(archives): don't adopt another event's legacy row, don't guess an alias

Round 3 of external review on #1243.

The global fallback matched on is_global alone. The very bug fixed here left
rows behind on upgraded instances — event-owned AND is_global true, because
the column defaults true — so restoring event B could adopt event A's
leftover, tying B's photos to a category that disappears when A is deleted.
The fallback now requires event_id IS NULL: genuinely global, not merely
flagged.

The original-filename alias map collapsed rows that share a basename.
archiveService treats `individual/IMG.jpg` and `collages/IMG.jpg` as distinct
paths and suffixes neither, so both manifest rows claimed one alias and
whichever won handed the other photo someone else's category. An alias claimed
by more than one row is now dropped and logged, so those photos fall back to
the directory: an unresolved category is recoverable, a confidently wrong one
is not.

11 tests, two new; both fail against the un-fixed route.

* fix(archives): make the manifest lookup order-independent and collision-safe

Two bugs found by an external review round, both in the manifest index.

The canonical map silently kept the last row for a duplicated
photos.filename. That column is not unique within an event — s3AutoImporter
takes path.basename(entry.key) and dedupes by path, so two imported files in
different subfolders both land as IMG_1234.jpg with different paths. At
restore both ZIP entries reduce to the same basename, so one photo got the
other's category. Contested names are dropped now, like ambiguous aliases
already were.

The alias pass could also evict a canonical key: when one row's
original_filename equalled another row's filename, the collision was marked
ambiguous and the sweep deleted the canonical entry. The comment two lines
above says a real filename key is authoritative and must never be
overwritten — the code did the opposite, and which way it went depended on
manifest iteration order, since the archive query has no ORDER BY.

Split into two passes so canonical names are claimed first and aliases only
fill names no canonical row wanted.

* fix(archives): treat a canonical/alias name clash as ambiguous, resolve categories lazily

Round-2 findings, one of which corrects my own round-1 fix.

Round 1 made a canonical filename outrank any alias. That is the wrong
tiebreak: when photo A's filename equals photo B's original_filename, which
file the ZIP actually emitted under that name depends on whether
original-filename archiving was on at archive time — with it ON the entry is
B's, with it OFF it is A's — and the manifest does not record the mode.
Preferring either silently mislabels the other half of the time, so the name
is dropped and both fall through to the directory. What the two-pass split
still buys is determinism: the archive query has no ORDER BY, so this used to
be a coin flip between dropping the name and overwriting it.

Categories are resolved inside the !existingPhoto branch. resolveCategoryId
find-or-CREATES, and archiveEvent retains photo rows, so restoring an archive
whose rows still exist created a category from the stale manifest name that
nothing then used — renaming a category while its event was archived left the
old name behind as an empty duplicate.

Not fixed: two event-scoped categories may share a display name with distinct
slugs, and the .first() lookup then picks either row, so manifest entries from
both collapse onto one id and can inherit the wrong allow_downloads. Detecting
it is easy; resolving it correctly needs a stable category identifier in the
manifest, which is a writer change and an archive-format bump.

* fix(archives): make a duplicate category name deterministic, and log it

Round-2 finding. Two event-scoped categories may share a display name when
their slugs differ, and the .first() lookup then picked one arbitrarily —
manifest entries for both collapsed onto a single id and half the photos
inherited the wrong per-category settings, allow_downloads above all.

Fixing it properly needs a stable category identifier in the manifest: a
writer change, an archive-format bump, and no help at all for archives
already written. Not worth building before knowing it happens. So the
collision is surfaced instead — a warning naming the category and the row
count — and the tiebreak is made deterministic (lowest id) so at least a
re-run lands the same way twice.

If this never fires in real logs, the format change was not worth making. If
it does, this is the evidence for it.

---------

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
Co-authored-by: Marian <lippitz.marian@yahoo.de>
2026-09-01 08:17:48 +02:00
Paul Nothaft 5470fbe406 fix(gallery): route single-photo downloads through the storage backend (#1246)
* fix(gallery): route single-photo downloads through the storage backend

Stable twin of #1048. The route resolved a local filesystem path
unconditionally and handed it to res.sendFile. On an S3/R2 deployment
managed photos are never on local disk, so every per-photo download failed
— while download-all and secure-images worked, because they already went
through getStorage(). That asymmetry is why it went unnoticed: the gallery
looks healthy until a guest clicks the download button on one photo.

Because sendFile is called WITH a callback, Express does not send a
response when the file is missing and the callback only logs — the request
does not 404, it hangs until the client gives up. The new tests pin this:
all five backend-path cases time out against the current implementation.

- watermark branch: materialize a tmp local copy via withLocalCopy in S3
  mode and hand applyWatermark the copy's PATH, so its path-keyed cache
  still applies. Same pattern the zip builders in this file already use.
- pass-through branch: local disk keeps res.sendFile, which emits
  Content-Length, Accept-Ranges, ETag and Last-Modified and answers Range
  with a 206. Sharing one bare stream.pipe(res) with S3 would silently drop
  all of it, and a resumed download would append a second full body onto
  the partial file. On S3 the parts that matter are reproduced via stat()
  and getRange().
- external/reference photos keep the local-path fallback unchanged —
  resolvePhotoStorageKey returns null for them.

Ranges are parsed defensively: an unchecked parse yields NaN bounds and a
206 with a nonsense Content-Range, which corrupts a resumed download rather
than failing it. Malformed or unsatisfiable ranges fall back to a 200.

Written against stable's shape rather than cherry-picked — main's version
delegates to renderPhotoForDownload (#858), which does not exist here.

Co-authored-by: Peifu Mo <peipeimo@users.noreply.github.com>

* fix(gallery): open the stream before staging download headers, honour If-Range

Both from an external review round on #1048.

stat() succeeding does not mean get() will — a concurrent delete or replace,
or a transient backend error, lands between them. The fetch was awaited
AFTER the headers went out, so the range branch had already called
writeHead(206) and the outer catch could only throw ERR_HTTP_HEADERS_SENT
(in practice the request hangs), while the full branch would have sent its
500 JSON underneath the staged image/jpeg attachment headers.

Opening the stream first also lets a vanished object answer 404 and a
transient failure answer 500.

If-Range: emitting Last-Modified without honouring the validator built from
it is the dangerous half. A client resuming after the object was replaced
would get 206 from the NEW bytes and splice two versions into one corrupt
file. A non-matching validator now falls back to a full 200.

* fix(gallery): HEAD without egress, classify render failures, stage 206 headers

Round-2 findings from the external reviewer on #1048, ported.

Express routes HEAD through this GET handler and Node discards the body, but
the pipe still drains the whole object out of S3 first — a metadata probe
cost a full transfer in egress and latency. Everything a HEAD needs is
already in stat().

The watermark branch reported every failure as 404. It can equally fail
because getToFile timed out, tmp filled up, or sharp died; calling that
"photo not found" misleads the guest and hides the incident.

The 206 path uses status()+set() instead of writeHead(), which commits
immediately and left a stream erroring at byte zero with no outcome but a
destroyed connection. Staged headers flush on first write, so that case now
returns a clean retryable status.

pipeStreamToResponse also cleared Content-Type, Content-Length, ETag and
Content-Disposition but not the range headers, so the 500 went out still
advertising Content-Range — telling a resuming client the error body IS the
partial content.

* fix(gallery): answer HEAD before the counters

Round-3 finding on #1048, ported. The HEAD short-circuit was inside the
storage branch, below both the download_count increment / access_logs insert
and the watermark path — so a download manager's metadata probe counted as a
real download, and on a watermarked gallery it also pulled the original from
S3 and ran sharp over it to build a body Node then discards.

HEAD now leaves right after the access checks. Content-Length is included
only when the photo ships untransformed and the size is readable from stat();
a watermark changes the length and the only way to learn it is to do the work
this branch exists to avoid.

Uses stable's inline watermark resolution — resolveWatermarkSettings comes
from downloadRendition (#858), which does not exist on this branch.

---------

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
Co-authored-by: Peifu Mo <peipeimo@users.noreply.github.com>
2026-09-01 08:17:19 +02:00
Paul Nothaft 7102687ee8 fix(events): delete stored objects when cascading an event delete (#1245)
* fix(events): delete stored objects when cascading an event delete

Stable twin of #1051. Deleting an event removed its database rows but left
every stored file behind: deleteEventCascade() cleaned up with fs.rm over
{STORAGE_PATH}/events/{active,archived}/{slug}, and on an S3-compatible
backend those paths don't exist locally — the call succeeds against nothing
and the real objects stay in the bucket, unreferenced by any row, invisible
in the UI, and billed every month.

Measured on a v3.45.16 install against Cloudflare R2, deleting one
403-photo event: bucket object count 5,400 before and 5,400 after, while
referenced rows dropped from 3,425 to 2,746.

Keys are collected BEFORE the transaction removes the photo rows — once
they are gone nothing records which objects belonged to the event, and only
a full-bucket audit against the whole database could find them again — and
deleted AFTER the commit, so a rolled-back delete can never destroy files
for an event that still exists.

Includes photo.watermark_path and event.archive_path, both storage-backed
and both previously fs.unlink-only. event.hero_logo_path is deliberately
excluded: multer writes logos to local disk with diskStorage regardless of
backend, so they are never bucket objects.

Reference/external photos are left alone — resolvePhotoStorageKey returns
null for them and PicPeak does not own those bytes.

This branch carries the higher priority of the pair: unlike main, stable's
deleteEventCascade never calls getStorage() at all, and the leak costs real
money for every month it goes unfixed.

Co-authored-by: Peifu Mo <peipeimo@users.noreply.github.com>

* fix(events): sweep the Download All cache, and delete objects concurrently

Both from an external review round on #1051.

The pre-built "Download All" zip (events.download_zip_path) lives under
events/active/{slug}/.download-cache/. On local disk the recursive fs.rm
already covered it, which is exactly why it was easy to miss — on S3 that
prefix is not a directory, nothing covered it, and it is gallery-sized.
downloadZipService exposes a cleanup() documented as "used on event
deletion" that the cascade never called.

download_jobs (main's #173) does not exist on this branch, so the
per-job archives main also sweeps have no counterpart here.

Deletes now run through a bounded pool instead of one await per key: a
400-photo gallery owns well over a thousand objects, and that many
sequential DeleteObject round trips runs to minutes — long enough for a
proxy to time the request out AFTER the commit, leaving the event deleted
and the sweep half-finished.

* fix(events): never delete a derivative another gallery still uses

Round-2 findings from the external reviewer on #1051, ported.

Canonical thumbnail/hero/preview keys are not event-scoped: the basename is
the photo's filename, and filenames are not unique across events. A legacy
gallery can share a canonical derivative with a photo in another event, and
deleting it here blanked a surviving gallery's tile. Derived keys are now
checked against photos outside this event and anything still referenced is
left alone; if the check itself fails, every derivative is kept — an orphan
costs storage, a deleted derivative costs someone else's gallery. Originals
need no check, their keys embed the slug.

Also cancel any in-flight or debounced Download All build before snapshotting
paths, via downloadZipService.cleanup() — the service's own entry point for
event deletion. A builder that started before the delete would otherwise
upload a gallery-sized zip after the sweep and write its path onto a row that
no longer exists.

* revert(events): drop the Download All build cancellation

It broke CI on this branch: the backend job went from ~2 minutes to
exceeding its 10-minute budget, twice, reproducibly.

downloadZipService.cleanup() reaches getStorage() through _cleanup(), and in
a suite where the S3 backend is configured but unreachable every cascade
delete then pays the adapter's retry backoff. The full suite passes locally
against SQLite, which is why this only showed up in CI.

The race it addressed is real but narrow — a builder that started before the
delete uploads its zip after the sweep and writes the path onto a row that
no longer exists, orphaning one object. That is a cheaper problem than an
unrunnable test suite, so it goes back to being a documented follow-up
rather than shipping behind a timeout.

The shared-derivative guard from the same review round stays: that one
prevented deleting a surviving gallery's thumbnail.

---------

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
Co-authored-by: Peifu Mo <peipeimo@users.noreply.github.com>
2026-09-01 08:16:51 +02:00
Paul Nothaft 5b69e3ec4c fix(auth): treat zxcvbn suggestions as advice, not blocking errors (#1247)
Stable twin of #1050. passwordValidation.js is byte-identical between the
branches, so this is the same change verbatim.

validatePassword() appended zxcvbn's feedback.suggestions to the errors
array unconditionally, and validity is errors.length === 0 — so any
password that merely earned a suggestion was rejected even when it
satisfied every configured rule. The effective policy was stricter than
the configured complexity level and invisible to the admin.

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
Co-authored-by: Peifu Mo <peipeimo@users.noreply.github.com>
2026-09-01 08:16:33 +02:00
Paul Nothaft eebca9900b fix(auth): treat zxcvbn suggestions as advice, not blocking errors (#1247)
Stable twin of #1050. passwordValidation.js is byte-identical between the
branches, so this is the same change verbatim.

validatePassword() appended zxcvbn's feedback.suggestions to the errors
array unconditionally, and validity is errors.length === 0 — so any
password that merely earned a suggestion was rejected even when it
satisfied every configured rule. The effective policy was stricter than
the configured complexity level and invisible to the admin.

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
Co-authored-by: Peifu Mo <peipeimo@users.noreply.github.com>
2026-09-01 08:05:57 +02:00
Paul Nothaft c05faa50d9 chore(stable): release 3.46.7 (#1221)
Build and Push Docker Images / build-backend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-backend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-backend (push) Blocked by required conditions
Build and Push Docker Images / build-frontend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-frontend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-frontend (push) Blocked by required conditions
Build and Push Docker Images / summary (push) Blocked by required conditions
2026-08-29 03:25:38 +02:00
Paul Nothaft 15c844db06 fix(admin): the "Uncategorized" photo filter returns every photo (#1211) (#1215)
Stable twin of the same fix on main.

The dropdown rendered as `value="0"` and adminPhotos.js:1001 skips '0', so no
category condition was applied and the whole event came back. The branch that
does the work sits four lines below, keyed on the literal 'uncategorized' that
nothing was sending.

Silent by nature — a full list reads as 'nothing to narrow' rather than 'the
filter did not run' — which is why it survived this long.

The reporter in #1209 is on 3.46.4, so this is the branch that reaches them.

Tests both ends of the contract, since the bug was the pairing rather than
either half.

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
2026-08-28 08:05:21 +02:00
Paul Nothaft c685a3e931 chore(stable): release 3.46.6 (#1207)
Build and Push Docker Images / build-backend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-backend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-backend (push) Blocked by required conditions
Build and Push Docker Images / build-frontend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-frontend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-frontend (push) Blocked by required conditions
Build and Push Docker Images / summary (push) Blocked by required conditions
2026-08-28 04:00:46 +02:00
Paul Nothaft 74ff236b51 fix(images): fence the capture-date backfill on the file it read (#1201) (#1205)
Stable twin of #1204.

The capture-date backfill committed its result keyed on the row id alone. It
snapshots every candidate up front, then walks them one at a time reading
originals off S3 or a NAS mount — a pass that can run for many minutes.

replacePhoto, reachable from the replace_by_name upload path, swaps a NEW file
under an existing row and rewrites path/filename. A replacement landing inside
that window carries no date of its own, so captured_at was still NULL, the
whereNull guard passed, and the previous file's EXIF date was written onto the
new photo. Silent: nothing errored, the run reported it as a success, and the
gallery just sorted that photo to the wrong place.

Fenced on path and filename as well as the id, so a replaced row matches zero
rows and is skipped. The candidate query already selects both columns, so no
query change. Knex renders a null value in the object form as `is null` on both
the pg and sqlite3 clients, so a row with a NULL path still matches itself.

Those skipped candidates are now counted rather than dropped. replacePhoto is
not the only writer of path/filename — eventRenameService rewrites both on an
event rename, which is not a content change — and another writer filling
captured_at first lands in the same place. Without a counter they fell out of
the run's arithmetic entirely: success + noExif + failed no longer added up to
the count the operator was shown when they started the job.

The card shows the count only when it is non-zero, and states what is known —
changed by something else, not updated — rather than promising a retry: for the
already-dated case there is nothing to retry, and the Missing Capture Date
figure above is what says whether work is left. Locale coverage: en, de, fr, sl,
with the defaultValue carrying the rest.

Regression test: a replacement landing mid-run leaves captured_at NULL and is
not counted as updated. Verified to fail against the unfenced code on this
branch.
2026-08-27 08:44:32 +02:00
Paul Nothaft 292dd4fa09 chore(stable): release 3.46.5 (#1193)
Build and Push Docker Images / build-backend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-backend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-backend (push) Blocked by required conditions
Build and Push Docker Images / build-frontend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-frontend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-frontend (push) Blocked by required conditions
Build and Push Docker Images / summary (push) Blocked by required conditions
2026-08-26 21:44:17 +02:00
Paul Nothaft 5559cd333d fix(images): respect EXIF orientation in thumbnails, heroes, previews and watermarks (#1185) (#1202)
Stable twin of #1194.

generateThumbnail, generateHeroImage and generatePreviewImage went straight
from sharp(imagePath) to .resize(), so a photo whose Orientation tag is not 1 —
routine for portrait shots on bodies that tag rather than rotate the sensor
data — was resized from the raw frame and came out sideways. The same pipelines
then call .withMetadata(false), stripping the tag from the output, so nothing
downstream could correct it either. The download path already had this right,
which is why the same photo looked correct on download and rotated in the
gallery.

watermarkService had it too, and it is the one a guest actually sees:
gallery.js serves photos.watermark_path ahead of the original when branding
watermarking is on. Two details there — metadata() is read from a separate
unrotated handle, because .rotate() does not change what it reports and every
use of those numbers is positioning; and the composite offsets are floored,
because getPositionCoordinates returns fractional pixels, sharp rejects them,
and applyWatermark catches its own error and silently returns the image
unwatermarked.

The rotate is unconditional in the thumbnail and hero generators — neither
passes `animated: true`, so both already flatten a multi-frame source and
guarding there would protect an animation that was being discarded anyway.
generatePreviewImage keeps the guard, since it genuinely does preserve
animation.

photos.width/height were stored from sharp's metadata, which reports pixels as
STORED, not displayed. For orientation 5-8 those are swapped, so a portrait
photo landed in the database as landscape and masonry sized its tile with the
wrong aspect ratio on top of the image being unrotated. A shared
orientedDimensions() helper now does the conversion at all eight image ingest
sites. The video path is deliberately untouched: its dimensions come from
ffprobe, where EXIF orientation does not apply.

Existing rows keep their pre-rotation dimensions until reprocessed; the backfill
for those is #1199 on main and is not ported here yet.

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
2026-08-26 21:13:08 +02:00
Paul Nothaft ac7ef266dc fix(admin): make "Storage used" report storage used (#1164) (#1177)
* fix(admin): make "Storage used" report storage used (#1164)

Stable twin of #1170.

The tile summed photos.size_bytes — the catalogued size of the ORIGINALS,
which in reference mode live on external storage and have no relationship to
the disk PicPeak runs on. The reporter's tile read ~80 GB against 21 GB of
real usage. Worse than the label: the same number drove the soft-limit warning
bar and, via /storage/info, the recommended soft limit — so a reference-mode
install got a disk-capacity recommendation computed from bytes that are not on
the disk.

- new localStorageUsage service walks the storage root and reports the total
  plus a breakdown. Walking rather than summing DB columns is the point:
  thumbnail/preview/hero rows record a key and never a byte count, and orphans
  from a deleted event or an interrupted import are real bytes.
- the external media root is excluded when it sits inside the storage root.
  Its compose default is <storage>/external-media, where the NAS is
  bind-mounted — a plain directory, not a symlink — so walking it would put
  every referenced original back into a figure whose purpose is to leave them
  out. Symlinks are not followed either.
- .download-cache gets its own line: it lives inside the event directory, so
  the naive rule files a multi-GB zip as photography.
- concurrent cold-cache callers share one walk; the dashboard, /storage/info
  and the sidebar are routinely requested together.
- S3 installs keep the catalogued figure and the walk is skipped before it
  runs, since the objects are in the bucket and STORAGE_PATH holds only
  incidental local files.
- an absent measurement reads as "unavailable" and a partial one is marked
  `+` across the dashboard, analytics, sidebar and status tab — a floor
  silently compared against a soft limit reads as "safely under".

Verified on this branch: 11 new service tests, dashboardScope updated for the
changed contract, full suite leaves the same 5 pre-existing failures as
origin/stable. Frontend 20 files / 104 tests, tsc clean.

* fix(admin): tell "no disk to measure" apart from "the measurement failed" (#1164)

External review found both of these on this branch.

Both were reported as `storage_measurement: 'catalog'`, so a failed local walk
made the dashboard claim the objects live in S3. They are different things —
one is a fact about the install, the other is a fault — and there is now an
`unavailable` state for the second.

The analytics percentage could reach the billions. `safeSoftLimit` fell back to
`storageUsed || 1`, and on S3 that is null → 1, while the figure beside it came
from `catalogedBytes`. An editor or viewer holds `analytics.view` but not
`settings.view`, so `/storage/info` 403s for them and `storageInfo` is
undefined — which is exactly when that fallback fires. It now falls back to the
measured figure, and suppresses the percentage entirely when there is no real
limit rather than dividing usage by itself and always reading 100%.

---------

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
2026-08-26 09:14:48 +02:00
Paul Nothaft 9ffbe2f98f fix(previews): preserve alpha and animation in the preview tier (#1176)
* fix(gallery): stop the lightbox loading originals to display a photo (#1166)

Stable twin of #1169.

The lightbox read preview_url, which the server only emits once an admin has
flipped lightbox_preview_enabled — off by default. So a stock install fell
straight through to url, the untouched original: a reporter measured 16.5 MB
for a photo whose preview is 345 KB. The lightbox renders its neighbours too,
so opening one photo pulled three originals.

slideshow_url is the same /preview/:id URL, watermark query included, and has
been emitted unconditionally for images since #1015. Preferring it fixes every
existing install with no migration and no admin action.

Two other surfaces bypass PhotoLightbox entirely and had the same bug:

- premium galleries build their own slides with `src: photo.url`. Fixing that
  also required carrying the photo id on the slide, because the download
  handler recovered the photo by matching slide.src against photo.url — a
  derivative src would have made Download a silent no-op.
- the Story layout rendered the full original as its GRID TILE, at
  object-cover in a small card, and its hero rendered one as a full-bleed
  background when hero_url exists for exactly that. Cards now use the preview
  tier (not the thumbnail: thumbnail_fit is seeded to 'cover', so a thumbnail
  would be cropped a second time and reframe every photo) and only load once
  within 200px of the viewport, since every card mounts at page load.

GIF, APNG and PNG keep the original: generatePreviewImage encodes JPEG, which
has neither a second frame nor an alpha channel. The backend fix that removes
this list is the next commit in this stack.

Divergence from the main twin: no responsive `?w=` tiers. #1095 is main-only,
so `lightboxImageUrl` here selects a URL and nothing more. It lives in
`imageTiers.ts` under the same path main uses, so that backporting #1095 later
merges into this file rather than landing beside it.

Verified on this branch: 8 new tests; frontend suite 21 files / 113 tests,
tsc clean.

* fix(gallery): make the Story hero fix actually work on external galleries (#1166)

External review. Same two fixes as the main twin.

hero_url was inert for external media. ensureHeroImage only ever called
resolvePhotoStorageKey, which returns null for external/reference photos by
design — and that null was handed straight to withLocalCopy, which throws, so
the hero route caught it and redirected to the full ORIGINAL. #1078 fixed
exactly this shape for ensurePreviewImage and nobody carried it across. It
stayed invisible until this PR pointed the Story hero at hero_url: on a
managed gallery that is a real saving, on a reference-mode gallery it quietly
changed nothing.

Needed one extra piece here that main already had: generateHeroImage on this
branch ignores outputBasename and always derives the key from the source
basename, so two events referencing the same NAS filename would clobber each
other's hero. It now honours the option, matching generateThumbnail and
generatePreviewImage.

The format bypass trusted mime_type, which is not trustworthy: migration 039
backfilled every pre-existing photo to image/jpeg regardless of what it was,
and adminExternalMedia inserts rows with no mime_type at all — so a
mislabelled PNG sailed past the guard and came back flattened. It now checks
the filename extension as well.

* fix(previews): preserve alpha and animation in the preview tier

Stable twin of #1171. Stacked on the #1166 twin, whose format bypass this
removes.

generatePreviewImage encoded JPEG unconditionally. JPEG has no alpha channel
and no second frame, so a transparent PNG came back flattened onto a solid
background and an animated GIF came back as its first frame — for every
consumer of this tier, not just the lightbox. It was only invisible by default
because the lightbox served originals.

Sources with alpha, or more than one page, are now encoded as WebP, which
carries both and is still far smaller than the original. Ordinary photos stay
JPEG.

- the output extension matches what was written. A PNG source previously
  produced `preview_foo.png` holding JPEG bytes; harmless while the route
  hard-coded image/jpeg, wrong once the encoding varies. Existing keys keep
  working — they are still JPEG and still served as such.
- the preview route derives Content-Type from the key. With nosniff set,
  mislabelling would show a broken image rather than being silently corrected.
  The watermark branch re-encodes to JPEG and now says so.

The frontend guess-by-MIME goes away entirely, including the case it could
never get right: a still and an animated WebP declare the same type.

Divergence from the main twin: no width-tier case. The responsive `?w=`
renditions (#1095) are main-only, so this branch has a single canonical
preview per photo.

Verified on this branch: 5 new backend tests against real Sharp output;
frontend 21 files / 114 tests; full backend suite leaves the same 5
pre-existing failures as origin/stable.

* fix(previews): retire the legacy preview keys, and stop mislabelling watermarked ones

External review. Same two defects as the main twin.

Legacy keys collide with the new naming. The old generator kept the SOURCE
basename verbatim while always writing JPEG, so a `.webp` upload produced
`previews/preview_shot.webp` holding a JPEG. The claim that pre-existing keys
have no .webp suffix was simply wrong. The route now derives Content-Type from
the key and the response carries nosniff, so every photo uploaded as WebP
would have rendered as a broken image in the lightbox. Legacy `.png` keys are
wrong the other way: flattened JPEGs of what may have been transparent
sources, which isPreviewValid would have let stand forever.

Migration 178 clears photos.preview_path outright — all of it, not just the
suspicious extensions, because a `.jpg` key can equally be a flattened
rendition and nothing in the key says so. Previews regenerate lazily on next
view under the new encoder.

The watermark branch mislabelled its output. applyWatermark PRESERVES the
source format on this branch too (watermarkService.js: png stays png, webp
stays webp), and its input is the preview — so the output already matches the
key the header was derived from. Forcing image/jpeg mislabelled every
watermarked WebP preview, and nosniff means the browser would not correct it.

Numbered 178, not 176: this stack does not carry the external-media
migrations, but that stack takes 176 and 177 on this same branch, and two
files sharing a numeric prefix would be confusing even though both would run.

---------

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
2026-08-26 09:14:42 +02:00
Paul Nothaft 75facb4d67 fix(gallery): stop the lightbox loading originals to display a photo (#1166) (#1175)
* fix(gallery): stop the lightbox loading originals to display a photo (#1166)

Stable twin of #1169.

The lightbox read preview_url, which the server only emits once an admin has
flipped lightbox_preview_enabled — off by default. So a stock install fell
straight through to url, the untouched original: a reporter measured 16.5 MB
for a photo whose preview is 345 KB. The lightbox renders its neighbours too,
so opening one photo pulled three originals.

slideshow_url is the same /preview/:id URL, watermark query included, and has
been emitted unconditionally for images since #1015. Preferring it fixes every
existing install with no migration and no admin action.

Two other surfaces bypass PhotoLightbox entirely and had the same bug:

- premium galleries build their own slides with `src: photo.url`. Fixing that
  also required carrying the photo id on the slide, because the download
  handler recovered the photo by matching slide.src against photo.url — a
  derivative src would have made Download a silent no-op.
- the Story layout rendered the full original as its GRID TILE, at
  object-cover in a small card, and its hero rendered one as a full-bleed
  background when hero_url exists for exactly that. Cards now use the preview
  tier (not the thumbnail: thumbnail_fit is seeded to 'cover', so a thumbnail
  would be cropped a second time and reframe every photo) and only load once
  within 200px of the viewport, since every card mounts at page load.

GIF, APNG and PNG keep the original: generatePreviewImage encodes JPEG, which
has neither a second frame nor an alpha channel. The backend fix that removes
this list is the next commit in this stack.

Divergence from the main twin: no responsive `?w=` tiers. #1095 is main-only,
so `lightboxImageUrl` here selects a URL and nothing more. It lives in
`imageTiers.ts` under the same path main uses, so that backporting #1095 later
merges into this file rather than landing beside it.

Verified on this branch: 8 new tests; frontend suite 21 files / 113 tests,
tsc clean.

* fix(gallery): make the Story hero fix actually work on external galleries (#1166)

External review. Same two fixes as the main twin.

hero_url was inert for external media. ensureHeroImage only ever called
resolvePhotoStorageKey, which returns null for external/reference photos by
design — and that null was handed straight to withLocalCopy, which throws, so
the hero route caught it and redirected to the full ORIGINAL. #1078 fixed
exactly this shape for ensurePreviewImage and nobody carried it across. It
stayed invisible until this PR pointed the Story hero at hero_url: on a
managed gallery that is a real saving, on a reference-mode gallery it quietly
changed nothing.

Needed one extra piece here that main already had: generateHeroImage on this
branch ignores outputBasename and always derives the key from the source
basename, so two events referencing the same NAS filename would clobber each
other's hero. It now honours the option, matching generateThumbnail and
generatePreviewImage.

The format bypass trusted mime_type, which is not trustworthy: migration 039
backfilled every pre-existing photo to image/jpeg regardless of what it was,
and adminExternalMedia inserts rows with no mime_type at all — so a
mislabelled PNG sailed past the guard and came back flattened. It now checks
the filename extension as well.

* test(gallery): the hero fixture follows the root-relative relpath contract (#1166)

Same fix as the main twin: external_relpath has been resolved from
EXTERNAL_MEDIA_ROOT rather than from event.external_path since #1163 landed,
and this fixture still carried the base-relative form, so the two tests stopped
resolving the moment that stack merged. Production was never affected.

---------

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
2026-08-26 09:08:50 +02:00
Paul Nothaft 58ccecc304 fix(admin): move the maintenance sweeps' run state into the database (#1181) (#1188)
Stable twin of #1184. Both photo sweeps tracked whether they were running in a
module-level variable, which is invisible to every other replica: a status poll
routed to an idle replica reports isRunning false while another is mid-run, and
the next POST starts a second pass over the whole library.

Migration 179 adds one row per job, claimed with a conditional UPDATE whose
affected-row count is the answer. The lease is fenced on a per-claim token so a
runner superseded by a stale takeover cannot renew a claim it has lost or
release one it no longer owns; renewal runs on a timer spanning the claim
through release, since one hung NAS read can outlast the stale window inside a
single iteration. maintenance_jobs is excluded from .picpeak archives.

Gated on settings.edit / settings.view rather than main's system.manage, which
does not exist on this branch — they are what settings.edit was later split
into, so both branches let the same people through.

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
2026-08-26 09:08:14 +02:00
Paul Nothaft 7f0ed23ea4 fix(external-media): record captured_at on import and add a backfill (stable) (#1183)
* fix(external-media): record captured_at on import and add a backfill (#1172)

Stable twin of #1179.

External media never went through photoProcessor, so captured_at stayed NULL
for every externally imported photo. The gallery's "Date Taken" sort then
degraded into import order through its own COALESCE fallback — a library
imported in two batches showed the first days of a trip after the last ones.

Ported whole:
- adminExternalMedia.js reads the capture date at import, off the file it has
  already opened for the dimensions. Best-effort, like the dimensions.
- A backfill endpoint for photos imported before this, so existing installs
  can fix historical rows rather than only new imports. Managed originals go
  through resolvePhotoStorageKey + withLocalCopy so S3 installs work; archived
  events are excluded because archiving deletes their originals; the run flag
  is claimed before the candidate query so two POSTs cannot both start.
- gallery.js carries photos.id as a tiebreaker on all three sorts. A bulk
  import writes hundreds of rows inside the same second, so uploaded_at ties
  are the normal case and the grid reshuffled between page loads.

One deliberate difference from main: the backfill is gated on settings.edit /
settings.view rather than system.manage / system.view, which do not exist on
this branch. They are what settings.edit was later split into, and main's
migration 175 projects every settings.edit holder forward onto system.manage,
so both branches let exactly the same people through.

* fix(external-media): gate the status card on the permission the button needs (#1172)

The built-in admin role holds settings.view but not settings.edit
(056_add_role_permissions_table.js:63), and StatusTab renders its card and
enabled button purely on a successful status payload. Gating the status
endpoint on settings.view therefore showed every admin a Backfill button whose
every click 403s with no error surfaced.

* fix(gallery): make the Date Taken sort correct on SQLite (#1172)

Same defect as the main twin: photos.captured_at holds three storage classes on
SQLite — an epoch-millisecond integer from managed uploads (photoProcessor.js:441
hands knex a Date), ISO text from external imports and the backfill, and null
falling through to uploaded_at's 'YYYY-MM-DD HH:MM:SS' text. SQLite sorts
INTEGER before TEXT unconditionally, so a 2027 capture came back before a 2020
one, and within the text values the 'T' separator outranked the space.

Normalised in the ORDER BY; Postgres keeps the plain COALESCE, its column being
a real timestamp. Regression tests drive the real gallery route on real SQLite.

* fix(gallery): normalise epoch-integer uploaded_at too, and stop polling a 403 (#1172)

Both follow-ups from the main twin's review, ported.

uploaded_at is not always text on SQLite: a .picpeak restore can carry epoch
milliseconds in from an install that stored them that way, and the fallback
branch read it with substr(), comparing '1830297600000' against
'2020-01-01 00:00:00' as text. Both columns now get the integer/real branch.

The status card also polled every ten seconds regardless of permission. On this
branch that hits every built-in admin — they hold settings.view but not
settings.edit — so each would have had a 403 and a logged denial every ten
seconds for a panel they were never shown.

* style: quote convention in the capture-sort test (#1172)

* fix(capture-dates): skip watcher-imported videos, and make the status counts consistent (#1172)

All three follow-ups from the main twin, ported: the three-marker video filter
(fileWatcher sets type/mime but not media_type, so those rows sat in the
backlog forever), the single-aggregate status counts (two queries could report
a negative backlog mid-import), and the card's render gated on settings.edit as
well as the cached payload.

---------

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
2026-08-26 09:04:18 +02:00
Paul Nothaft 2b1c3588ae fix(external-media): store external paths from the media root (#1163) (#1174)
* fix(external-media): store external paths from the media root (#1163)

Stable twin of #1168. Stacked on the #1162 twin, which supplies
deleteDuplicatePhotos.

Importing a second folder into an event silently invalidated every photo
already in it. external_relpath was stored relative to events.external_path,
and every import overwrites that column, so the older rows were rebased onto
the new folder. Nothing errored and the grid still rendered — thumbnails are
written to local storage during the import while the base path is still
correct — so only the things that need the original broke. The reporter had
7547 of 8004 rows pointing into the void.

- external_relpath is now relative to EXTERNAL_MEDIA_ROOT, so a row is
  self-describing and nothing an admin does to the event can move it.
- migration 177 folds each event's base into its rows. Where the current
  resolution is missing it walks up for an ancestor holding a file of the same
  name AND the size the import recorded — existence alone would let a deleted
  file adopt an unrelated namesake and serve the wrong original. Rows it
  cannot place keep resolving where they resolve today, and the probe is
  skipped entirely when the mount is unreachable.
- probing is read-only and runs first; the rewrites and the marker commit
  together, so an interrupted fold cannot be folded twice.
- rewrites are staged through a per-row parking value, because a final path
  can equal another row's current one; and migration 177 re-throws without the
  driver's error code, which run-migrations-safe would otherwise read as
  "schema already exists".
- the fold also runs after a .picpeak restore, since knex_migrations is
  excluded from the archive, and a failure there is reported rather than
  presented as a clean restore.
- drops the duplicate-leaf-segment guess in photoResolver, which papered over
  this same double-prefixing.

Divergence from the main twin: no face-scan requeue reordering. Face
recognition is main-only, so the hazard of queueing rows against unconverted
paths does not exist on this branch — in picpeakImportService or in
restoreService.

Verified on this branch: 23 new tests pass, and the four suites carrying
base-relative fixtures were updated. Full suite leaves the same 5 pre-existing
failures as origin/stable, unchanged.

* fix(external-media): the fold's staging value must be storable on Postgres (#1163)

External review found this on this branch first; it was on both.

The two-pass rewrite parks each row on a temporary value, and that value was
written with a leading NUL. SQLite stores NUL in TEXT without complaint;
Postgres rejects it outright — "invalid byte sequence for encoding UTF8: 0x00"
— so migration 177 rolled back on exactly the installs that need the two-pass
repair, and only on the engine most of them run. Restores hit the same wall
and reported the conversion as failed.

The prefix is ordinary text now. Adds a gated Postgres test alongside the
existing picpeakRestorePg one, because a SQLite-only suite structurally cannot
catch this class: restoring the NUL makes exactly the two-pass repair case
fail with that error, and nothing else.

---------

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
2026-08-26 09:00:05 +02:00
Paul Nothaft e9fcf4960e fix(external-media): one row per external file per event (#1162) (#1173)
* fix(external-media): one row per external file per event (#1162)

Stable twin of #1167.

Two overlapping import-external runs against the same event inserted every
file twice. The route checked for an existing external_relpath and then
inserted, with an fs.stat and a sharp().metadata() read sitting in between — a
window wide enough for both runs to see "not there". A reporter's event held
8004 rows for 6012 distinct paths. Nothing at the storage layer stopped it:
migration 041 created only a NON-unique (event_id, source_origin) index.

- migration 176 removes the existing duplicates and adds a partial unique
  index on (event_id, external_relpath), verified against the catalog
  afterwards — a failed CREATE INDEX raises 23505 on Postgres, which
  run-migrations-safe treats as "schema already exists" and would record as
  applied on an install that never got the index.
- dependent rows are removed explicitly rather than by cascade: PicPeak never
  sets `PRAGMA foreign_keys = ON`, so on SQLite the declared CASCADE is inert
  and a bare delete strands feedback and access-log rows. Guest feedback moves
  to the survivor instead of being discarded, keyed on guest identity the way
  feedbackService defines it, and the survivor's denormalized counters are
  recomputed.
- the route treats a unique violation as a skip, so a writer this process
  cannot see converges instead of duplicating, and a second import while one
  is running gets a 409.
- a .picpeak taken before migration 176 carries exactly these duplicates, and
  suspending FK enforcement does not suspend a unique index — so the restore
  drops the index for the load and rebuilds it after running the same dedupe.

Divergences from the main twin, both because the feature is absent here:
faces (no faceProcessor, so no purgePhotoFaces reconciliation — the rows are
still deleted so nothing dangles), admin marks, transfer membership, and
photos.view_count/download_count. The service guards each on hasTable /
hasColumn, so those branches simply do not fire.

Verified on this branch: 36 new tests pass; full suite leaves the same 5
pre-existing failures as origin/stable, unchanged.

* fix(external-media): invalidate the download zip when duplicates are removed (#1162)

External review. Same fix as the main twin.

The pre-built "download everything" archive still contained the duplicate rows
the dedupe had just deleted, so guests kept receiving them. Every ordinary
photo-deletion path calls downloadZipService.invalidate for exactly this
reason.

The columns are cleared rather than the service being called: that service
carries debounce timers and a regeneration queue, which a migration should not
start. getZipInfo already treats a cleared record as a cache miss and rebuilds
on the next request. The stale object is left in storage, as elsewhere.

---------

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
2026-08-26 08:55:17 +02:00
Paul Nothaft f83d144f28 chore(stable): release 3.46.4 (#1159)
Build and Push Docker Images / build-backend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-backend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-backend (push) Blocked by required conditions
Build and Push Docker Images / build-frontend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-frontend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-frontend (push) Blocked by required conditions
Build and Push Docker Images / summary (push) Blocked by required conditions
2026-08-24 20:34:25 +02:00
Paul Nothaft b62cd2c290 fix(gallery): a guest's own hidden feedback is hidden from them too (#1150) (#1157)
Stable twin of #1153.

Everything in the system treats a hidden row as absent, but the per-viewer is_liked heart read the row without looking at is_hidden — so a like the photographer had hidden still showed as liked on a photo whose like_count was zero.

Making that agree exposes the second half: the duplicate check behind like/favorite toggling did not skip hidden rows either, so the now-empty heart, when clicked, found the hidden row and toggled it OFF. Skipping hidden rows there makes the click create a fresh, visible row.

Also carried from review: the per-guest caps, /my-feedback and getEventFeedbackSummary no longer count hidden rows, and unhiding collapses the guest's replacement — skipped when there is no stable identity, since that fallback was 'guest_identifier IS NULL', i.e. other visitors' rows.

Not carried: the my_color_label badge (colour labels are #1044) and the clearScope / singleValueScope visibility fix, neither of which exists on this branch.

Merged with admin privileges: the author cannot self-approve.
2026-08-23 22:09:41 +02:00
Paul Nothaft eaa8b41ba3 fix(gallery): guest filters respect show_feedback_to_guests (#1044) (#1156)
Stable twin of #1147, filter half only.

Every filter token on /photos is an OR of two halves: what THIS viewer marked, and what ANYONE marked. The fields built from the second half are gated on show_feedback_to_guests; the filter was not, so with the setting off ?filter=liked still returned exactly the photos other people liked — the membership instead of the count, one token at a time.

The half it left standing was also the wrong half: it read guest_identifier from the guest_id query parameter, which never matched anything, and accepting a caller-supplied identifier was a way back through the gate. Resolved from the request now, hidden rows excluded to match what the viewer can see.

Not carried: the color: token and the photo_admin_marks concurrent-write fix — colour labels and admin marks are not on this branch.

Merged with admin privileges: the author cannot self-approve.
2026-08-23 22:08:23 +02:00
Paul Nothaft d46397d92a fix(gallery): no Logout button on galleries that don't require a password (#1149) (#1154)
Stable twin of #1152. The reporter is on v3.46.1, so this branch is where the bug was actually seen.

showLogout was hard-coded true, so a gallery with no password showed a Logout button, and clicking it stranded the visitor on the loading skeleton — GalleryPage's auto-login is a one-shot latch that never re-fires.

The button is gated at both call sites, including the full-page layouts which render it on the callback rather than a flag. accessLevel and viaCustomer now come from /auth/session instead of per-tab sessionStorage, which silently downgraded a PIN-client session in a second tab. The public-gallery branch shows a reason and a Retry once auto-login has run and failed, instead of a skeleton that never stops.

Carries the full main fix including viaCustomer, even though reveal mode does not exist here, so the branches do not drift.

Merged with admin privileges: the author cannot self-approve.
2026-08-23 22:07:05 +02:00
Paul Nothaft e46260ad07 fix(scripts): regenerate-thumbnails resolves external sources through ensureThumbnail (#1148) (#1155)
Stable twin of #1151.

The script here carried the identical defect: it computed `storage/events/active/<photo.path>` and fs.access'd it, which does not exist for external or reference rows. #1129 already landed on this branch, so the route was fixed and the script was the remaining half.

Resolution goes through ensureThumbnail, which stable already exports with the external branch intact. Also carried: videos skipped on every marker, skip-vs-generate asked from isThumbnailValid, and a nonzero exit when a photo could not be built.

Not carried: the responsive-tier backfill — THUMBNAIL_WIDTHS and ensureThumbnailAtWidth are #1095/#1109 and do not exist on this branch.

Merged with admin privileges: the author cannot self-approve.
2026-08-23 22:06:30 +02:00
Paul Nothaft e9fadd2ef4 chore(stable): release 3.46.3 (#1143)
Build and Push Docker Images / build-backend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-backend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-backend (push) Blocked by required conditions
Build and Push Docker Images / build-frontend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-frontend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-frontend (push) Blocked by required conditions
Build and Push Docker Images / summary (push) Blocked by required conditions
2026-08-23 20:23:29 +02:00
Paul Nothaft d977e3e296 fix(gallery): give masonry tiles their real shape back (#1130, #1131)
Two independent causes of the same symptom — an aspect-ratio layout that does
not lay anything out.

gallery-premium discarded the tile height MasonryPhotoAlbum computed from
photos.width/height and set height:auto on both card and image, so the rendered
shape came from whatever rendition was served. With thumbnail_fit seeded 'cover'
every rendition is square, so the masonry drew identical squares.

The bundled CSS templates pinned images to a fixed pixel height, which beats the
.h-full utility six of the seven layouts rely on. Elegant Dark is seeded
is_default, so that was the out-of-the-box result for any layout other than
grid/timeline.

Migrations 052/053 corrected for fresh installs; 175 repairs the rows already
seeded. The repair is whitespace-tolerant because sanitizeCSS strips newlines
from any template ever saved through the editor, matches the height property
with a lookbehind so line-height/max-height are untouched, handles grouped
selectors and skips nested rules.

Stable twin of #1135.
2026-08-22 21:36:51 +02:00
Paul Nothaft da44f1947b fix(gallery): a missing file must not take the backend down (#1128)
LocalFsStorage.get() returns an fs.createReadStream, which is lazy: it resolves
immediately and opens the file on a later tick, so an ENOENT arrives after the
await returned and outside the route's try/catch. An unhandled 'error' event is
a process-level throw Express cannot catch — the backend exits and every gallery
goes blank until the container restarts.

gallery.js had ten .pipe(res) calls and zero error handlers.

pipeStreamToResponse attaches the missing handler: a vanished source becomes a
404 (410 for a prepared zip), anything else a 500, and a source that dies
mid-response destroys the connection rather than rewriting a status already on
the wire. Headers staged for the file are cleared first — Express does not
overwrite an existing Content-Type, and a surviving Cache-Control would let a
transient 404 be cached as a broken tile for up to an hour. It also releases the
source when a client hangs up.

Applied to all eight streaming responses, not just the thumbnail route.

Stable twin of #1133, reduced: the tier-race half does not apply here because
ensureThumbnailAtWidth does not exist on this branch.
2026-08-22 21:36:43 +02:00
Paul Nothaft dc9e3cdc5e fix(thumbnails): regenerate external photos, and stop destroying good ones (#1129)
POST /admin/thumbnails/regenerate resolved every source as
storage/events/active/<photo.path> and fs.access'd it. External and reference
rows are not there — their originals live under events.external_path — so every
one failed the check and was counted as an error, while the UI reported success
because the response is sent before the background loop starts.

It now goes through ensureThumbnail, which resolves both source kinds and writes
thumbnail_path back itself. Nulling thumbnail_path stops it short-circuiting on
isThumbnailValid, which matters because the old thumbnail is normally still
readable at exactly the moment someone presses regenerate.

And the half that destroys data: generateThumbnail deleted the target BEFORE
sharp had opened the source, and again in its catch. A source that could not be
read — a NAS mount that blipped — left the previous rendition gone and the
database pointing at it. Across a bulk regenerate that is the whole gallery.
Neither delete was needed: put stages to a temp file and renames atomically, and
put is the last statement in the try so no partial object can exist.

Also: videos filtered out, and the superseded rendition removed only when the
storage key actually moved, compared through the same canonicalisation the
backends apply.

Stable twin of #1134.
2026-08-22 21:36:19 +02:00
145 changed files with 11704 additions and 1443 deletions
+19
View File
@@ -135,3 +135,22 @@ new-layouts/
# writes event photos into backend/storage/, and the narrower
# business-docs-only rule let `git add -A` sweep them into a commit.
backend/storage/
# Python bytecode. The ML sidecar lives on main only, so this branch never
# needed the rule — which is how a `git add -A` from a shared working tree
# committed 16 .pyc files here in #1247.
__pycache__/
*.pyc
# Issue / PR screenshots belong on a `screenshots/*` branch, never on main or
# stable — that is what those branches exist for. Two landed at the repo root
# on main in #1241 and shipped as part of the source tree; nothing stopped it.
#
# Anchored with a leading slash so docs/ keeps its own images.
/issue-*.png
/issue-*.jpg
/screenshot-*.png
/screenshot-*.jpg
/*-screenshot.png
/*-screenshot.jpg
+1 -1
View File
@@ -1 +1 @@
{".":"3.46.2"}
{".":"3.46.10"}
+91
View File
@@ -5,6 +5,97 @@ All notable changes to PicPeak will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [3.46.10](https://github.com/PicPeak/picpeak/compare/v3.46.9...v3.46.10) (2026-09-07)
### Bug Fixes
* **security:** bump sanitize-html to 2.17.7 ([0f426ef](https://github.com/PicPeak/picpeak/commit/0f426ef69968b395c6e3fbd0301fe3a7759a5f44))
* **security:** bump sanitize-html to 2.17.7 (stable) ([95e3af0](https://github.com/PicPeak/picpeak/commit/95e3af080039f2d31e1cb9c85a3d93b22c80ba7c))
* **setup:** require Node 22.12 for sanitize-html ([8421b7b](https://github.com/PicPeak/picpeak/commit/8421b7b668484f87cd2bacda8fb4d95a3bc07ab5))
## [3.46.9](https://github.com/PicPeak/picpeak/compare/v3.46.8...v3.46.9) (2026-09-03)
### Bug Fixes
* **security:** batch 1 (stable) — zxcvbn DoS, revocation forgery, unlink traversals, stored Content-Type, edge middleware ([3f90221](https://github.com/PicPeak/picpeak/commit/3f90221f40b604dd5cebc016aad9478e84035b97))
* **security:** bound password input before zxcvbn, and drop the legacy media mounts ([ed08ff8](https://github.com/PicPeak/picpeak/commit/ed08ff84ff014226f1e17cc17167f80afa366f7c))
* **security:** close three middleware gaps around the API edge ([b136906](https://github.com/PicPeak/picpeak/commit/b1369068ae1327fc29e8aa671029548e2e93d827))
* **security:** contain logo, favicon and PDF-logo unlinks to their upload directories ([882101b](https://github.com/PicPeak/picpeak/commit/882101b58670e99ac3aea560b83fc4123fe4b359))
* **security:** enforce the strength-endpoint validators, and stop the generator spinning ([706d402](https://github.com/PicPeak/picpeak/commit/706d402c1e979d8419396c451487fb9be756a449))
* **security:** harden four smaller gallery and contract paths, drop the unmounted photo auth middleware ([d81cade](https://github.com/PicPeak/picpeak/commit/d81cade7cc9a39179b05ace5ae47b13bbe1d8196))
* **security:** never serve a photo under its stored MIME, and stop trusting the chunked-upload type ([a8d57f0](https://github.com/PicPeak/picpeak/commit/a8d57f0d696b9e0e92d6ae91beff9f3ad0fa1695))
* **security:** stop reflecting submitted passwords in validation errors ([6481708](https://github.com/PicPeak/picpeak/commit/6481708def49bc9cdf424752a633e320813cf280))
* **security:** stop reflecting submitted values in validation errors everywhere, cap credential lengths, close the login timing oracle ([406c638](https://github.com/PicPeak/picpeak/commit/406c6384513da2d7582dc223d799fba7cad56745))
* **security:** verify the signature before writing a token to the revocation list ([c6d4016](https://github.com/PicPeak/picpeak/commit/c6d401685f4eb4d9fd5fb70636962d73fc631cde))
### Documentation
* say the upload allow-list covers every path, video extensions must be added ([c89ce8e](https://github.com/PicPeak/picpeak/commit/c89ce8e1721adfd67598cf35ae307e97ed185827))
## [3.46.8](https://github.com/PicPeak/picpeak/compare/v3.46.7...v3.46.8) (2026-09-01)
### Bug Fixes
* **archives:** take the restored category from the manifest ([#1240](https://github.com/PicPeak/picpeak/issues/1240)) (stable) ([#1243](https://github.com/PicPeak/picpeak/issues/1243)) ([261e243](https://github.com/PicPeak/picpeak/commit/261e243070b6082ccd8b972de68d2fee15329235))
* **archives:** write a real timestamp on restored photos ([#1257](https://github.com/PicPeak/picpeak/issues/1257)) ([fed99ac](https://github.com/PicPeak/picpeak/commit/fed99ac03dfde03cf4c55fcb4b1419fea7564156))
* **auth:** treat zxcvbn suggestions as advice, not blocking errors ([#1247](https://github.com/PicPeak/picpeak/issues/1247)) ([5b69e3e](https://github.com/PicPeak/picpeak/commit/5b69e3ec4c898204c9fcde0f1d49f24edc688891))
* **auth:** treat zxcvbn suggestions as advice, not blocking errors ([#1247](https://github.com/PicPeak/picpeak/issues/1247)) ([eebca99](https://github.com/PicPeak/picpeak/commit/eebca9900b6f00b222eec16e27fa6be4fe2ce9fa))
* **events:** apply the gallery password policy to publish ([#1255](https://github.com/PicPeak/picpeak/issues/1255)) ([1d9f0b6](https://github.com/PicPeak/picpeak/commit/1d9f0b6c6491cface22f12651131fd5dfba330f0))
* **events:** delete stored objects when cascading an event delete ([#1245](https://github.com/PicPeak/picpeak/issues/1245)) ([7102687](https://github.com/PicPeak/picpeak/commit/7102687ee804140bfaca420d2eb7ec0078e50f25))
* **gallery:** route single-photo downloads through the storage backend ([#1246](https://github.com/PicPeak/picpeak/issues/1246)) ([5470fbe](https://github.com/PicPeak/picpeak/commit/5470fbe4063c3d6c0aeeb50fdb1ce6af74df1b53))
* **upload:** let Android guests reach the camera without breaking video ([#1248](https://github.com/PicPeak/picpeak/issues/1248)) ([ccc725f](https://github.com/PicPeak/picpeak/commit/ccc725f36edcf20643ab9c3a7aff16b5b93674c1))
## [3.46.7](https://github.com/PicPeak/picpeak/compare/v3.46.6...v3.46.7) (2026-08-28)
### Bug Fixes
* **admin:** the "Uncategorized" photo filter returns every photo ([#1211](https://github.com/PicPeak/picpeak/issues/1211)) ([#1215](https://github.com/PicPeak/picpeak/issues/1215)) ([15c844d](https://github.com/PicPeak/picpeak/commit/15c844db067de7bc04a88ddd407f3ed8f5df0fe2))
## [3.46.6](https://github.com/PicPeak/picpeak/compare/v3.46.5...v3.46.6) (2026-08-27)
### Bug Fixes
* **images:** fence the capture-date backfill on the file it read ([#1201](https://github.com/PicPeak/picpeak/issues/1201)) ([#1205](https://github.com/PicPeak/picpeak/issues/1205)) ([74ff236](https://github.com/PicPeak/picpeak/commit/74ff236b516b6df00e83d3c314fde552d18605ad))
## [3.46.5](https://github.com/PicPeak/picpeak/compare/v3.46.4...v3.46.5) (2026-08-26)
### Bug Fixes
* **admin:** make "Storage used" report storage used ([#1164](https://github.com/PicPeak/picpeak/issues/1164)) ([#1177](https://github.com/PicPeak/picpeak/issues/1177)) ([ac7ef26](https://github.com/PicPeak/picpeak/commit/ac7ef266dcd0d2b146c9739710c7631e672ef40f))
* **admin:** move the maintenance sweeps' run state into the database ([#1181](https://github.com/PicPeak/picpeak/issues/1181)) ([#1188](https://github.com/PicPeak/picpeak/issues/1188)) ([58ccecc](https://github.com/PicPeak/picpeak/commit/58ccecc304ff308bac3d334553ca5c7ef282eae2))
* **external-media:** one row per external file per event ([#1162](https://github.com/PicPeak/picpeak/issues/1162)) ([#1173](https://github.com/PicPeak/picpeak/issues/1173)) ([e9fcf49](https://github.com/PicPeak/picpeak/commit/e9fcf4960eb998c7e18528d773239f08e42e53bf))
* **external-media:** record captured_at on import and add a backfill (stable) ([#1183](https://github.com/PicPeak/picpeak/issues/1183)) ([7f0ed23](https://github.com/PicPeak/picpeak/commit/7f0ed23ea4c1d9379272b7267da74bc3addc1318))
* **external-media:** store external paths from the media root ([#1163](https://github.com/PicPeak/picpeak/issues/1163)) ([#1174](https://github.com/PicPeak/picpeak/issues/1174)) ([2b1c358](https://github.com/PicPeak/picpeak/commit/2b1c3588aeb26b1503445698efc6ffe4f483e645))
* **gallery:** stop the lightbox loading originals to display a photo ([#1166](https://github.com/PicPeak/picpeak/issues/1166)) ([#1175](https://github.com/PicPeak/picpeak/issues/1175)) ([75facb4](https://github.com/PicPeak/picpeak/commit/75facb4d67d026d312a99252abc7a6c420b864fb))
* **images:** respect EXIF orientation in thumbnails, heroes, previews and watermarks ([#1185](https://github.com/PicPeak/picpeak/issues/1185)) ([#1202](https://github.com/PicPeak/picpeak/issues/1202)) ([5559cd3](https://github.com/PicPeak/picpeak/commit/5559cd333d1a1f0b5ae22ad3361c13b3954ce344))
* **previews:** preserve alpha and animation in the preview tier ([#1176](https://github.com/PicPeak/picpeak/issues/1176)) ([9ffbe2f](https://github.com/PicPeak/picpeak/commit/9ffbe2f98fba53b22200081ae1b6ad2f94003345))
## [3.46.4](https://github.com/PicPeak/picpeak/compare/v3.46.3...v3.46.4) (2026-08-23)
### Bug Fixes
* **gallery:** a guest's own hidden feedback is hidden from them too ([#1150](https://github.com/PicPeak/picpeak/issues/1150)) ([#1157](https://github.com/PicPeak/picpeak/issues/1157)) ([b62cd2c](https://github.com/PicPeak/picpeak/commit/b62cd2c290d54820e8f58d11719d48592a1cd1f1))
* **gallery:** guest filters respect show_feedback_to_guests ([#1044](https://github.com/PicPeak/picpeak/issues/1044)) ([#1156](https://github.com/PicPeak/picpeak/issues/1156)) ([eaa8b41](https://github.com/PicPeak/picpeak/commit/eaa8b41ba323c7eac22e04947fead8e468e9c6c2))
* **gallery:** no Logout button on galleries that don't require a password ([#1149](https://github.com/PicPeak/picpeak/issues/1149)) ([#1154](https://github.com/PicPeak/picpeak/issues/1154)) ([d46397d](https://github.com/PicPeak/picpeak/commit/d46397d92a7648910075fb774b14abf77d893865))
* **scripts:** regenerate-thumbnails resolves external sources through ensureThumbnail ([#1148](https://github.com/PicPeak/picpeak/issues/1148)) ([#1155](https://github.com/PicPeak/picpeak/issues/1155)) ([e46260a](https://github.com/PicPeak/picpeak/commit/e46260ad0799bd411a4158c4cc31d587ba85d4ca))
## [3.46.3](https://github.com/PicPeak/picpeak/compare/v3.46.2...v3.46.3) (2026-08-22)
### Bug Fixes
* **gallery:** a missing file must not take the backend down ([#1128](https://github.com/PicPeak/picpeak/issues/1128)) ([da44f19](https://github.com/PicPeak/picpeak/commit/da44f1947b8317b47271f4f2a98b284b25d752c1))
* **gallery:** give masonry tiles their real shape back ([#1130](https://github.com/PicPeak/picpeak/issues/1130), [#1131](https://github.com/PicPeak/picpeak/issues/1131)) ([d977e3e](https://github.com/PicPeak/picpeak/commit/d977e3e296deeb19c26f1e5a98258eec323d120d))
* **thumbnails:** regenerate external photos, and stop destroying good ones ([#1129](https://github.com/PicPeak/picpeak/issues/1129)) ([dc9e3cd](https://github.com/PicPeak/picpeak/commit/dc9e3cdc5e00ac634f581e8d6b13107fe4839152))
## [3.46.2](https://github.com/PicPeak/picpeak/compare/v3.46.1...v3.46.2) (2026-08-21)
@@ -0,0 +1,498 @@
/**
* Restoring an archive must put the photos back into their categories.
*
* The archive writer already persists `category_name` per photo in
* `photos_manifest.json` — that is why the manifest exists, and the comment
* above it says so: "(and category linkage) can't be derived from the
* extracted files alone". The restore route then read only
* `original_filename` from it and kept deriving the category from the ZIP's
* first path segment.
*
* Archives store photos exactly as they sit on disk, so an event whose photos
* live in the gallery root produces a FLAT zip. `path.dirname()` is '.' for
* every entry, no category is resolved, and every restored photo lands with
* `category_id = null` — silently, with a 200 response.
*
* These pin the manifest as the source of truth, with the directory as the
* fallback that keeps foldered and legacy archives working.
*/
const fs = require('fs');
const path = require('path');
const os = require('os');
const express = require('express');
const request = require('supertest');
describe('archive restore restores categories (flat archives included)', () => {
let tmpDir; let db; let cleanup; let app; let storagePath;
beforeAll(async () => {
tmpDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'picpeak-restore-cat-'));
storagePath = path.join(tmpDir, 'storage');
process.env.NODE_ENV = 'test';
process.env.TEST_DATABASE_PATH = path.join(tmpDir, 'data', 'test.db');
process.env.STORAGE_PATH = storagePath;
await fs.promises.mkdir(path.dirname(process.env.TEST_DATABASE_PATH), { recursive: true });
await fs.promises.mkdir(path.join(storagePath, 'archives'), { recursive: true });
jest.resetModules();
jest.doMock('../../src/middleware/auth', () => ({
adminAuth: (req, _res, next) => { req.admin = { id: 1, username: 'tester' }; next(); },
}));
jest.doMock('../../src/middleware/permissions', () => ({
requirePermission: () => (_req, _res, next) => next(),
}));
jest.doMock('../../src/middleware/ownership', () => ({
requireEventOwnership: (_req, _res, next) => next(),
}));
({ db, cleanup } = await require('./helpers/crmDb').bootCrmDb());
// bootCrmDb points STORAGE_PATH at its own tmp dir; follow it rather than
// fighting it, so the archives the tests write are where the route looks.
storagePath = process.env.STORAGE_PATH;
await fs.promises.mkdir(path.join(storagePath, 'archives'), { recursive: true });
app = express();
app.use(express.json());
app.use('/admin/archives', require('../../src/routes/adminArchives'));
}, 180000);
afterAll(async () => {
if (cleanup) await cleanup();
await fs.promises.rm(tmpDir, { recursive: true, force: true }).catch(() => {});
});
beforeEach(async () => {
await db('photos').del();
await db('photo_categories').del();
await db('events').del();
});
/** A one-pixel JPEG is enough; the route only stats the extracted file. */
const PIXEL = Buffer.from(
'/9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0a'
+ 'HBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/wAALCAABAAEBAREA/8QAFAABAAAAAAAA'
+ 'AAAAAAAAAAAACf/EABQQAQAAAAAAAAAAAAAAAAAAAAD/2gAIAQEAAD8AKp//2Q==',
'base64',
);
async function writeArchive(name, entries) {
// Required lazily: the suite calls jest.resetModules() in beforeAll, and
// archiver's readable-stream copy does not survive being split across the
// two module registries.
const archiver = require('archiver');
const archivePath = path.join(storagePath, 'archives', name);
await new Promise((resolve, reject) => {
const output = fs.createWriteStream(archivePath);
const zip = archiver('zip', { zlib: { level: 0 } });
output.on('close', resolve);
zip.on('error', reject);
zip.pipe(output);
for (const [entryName, buffer] of Object.entries(entries)) {
zip.append(buffer, { name: entryName });
}
zip.finalize();
});
return path.join('archives', name);
}
async function seedArchivedEvent(archiveRelPath, slug) {
const [row] = await db('events').insert({
slug,
event_type: 'wedding',
event_name: slug,
event_date: '2026-06-27',
host_email: 'h@example.com',
admin_email: 'a@example.com',
password_hash: 'x',
share_link: `${slug}-share`,
expires_at: new Date().toISOString(),
is_archived: 1, // sqlite stores booleans as 0/1, see utils/dbCompat
archive_path: archiveRelPath,
}).returning('id');
return typeof row === 'object' ? row.id : row;
}
const categoryOf = async (filename) => {
const photo = await db('photos').where('filename', filename).first();
if (!photo || !photo.category_id) return null;
const category = await db('photo_categories').where('id', photo.category_id).first();
return category ? category.name : null;
};
it('takes the category from the manifest when the archive is flat', async () => {
// Exactly the shape a gallery-root event archives to: no directories.
const manifest = JSON.stringify([
{ filename: 'a.jpg', original_filename: 'DSC_0001.jpg', category_name: 'Polterabend' },
{ filename: 'b.jpg', original_filename: 'DSC_0002.jpg', category_name: 'Ceremony' },
]);
const archiveRelPath = await writeArchive('flat.zip', {
'a.jpg': PIXEL,
'b.jpg': PIXEL,
'photos_manifest.json': Buffer.from(manifest, 'utf8'),
});
const eventId = await seedArchivedEvent(archiveRelPath, 'flat-event');
const res = await request(app).post(`/admin/archives/${eventId}/restore`).send({});
expect(res.status).toBe(200);
// The whole bug: both of these used to be null.
expect(await categoryOf('a.jpg')).toBe('Polterabend');
expect(await categoryOf('b.jpg')).toBe('Ceremony');
});
it('stores a real timestamp on restored photos, not "[object Object]"', async () => {
// The jest+sqlite landmine: a Date handed to knex inside jest stores as
// the literal string "[object Object]". Production writes ms-numbers and
// is unaffected, so this only ever corrupts what tests read back — which
// is how it survives unnoticed.
const archiveRelPath = await writeArchive('timestamp.zip', {
'individual/STAMPED.jpg': PIXEL,
'photos_manifest.json': Buffer.from(JSON.stringify([
{ filename: 'STAMPED.jpg', original_filename: 'STAMPED.jpg', category_name: 'Ceremony' },
]), 'utf8'),
});
const eventId = await seedArchivedEvent(archiveRelPath, 'timestamp-event');
const res = await request(app).post(`/admin/archives/${eventId}/restore`).send({});
expect(res.status).toBe(200);
const photo = await db('photos').where({ event_id: eventId, filename: 'STAMPED.jpg' }).first();
expect(String(photo.uploaded_at)).not.toBe('[object Object]');
expect(Number.isNaN(new Date(photo.uploaded_at).getTime())).toBe(false);
});
it('reuses an existing category row instead of creating a duplicate', async () => {
const archiveRelPath = await writeArchive('reuse.zip', {
'c.jpg': PIXEL,
'photos_manifest.json': Buffer.from(JSON.stringify([
{ filename: 'c.jpg', original_filename: 'DSC_0003.jpg', category_name: 'Party' },
]), 'utf8'),
});
const eventId = await seedArchivedEvent(archiveRelPath, 'reuse-event');
await db('photo_categories').insert({
event_id: eventId, name: 'Party', slug: 'party', created_at: new Date(),
});
const res = await request(app).post(`/admin/archives/${eventId}/restore`).send({});
expect(res.status).toBe(200);
expect(await categoryOf('c.jpg')).toBe('Party');
const rows = await db('photo_categories').where({ event_id: eventId, name: 'Party' });
expect(rows).toHaveLength(1);
});
it('still falls back to the directory for legacy archives with no manifest', async () => {
// No manifest at all — the shape every archive had before the manifest
// landed. The directory is the only signal left, and it must keep working.
//
// `individual/` is what a REAL archive contains: entry names are the
// storage key minus `events/active/{slug}`, and that layout is
// `individual/` / `collages/`. Categories have never been directories, so
// the fallback invents a category with that name — not useful, but better
// than losing every category, and this pins what actually happens rather
// than a category-shaped folder no archive produces.
const archiveRelPath = await writeArchive('foldered.zip', {
'individual/d.jpg': PIXEL,
});
const eventId = await seedArchivedEvent(archiveRelPath, 'foldered-event');
const res = await request(app).post(`/admin/archives/${eventId}/restore`).send({});
expect(res.status).toBe(200);
expect(await categoryOf('d.jpg')).toBe('individual');
});
it('reuses a GLOBAL category instead of cloning it into the event', async () => {
// Seeded categories (Ceremony, Reception, ...) have event_id NULL. An
// event-only lookup misses them, so the restore used to create a second
// "Ceremony" — and because is_global defaults to TRUE, that duplicate then
// appeared in every other event's category list.
const [g] = await db('photo_categories').insert({
event_id: null, name: 'Ceremony', slug: 'ceremony', is_global: true, created_at: new Date(),
}).returning('id');
const globalId = typeof g === 'object' ? g.id : g;
const archiveRelPath = await writeArchive('global.zip', {
'individual/gl.jpg': PIXEL,
'photos_manifest.json': Buffer.from(JSON.stringify([
{ filename: 'gl.jpg', original_filename: 'DSC_1.jpg', category_name: 'Ceremony' },
]), 'utf8'),
});
const eventId = await seedArchivedEvent(archiveRelPath, 'global-event');
const res = await request(app).post(`/admin/archives/${eventId}/restore`).send({});
expect(res.status).toBe(200);
const photo = await db('photos').where('filename', 'gl.jpg').first();
expect(photo.category_id).toBe(globalId);
// No clone, global or otherwise.
const all = await db('photo_categories').where('name', 'Ceremony');
expect(all).toHaveLength(1);
});
it('does not create a GLOBAL category when it has to invent one', async () => {
// is_global defaults to true on this column, so an unqualified insert would
// leak a restore's category name into every gallery on the instance.
const archiveRelPath = await writeArchive('newcat.zip', {
'individual/nc.jpg': PIXEL,
'photos_manifest.json': Buffer.from(JSON.stringify([
{ filename: 'nc.jpg', original_filename: 'DSC_2.jpg', category_name: 'Polterabend' },
]), 'utf8'),
});
const eventId = await seedArchivedEvent(archiveRelPath, 'newcat-event');
const res = await request(app).post(`/admin/archives/${eventId}/restore`).send({});
expect(res.status).toBe(200);
const created = await db('photo_categories').where('name', 'Polterabend').first();
expect(created.event_id).toBe(eventId);
expect(created.is_global === false || created.is_global === 0).toBe(true);
});
it('matches the manifest when the ZIP was written with original filenames', async () => {
// With general_use_original_filenames_for_downloads on at archive time,
// archiveService names entries after the ORIGINAL filename while the
// manifest stays keyed by photos.filename. Looking up the extracted
// basename missed every entry, so categories were lost on exactly those
// archives.
const archiveRelPath = await writeArchive('original-names.zip', {
'individual/DSC_4242.jpg': PIXEL,
'photos_manifest.json': Buffer.from(JSON.stringify([
{ filename: 'stored_9f8e7d.jpg', original_filename: 'DSC_4242.jpg', category_name: 'Drohne' },
]), 'utf8'),
});
const eventId = await seedArchivedEvent(archiveRelPath, 'original-names-event');
const res = await request(app).post(`/admin/archives/${eventId}/restore`).send({});
expect(res.status).toBe(200);
expect(await categoryOf('DSC_4242.jpg')).toBe('Drohne');
});
it('prefers the event-scoped category when a global shares its name', async () => {
// The category API permits both. A single OR-lookup with .first() returned
// whichever the engine chose, so a photo could be reassigned to the global
// row and lose event-local settings such as allow_downloads.
const archiveRelPath = await writeArchive('collide.zip', {
'individual/co.jpg': PIXEL,
'photos_manifest.json': Buffer.from(JSON.stringify([
{ filename: 'co.jpg', original_filename: 'DSC_3.jpg', category_name: 'Reception' },
]), 'utf8'),
});
const eventId = await seedArchivedEvent(archiveRelPath, 'collide-event');
await db('photo_categories').insert({
event_id: null, name: 'Reception', slug: 'reception-global', is_global: true, created_at: new Date(),
});
const [own] = await db('photo_categories').insert({
event_id: eventId, name: 'Reception', slug: 'reception-own', is_global: false, created_at: new Date(),
}).returning('id');
const ownId = typeof own === 'object' ? own.id : own;
const res = await request(app).post(`/admin/archives/${eventId}/restore`).send({});
expect(res.status).toBe(200);
const photo = await db('photos').where('filename', 'co.jpg').first();
expect(photo.category_id).toBe(ownId);
});
it('matches a sanitized original filename, as the ZIP would have written it', async () => {
// archiveService runs original names through sanitizeForZipEntry() before
// writing the entry, so the emitted name differs from the manifest column.
const archiveRelPath = await writeArchive('sanitized.zip', {
'individual/od_dr_DSC_5.jpg': PIXEL,
'photos_manifest.json': Buffer.from(JSON.stringify([
{ filename: 'stored_abc.jpg', original_filename: 'od/dr/DSC_5.jpg', category_name: 'Strand' },
]), 'utf8'),
});
const eventId = await seedArchivedEvent(archiveRelPath, 'sanitized-event');
const res = await request(app).post(`/admin/archives/${eventId}/restore`).send({});
expect(res.status).toBe(200);
expect(await categoryOf('od_dr_DSC_5.jpg')).toBe('Strand');
});
it('ignores a legacy event-owned row when falling back to globals', async () => {
// The bug fixed here left rows behind on upgraded instances: event-owned
// AND is_global true, because the column defaults true. Matching on the
// flag alone would let one event's leftover be adopted by another event's
// restore, tying photos to a category that vanishes with someone else's
// gallery.
const otherEventId = await seedArchivedEvent('archives/none.zip', 'legacy-owner-event');
await db('photo_categories').insert({
event_id: otherEventId, name: 'Sunset', slug: 'sunset-legacy',
is_global: true, created_at: new Date(),
});
const archiveRelPath = await writeArchive('legacy-global.zip', {
'individual/lg.jpg': PIXEL,
'photos_manifest.json': Buffer.from(JSON.stringify([
{ filename: 'lg.jpg', original_filename: 'DSC_6.jpg', category_name: 'Sunset' },
]), 'utf8'),
});
const eventId = await seedArchivedEvent(archiveRelPath, 'legacy-global-event');
const res = await request(app).post(`/admin/archives/${eventId}/restore`).send({});
expect(res.status).toBe(200);
const photo = await db('photos').where('filename', 'lg.jpg').first();
const cat = await db('photo_categories').where('id', photo.category_id).first();
// Its own row, not the other event's leftover.
expect(cat.event_id).toBe(eventId);
});
it('drops an ambiguous original-name alias rather than guessing', async () => {
// Two photos in different ZIP folders can share an original basename;
// archiveService treats the paths as distinct and suffixes neither. Both
// would collapse onto one alias, and whichever won would hand the other
// photo someone else's category.
const archiveRelPath = await writeArchive('ambiguous.zip', {
'individual/SHARED.jpg': PIXEL,
'photos_manifest.json': Buffer.from(JSON.stringify([
{ filename: 'a_stored.jpg', original_filename: 'SHARED.jpg', category_name: 'Alpha' },
{ filename: 'b_stored.jpg', original_filename: 'SHARED.jpg', category_name: 'Beta' },
]), 'utf8'),
});
const eventId = await seedArchivedEvent(archiveRelPath, 'ambiguous-event');
const res = await request(app).post(`/admin/archives/${eventId}/restore`).send({});
expect(res.status).toBe(200);
// Falls back to the directory rather than picking Alpha or Beta at random.
expect(await categoryOf('SHARED.jpg')).toBe('individual');
for (const name of ['Alpha', 'Beta']) {
expect(await db('photo_categories').where({ event_id: eventId, name }).first()).toBeFalsy();
}
});
it('honours a manifest that says UNCATEGORIZED, instead of inventing one from the directory', async () => {
// The case the manifest-first change was for. A real archive puts every
// photo under `individual/`, so a photo the manifest records as having no
// category used to come back filed under a category called "individual" —
// the manifest being authoritative for "category X" but not for "none".
const manifest = JSON.stringify([
{ filename: 'u.jpg', original_filename: 'DSC_7000.jpg', category_name: null },
]);
const archiveRelPath = await writeArchive('uncategorized.zip', {
'individual/u.jpg': PIXEL,
'photos_manifest.json': Buffer.from(manifest, 'utf8'),
});
const eventId = await seedArchivedEvent(archiveRelPath, 'uncategorized-event');
const res = await request(app).post(`/admin/archives/${eventId}/restore`).send({});
expect(res.status).toBe(200);
expect(await categoryOf('u.jpg')).toBeNull();
// And no junk category row was created as a side effect.
const rows = await db('photo_categories').where({ event_id: eventId });
expect(rows).toHaveLength(0);
});
it('drops a canonical filename that two photos claim, rather than guessing', async () => {
// photos.filename is not unique within an event: s3AutoImporter takes
// path.basename(entry.key) and dedupes by path, so two imported files in
// different subfolders both land as IMG_1234.jpg. Both ZIP entries reduce
// to the same basename at restore, so keeping the last row seen would give
// one photo the other's category.
const archiveRelPath = await writeArchive('dup-canonical.zip', {
'individual/IMG_1234.jpg': PIXEL,
'photos_manifest.json': Buffer.from(JSON.stringify([
{ filename: 'IMG_1234.jpg', original_filename: 'a.jpg', category_name: 'Alpha' },
{ filename: 'IMG_1234.jpg', original_filename: 'b.jpg', category_name: 'Beta' },
]), 'utf8'),
});
const eventId = await seedArchivedEvent(archiveRelPath, 'dup-canonical-event');
const res = await request(app).post(`/admin/archives/${eventId}/restore`).send({});
expect(res.status).toBe(200);
expect(await categoryOf('IMG_1234.jpg')).toBe('individual');
for (const name of ['Alpha', 'Beta']) {
expect(await db('photo_categories').where({ event_id: eventId, name }).first()).toBeFalsy();
}
});
it("drops a name that one row owns canonically and another claims as an alias", async () => {
// Undecidable: with original-filename archiving ON the ZIP entry under
// this name is the ALIAS owner's file, with it OFF it is the canonical
// owner's, and the manifest does not record which mode was used. The
// point of the two-pass split is that this now resolves the same way
// every run — the archive query has no ORDER BY, so it used to be a coin
// flip between dropping the name and overwriting it.
const archiveRelPath = await writeArchive('alias-vs-canonical.zip', {
'individual/CANON.jpg': PIXEL,
'photos_manifest.json': Buffer.from(JSON.stringify([
{ filename: 'CANON.jpg', original_filename: 'unrelated.jpg', category_name: 'Canonical' },
{ filename: 'other_stored.jpg', original_filename: 'CANON.jpg', category_name: 'Aliased' },
]), 'utf8'),
});
const eventId = await seedArchivedEvent(archiveRelPath, 'alias-vs-canonical-event');
const res = await request(app).post(`/admin/archives/${eventId}/restore`).send({});
expect(res.status).toBe(200);
// Falls back to the directory rather than guessing either row.
expect(await categoryOf('CANON.jpg')).toBe('individual');
for (const name of ['Canonical', 'Aliased']) {
expect(await db('photo_categories').where({ event_id: eventId, name }).first()).toBeFalsy();
}
});
it('picks the lowest id and warns when two categories share a name', async () => {
// Allowed: two event-scoped categories with the same display name and
// different slugs. .first() used to pick either, so a re-run could move
// photos between them and inherit the wrong allow_downloads.
const archiveRelPath = await writeArchive('dupe-category.zip', {
'individual/DUPE.jpg': PIXEL,
'photos_manifest.json': Buffer.from(JSON.stringify([
{ filename: 'DUPE.jpg', original_filename: 'DUPE.jpg', category_name: 'Ceremony' },
]), 'utf8'),
});
const eventId = await seedArchivedEvent(archiveRelPath, 'dupe-category-event');
const [first] = await db('photo_categories').insert({
name: 'Ceremony', slug: 'ceremony-a', is_global: 0, event_id: eventId,
}).returning('id');
await db('photo_categories').insert({
name: 'Ceremony', slug: 'ceremony-b', is_global: 0, event_id: eventId,
});
const firstId = typeof first === 'object' ? first.id : first;
const res = await request(app).post(`/admin/archives/${eventId}/restore`).send({});
expect(res.status).toBe(200);
// Stable, not arbitrary: the same run twice lands on the same row.
const photo = await db('photos').where({ event_id: eventId, filename: 'DUPE.jpg' }).first();
expect(photo.category_id).toBe(firstId);
// And no third "Ceremony" was invented.
expect((await db('photo_categories').where({ event_id: eventId, name: 'Ceremony' })).length)
.toBe(2);
});
it('does not invent a category for a photo row that already exists', async () => {
// archiveEvent retains photo rows, so a restore can skip every insert.
// Resolving categories before that check created one from the stale
// manifest name that nothing then used — renaming a category while its
// event was archived left the old name behind as an empty duplicate.
const archiveRelPath = await writeArchive('existing-rows.zip', {
'individual/KEPT.jpg': PIXEL,
'photos_manifest.json': Buffer.from(JSON.stringify([
{ filename: 'KEPT.jpg', original_filename: 'KEPT.jpg', category_name: 'OldName' },
]), 'utf8'),
});
const eventId = await seedArchivedEvent(archiveRelPath, 'existing-rows-event');
await db('photos').insert({
event_id: eventId, filename: 'KEPT.jpg', path: 'whatever/KEPT.jpg', type: 'jpg',
uploaded_at: new Date().toISOString(),
});
const res = await request(app).post(`/admin/archives/${eventId}/restore`).send({});
expect(res.status).toBe(200);
expect(await db('photo_categories').where({ event_id: eventId, name: 'OldName' }).first())
.toBeFalsy();
});
});
@@ -0,0 +1,246 @@
/**
* POST /admin/thumbnails/regenerate for external/reference photos (#1129).
*
* STABLE TWIN. Diverges from the main version in one place: stable has no
* responsive ?w= tiers (#1095/#1109), so there is no deleteThumbnailTiers call
* to assert and the "drops the tiers first" test is absent here. Everything
* else — the external rebuild, the thumbnail_path:null contract, video
* skipping, per-event scoping and the superseded-key deletion — is identical.
*
* The route used to resolve every source as `storage/events/active/<path>` and
* `fs.access` it. External and reference rows do not live there — their
* originals sit under `events.external_path` — so every one of them failed the
* check and was counted as an error.
*
* That alone would be inert. What made it destructive is that the tier
* deletion runs FIRST (deliberately, so S3 and external rows are not skipped):
* on a reference install the button dropped every ?w= tier and rebuilt
* nothing, while the UI reported success — the response is sent before the
* background loop starts.
*
* The background work is fired with setImmediate, so every assertion here has
* to wait for it to drain rather than trusting the response.
*/
const fs = require('fs');
const path = require('path');
const os = require('os');
const express = require('express');
const request = require('supertest');
describe('admin thumbnail regeneration (#1129)', () => {
let tmpDir; let db; let cleanup; let app; let imageProcessor; let storage;
beforeAll(async () => {
tmpDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'picpeak-regen-'));
process.env.NODE_ENV = 'test';
process.env.TEST_DATABASE_PATH = path.join(tmpDir, 'data', 'test.db');
process.env.STORAGE_PATH = path.join(tmpDir, 'storage');
await fs.promises.mkdir(path.dirname(process.env.TEST_DATABASE_PATH), { recursive: true });
await fs.promises.mkdir(process.env.STORAGE_PATH, { recursive: true });
jest.resetModules();
jest.doMock('../../src/middleware/auth', () => ({
adminAuth: (req, _res, next) => { req.admin = { id: 1, username: 'tester' }; next(); },
}));
jest.doMock('../../src/middleware/permissions', () => ({
requirePermission: () => (_req, _res, next) => next(),
}));
// One instance, not a fresh object per call — the route and the
// assertions have to be looking at the same mock.
jest.doMock('../../src/services/storage', () => {
const instance = { delete: jest.fn().mockResolvedValue(undefined) };
return { getStorage: () => instance };
});
jest.doMock('../../src/services/imageProcessor', () => ({
ensureThumbnail: jest.fn().mockResolvedValue('thumbnails/thumb_ext1_shot.jpg'),
ensurePreviewImage: jest.fn().mockResolvedValue('previews/p.jpg'),
deletePreviewTiers: jest.fn().mockResolvedValue(undefined),
}));
// bootCrmDb, not run-migrations: the latter calls process.exit(0) on
// success, which ends the jest worker mid-suite.
({ db, cleanup } = await require('./helpers/crmDb').bootCrmDb());
imageProcessor = require('../../src/services/imageProcessor');
storage = require('../../src/services/storage').getStorage();
app = express();
app.use(express.json());
app.use('/admin/thumbnails', require('../../src/routes/adminThumbnails'));
}, 180000);
afterAll(async () => {
if (cleanup) await cleanup();
await fs.promises.rm(tmpDir, { recursive: true, force: true }).catch(() => {});
});
beforeEach(async () => {
jest.clearAllMocks();
await db('photos').del();
await db('events').del();
});
async function seedEvent() {
const [row] = await db('events').insert({
slug: 'nas-wedding', event_type: 'wedding', event_name: 'nas',
event_date: '2026-01-01', host_email: 'h@example.com', admin_email: 'a@example.com',
password_hash: 'x', share_link: 'nas-share', expires_at: new Date().toISOString(),
source_mode: 'reference', external_path: 'weddings/2026-08',
}).returning('id');
return typeof row === 'object' ? row.id : row;
}
async function seedPhoto(eventId, overrides = {}) {
const [row] = await db('photos').insert({
event_id: eventId, filename: 'shot.jpg', path: 'nas-wedding/shot.jpg',
type: 'individual', ...overrides,
}).returning('id');
return typeof row === 'object' ? row.id : row;
}
/** The work runs in setImmediate; give it room to finish. */
const drain = () => new Promise((resolve) => setTimeout(resolve, 150));
it('rebuilds the canonical thumbnail for an external photo instead of erroring', async () => {
const eventId = await seedEvent();
await seedPhoto(eventId, {
source_origin: 'external',
external_relpath: 'shot.jpg',
thumbnail_path: 'thumbnails/stale.jpg',
});
const res = await request(app).post('/admin/thumbnails/regenerate').send({});
expect(res.status).toBe(200);
await drain();
// The whole bug: this used to be zero calls and one logged
// "Original file not found" per photo.
expect(imageProcessor.ensureThumbnail).toHaveBeenCalledTimes(1);
});
it('nulls thumbnail_path so the valid-thumbnail short-circuit cannot skip the rebuild', async () => {
const eventId = await seedEvent();
await seedPhoto(eventId, {
source_origin: 'external',
external_relpath: 'shot.jpg',
thumbnail_path: 'thumbnails/still-on-disk.jpg',
});
await request(app).post('/admin/thumbnails/regenerate').send({});
await drain();
// Without this the endpoint is a no-op whenever the OLD thumbnail is still
// readable — which is the normal case after a settings change, and exactly
// when the admin pressed the button.
const [photoArg] = imageProcessor.ensureThumbnail.mock.calls[0];
expect(photoArg.thumbnail_path).toBeNull();
expect(photoArg.source_origin).toBe('external');
// Carried through so ensureThumbnail can resolve off the mount rather than
// under events/active.
expect(photoArg.external_relpath).toBe('shot.jpg');
});
it('leaves videos alone rather than handing a container file to Sharp', async () => {
const eventId = await seedEvent();
await seedPhoto(eventId, { source_origin: 'managed', media_type: 'video', filename: 'clip.mp4' });
await seedPhoto(eventId, { source_origin: 'managed', filename: 'still.jpg' });
const res = await request(app).post('/admin/thumbnails/regenerate').send({});
await drain();
expect(res.body.count).toBe(1);
expect(imageProcessor.ensureThumbnail).toHaveBeenCalledTimes(1);
expect(imageProcessor.ensureThumbnail.mock.calls[0][0].filename).toBe('still.jpg');
});
/**
* On S3, ensureThumbnail downloads the source to a randomly-named temp file,
* and for non-RAW input withProcessableImage passes no outputBasename — so
* generateThumbnail derives the key from that random name and it differs on
* every run. Nulling thumbnail_path hides the old key from everything that
* would otherwise clean it up, so each regeneration would strand a full
* thumbnail in the bucket, once per photo per run.
*/
describe('superseded canonical renditions', () => {
it('removes the old thumbnail when the key moved', async () => {
const eventId = await seedEvent();
await seedPhoto(eventId, {
source_origin: 'managed',
thumbnail_path: 'thumbnails/thumb_OLDRANDOM_shot.jpg',
});
imageProcessor.ensureThumbnail.mockResolvedValueOnce('thumbnails/thumb_NEWRANDOM_shot.jpg');
await request(app).post('/admin/thumbnails/regenerate').send({});
await drain();
expect(storage.delete).toHaveBeenCalledWith('thumbnails/thumb_OLDRANDOM_shot.jpg');
});
it('does NOT delete when the key is unchanged — that is the new file', async () => {
const eventId = await seedEvent();
await seedPhoto(eventId, {
source_origin: 'managed',
thumbnail_path: 'thumbnails/thumb_stable.jpg',
});
// Local storage resolves to a stable path, so the key is identical.
imageProcessor.ensureThumbnail.mockResolvedValueOnce('thumbnails/thumb_stable.jpg');
await request(app).post('/admin/thumbnails/regenerate').send({});
await drain();
expect(storage.delete).not.toHaveBeenCalled();
});
it.each([
['a Windows-style legacy path', 'thumbnails\\thumb_ext1_shot.jpg'],
['a leading ./', './thumbnails/thumb_ext1_shot.jpg'],
['a doubled separator', 'thumbnails//thumb_ext1_shot.jpg'],
])('does not delete the file it just wrote when the old path is %s', async (_name, stored) => {
const eventId = await seedEvent();
await seedPhoto(eventId, { source_origin: 'managed', thumbnail_path: stored });
// Both storage backends fold these to the same key, so this is the SAME
// object — deleting it would remove the freshly generated thumbnail and
// leave the row pointing at nothing.
imageProcessor.ensureThumbnail.mockResolvedValueOnce('thumbnails/thumb_ext1_shot.jpg');
await request(app).post('/admin/thumbnails/regenerate').send({});
await drain();
expect(storage.delete).not.toHaveBeenCalled();
});
it('counts the photo as regenerated even if the old object cannot be removed', async () => {
const eventId = await seedEvent();
await seedPhoto(eventId, {
source_origin: 'managed',
thumbnail_path: 'thumbnails/thumb_OLD.jpg',
});
imageProcessor.ensureThumbnail.mockResolvedValueOnce('thumbnails/thumb_NEW.jpg');
storage.delete.mockRejectedValueOnce(new Error('bucket said no'));
await request(app).post('/admin/thumbnails/regenerate').send({});
await drain();
// Losing the old object is untidy; the regeneration itself succeeded.
expect(imageProcessor.ensureThumbnail).toHaveBeenCalledTimes(1);
});
});
it('scopes to one event when asked', async () => {
const a = await seedEvent();
await seedPhoto(a, { source_origin: 'external', external_relpath: 'a.jpg' });
const [b] = await db('events').insert({
slug: 'other', event_type: 'wedding', event_name: 'other', event_date: '2026-01-01',
host_email: 'h@example.com', admin_email: 'a@example.com', password_hash: 'x',
share_link: 'other-share', expires_at: new Date().toISOString(),
}).returning('id');
await seedPhoto(typeof b === 'object' ? b.id : b, { source_origin: 'managed' });
const res = await request(app).post('/admin/thumbnails/regenerate').send({ eventId: a });
await drain();
expect(res.body.count).toBe(1);
expect(imageProcessor.ensureThumbnail).toHaveBeenCalledTimes(1);
});
});
@@ -0,0 +1,227 @@
/**
* Backfilling captured_at on a library imported before #1172.
*
* The point of the endpoint, rather than a migration: it resolves originals
* through resolvePhotoFilePath, which is the only path that reaches an
* external row. The thumbnail regenerator resolves under
* storage/events/active/<photo.path>, which never exists for those (#1129) —
* so it cannot be the model.
*/
const fs = require('fs');
const path = require('path');
const os = require('os');
const express = require('express');
const request = require('supertest');
const sharp = require('sharp');
describe('capture date backfill (#1172)', () => {
let tmpDir; let db; let app; let mediaRoot;
const writeJpegWithExif = async (abs, iso) => {
await fs.promises.mkdir(path.dirname(abs), { recursive: true });
const d = new Date(iso);
const pad = (n) => String(n).padStart(2, '0');
const exifDate = `${d.getUTCFullYear()}:${pad(d.getUTCMonth() + 1)}:${pad(d.getUTCDate())} `
+ `${pad(d.getUTCHours())}:${pad(d.getUTCMinutes())}:${pad(d.getUTCSeconds())}`;
await sharp({ create: { width: 60, height: 40, channels: 3, background: { r: 9, g: 9, b: 9 } } })
.withExif({ IFD2: { DateTimeOriginal: exifDate } }).jpeg().toFile(abs);
};
const settle = async () => { for (let i = 0; i < 60; i++) { await new Promise((r) => setTimeout(r, 50)); const s = await status(); if (!s.body.isRunning) return s; } throw new Error('backfill did not settle'); };
const status = () => request(app).get('/api/admin/photos/repair-capture-dates/status');
beforeAll(async () => {
tmpDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'picpeak-capfill-'));
mediaRoot = path.join(tmpDir, 'media');
await fs.promises.mkdir(mediaRoot, { recursive: true });
process.env.NODE_ENV = 'test';
process.env.TEST_DATABASE_PATH = path.join(tmpDir, 'data', 'db.sqlite');
await fs.promises.mkdir(path.dirname(process.env.TEST_DATABASE_PATH), { recursive: true });
process.env.STORAGE_PATH = path.join(tmpDir, 'storage');
process.env.EXTERNAL_MEDIA_ROOT = mediaRoot;
process.env.JWT_SECRET = process.env.JWT_SECRET || 'capfill-secret';
jest.resetModules();
jest.doMock('../../src/middleware/auth', () => ({
adminAuth: (req, _res, next) => { req.admin = { id: 1, username: 'tester', roleName: 'admin' }; next(); },
}));
jest.doMock('../../src/middleware/permissions', () => ({
requirePermission: () => (_req, _res, next) => next(),
}));
jest.doMock('../../src/utils/logger', () => ({
debug: jest.fn(), info: jest.fn(), warn: jest.fn(), error: jest.fn(),
}));
({ db } = await require('./helpers/crmDb').bootCrmDb());
app = express();
app.use(express.json());
app.use('/api/admin/photos', require('../../src/routes/adminPhotoDimensions'));
}, 180000);
afterAll(async () => {
if (db) await db.destroy?.();
await fs.promises.rm(tmpDir, { recursive: true, force: true }).catch(() => {});
});
async function seed({ relpath, exifIso, writeFile = true, archived = false }) {
await db('photos').del();
await db('events').del();
const [e] = await db('events').insert({
slug: 'capfill', event_type: 'wedding', event_name: 'capfill', event_date: '2026-01-01',
host_email: 'h@example.com', admin_email: 'a@example.com', password_hash: 'x',
share_link: `capfill-${Math.random()}`, expires_at: new Date().toISOString(),
source_mode: 'reference', external_path: 'trip', is_archived: archived,
}).returning('id');
const eventId = typeof e === 'object' ? e.id : e;
if (writeFile) await writeJpegWithExif(path.join(mediaRoot, 'trip', relpath), exifIso);
const [p] = await db('photos').insert({
event_id: eventId, filename: path.basename(relpath), path: `capfill/${path.basename(relpath)}`,
// Root-relative, as this branch stores it (#1163) — the file lives at
// <mediaRoot>/trip/<relpath>.
type: 'individual', source_origin: 'external', external_relpath: `trip/${relpath}`,
uploaded_at: new Date().toISOString(), captured_at: null,
}).returning('id');
return { eventId, photoId: typeof p === 'object' ? p.id : p };
}
it('fills captured_at for an external photo the thumbnail regenerator cannot reach', async () => {
const { photoId } = await seed({ relpath: 'a.jpg', exifIso: '2026-06-01T09:45:03Z' });
const res = await request(app).post('/api/admin/photos/repair-capture-dates');
expect(res.status).toBe(200);
expect(res.body.count).toBe(1);
const done = await settle();
expect(done.body.lastResult.success).toBe(1);
expect((await db('photos').where({ id: photoId }).first()).captured_at).toBeTruthy();
});
it('counts a photo with no EXIF separately from a failure', async () => {
// "The mount is broken" and "these files carry no date" need different
// answers from an operator, so they are not the same number.
await db('photos').del(); await db('events').del();
const { photoId } = await seed({ relpath: 'plain.jpg', exifIso: '2026-06-01T09:45:03Z', writeFile: false });
await sharp({ create: { width: 40, height: 30, channels: 3, background: { r: 1, g: 1, b: 1 } } })
.jpeg().toFile(path.join(mediaRoot, 'trip', 'plain.jpg'));
await request(app).post('/api/admin/photos/repair-capture-dates');
const done = await settle();
expect(done.body.lastResult).toMatchObject({ success: 0, noExif: 1, failed: 0 });
expect((await db('photos').where({ id: photoId }).first()).captured_at).toBeNull();
});
it('counts an unreachable original as a failure, not as missing EXIF', async () => {
await seed({ relpath: 'gone.jpg', exifIso: '2026-06-01T09:45:03Z', writeFile: false });
await request(app).post('/api/admin/photos/repair-capture-dates');
const done = await settle();
expect(done.body.lastResult).toMatchObject({ success: 0, noExif: 0, failed: 1 });
});
it('reports nothing to do once every photo has a date', async () => {
const { photoId } = await seed({ relpath: 'b.jpg', exifIso: '2026-06-02T09:00:00Z' });
await db('photos').where({ id: photoId }).update({ captured_at: new Date().toISOString() });
const res = await request(app).post('/api/admin/photos/repair-capture-dates');
expect(res.body.count).toBe(0);
expect((await status()).body.withoutCaptureDate).toBe(0);
});
it('skips a watcher-imported video, which carries media_type "image"', async () => {
// fileWatcher.processNewPhoto sets type='video' and a video/* mime but
// never media_type (fileWatcher.js:128-130), so the row keeps the 'image'
// default from migration 048. Filtering on media_type alone queued it every
// run: extractCaptureDate returns null for a video, captured_at stays null,
// and the backlog never cleared.
const { eventId } = await seed({ relpath: 'clip.jpg', exifIso: '2026-06-01T09:45:03Z', writeFile: false });
await db('photos').del();
await db('photos').insert({
event_id: eventId, filename: 'clip.mp4', path: 'capfill/clip.mp4',
type: 'video', media_type: 'image', mime_type: 'video/mp4',
source_origin: 'external', external_relpath: 'trip/clip.mp4',
uploaded_at: new Date().toISOString(), captured_at: null,
});
const res = await request(app).post('/api/admin/photos/repair-capture-dates');
expect(res.body.count).toBe(0);
const s = await status();
// And it is not counted as a permanent backlog either.
expect(s.body.total).toBe(0);
expect(s.body.withoutCaptureDate).toBe(0);
});
it('never reports more dated photos than it has photos', async () => {
// Both counts come from one aggregate; as two queries an import committing
// between them produced withCaptureDate > total and a negative backlog.
const { photoId } = await seed({ relpath: 'counted.jpg', exifIso: '2026-06-05T08:00:00Z' });
await db('photos').where({ id: photoId }).update({ captured_at: new Date().toISOString() });
const s = await status();
expect(s.body.total).toBe(1);
expect(s.body.withCaptureDate).toBe(1);
expect(s.body.withoutCaptureDate).toBe(0);
expect(s.body.withoutCaptureDate).toBeGreaterThanOrEqual(0);
});
it('skips archived events instead of failing them on every run', async () => {
// Archiving deletes the originals and keeps the rows, so an archived photo
// can never get a date. Counting it would fail it every pass and leave the
// status endpoint permanently reporting a backlog.
await seed({ relpath: 'archived.jpg', exifIso: '2026-06-04T09:00:00Z', archived: true });
const res = await request(app).post('/api/admin/photos/repair-capture-dates');
expect(res.body.count).toBe(0);
const s = await status();
expect(s.body.total).toBe(0);
expect(s.body.withoutCaptureDate).toBe(0);
expect(s.body.isRunning).toBe(false);
});
it('does not overwrite a date written while it was running', async () => {
// whereNull on the update: an import or a replacement finishing mid-run has
// already written a better value than this pass would.
const { photoId } = await seed({ relpath: 'c.jpg', exifIso: '2026-06-03T09:00:00Z' });
const claimed = '2020-01-01T00:00:00.000Z';
const res = await request(app).post('/api/admin/photos/repair-capture-dates');
expect(res.body.count).toBe(1);
await db('photos').where({ id: photoId }).update({ captured_at: claimed });
const done = await settle();
expect(new Date((await db('photos').where({ id: photoId }).first()).captured_at).toISOString()).toBe(claimed);
expect(done.body.lastResult.success).toBe(0);
// Read but not written, so it is accounted for rather than dropped.
expect(done.body.lastResult.skipped).toBe(1);
});
it('does not date a row whose file was replaced while it was reading (#1201)', async () => {
// replacePhoto swaps a NEW file under an existing row and rewrites
// path/filename (reachable from replace_by_name). The replacement carries
// no date of its own, so captured_at is still NULL and the whereNull guard
// alone would let the previous file's EXIF date land on it. The write is
// fenced on the identity that was read, so the row is skipped instead —
// and not counted as updated either.
const { photoId } = await seed({ relpath: 'orig.jpg', exifIso: '2026-06-03T09:00:00Z' });
const res = await request(app).post('/api/admin/photos/repair-capture-dates');
expect(res.body.count).toBe(1);
// Simulate the replacement landing before the loop writes.
await db('photos').where({ id: photoId })
.update({ path: 'capfill/replaced.jpg', filename: 'replaced.jpg' });
const done = await settle();
expect((await db('photos').where({ id: photoId }).first()).captured_at).toBeNull();
expect(done.body.lastResult.success).toBe(0);
// Not an error and not "no EXIF" — the date was found, another writer just
// got there first. It stays in the backlog for the next run.
expect(done.body.lastResult).toMatchObject({ noExif: 0, failed: 0, skipped: 1 });
});
});
@@ -0,0 +1,161 @@
/**
* External imports must record captured_at (#1172).
*
* Managed uploads get it from photoProcessor, which external media never goes
* through — so every externally imported photo carried captured_at NULL, and
* the gallery's "Date Taken" sort fell back to uploaded_at through its
* COALESCE. On a library imported in two batches that ordered a 12-day trip by
* which folder was imported first: the reporter's first two days landed at
* positions 4204-5296 of 5555.
*
* Driven through the real route against real files carrying real EXIF, because
* the whole question is whether the import reads the file it already has open.
*/
const fs = require('fs');
const path = require('path');
const os = require('os');
const express = require('express');
const request = require('supertest');
const sharp = require('sharp');
describe('external import capture dates (#1172)', () => {
let tmpDir; let db; let app; let mediaRoot;
/**
* A real JPEG carrying DateTimeOriginal.
*
* IFD2, not IFD0 — DateTimeOriginal lives in the Exif IFD, and exifr does not
* see it anywhere else (IFD0 takes plain DateTime, which surfaces as
* ModifyDate instead).
*/
const writeJpegWithExif = async (rel, iso) => {
const full = path.join(mediaRoot, rel);
await fs.promises.mkdir(path.dirname(full), { recursive: true });
const d = new Date(iso);
const pad = (n) => String(n).padStart(2, '0');
const exifDate = `${d.getUTCFullYear()}:${pad(d.getUTCMonth() + 1)}:${pad(d.getUTCDate())} `
+ `${pad(d.getUTCHours())}:${pad(d.getUTCMinutes())}:${pad(d.getUTCSeconds())}`;
await sharp({ create: { width: 60, height: 40, channels: 3, background: { r: 10, g: 20, b: 30 } } })
.withExif({ IFD2: { DateTimeOriginal: exifDate } })
.jpeg()
.toFile(full);
return full;
};
const writeJpegNoExif = async (rel) => {
const full = path.join(mediaRoot, rel);
await fs.promises.mkdir(path.dirname(full), { recursive: true });
await sharp({ create: { width: 60, height: 40, channels: 3, background: { r: 200, g: 10, b: 10 } } })
.jpeg().toFile(full);
};
beforeAll(async () => {
tmpDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'picpeak-capdate-'));
mediaRoot = path.join(tmpDir, 'media');
await fs.promises.mkdir(mediaRoot, { recursive: true });
process.env.NODE_ENV = 'test';
process.env.TEST_DATABASE_PATH = path.join(tmpDir, 'data', 'db.sqlite');
await fs.promises.mkdir(path.dirname(process.env.TEST_DATABASE_PATH), { recursive: true });
process.env.STORAGE_PATH = path.join(tmpDir, 'storage');
process.env.EXTERNAL_MEDIA_ROOT = mediaRoot;
process.env.JWT_SECRET = process.env.JWT_SECRET || 'capdate-secret';
jest.resetModules();
jest.doMock('../../src/middleware/auth', () => ({
adminAuth: (req, _res, next) => { req.admin = { id: 1, username: 'tester', roleName: 'admin' }; next(); },
}));
jest.doMock('../../src/middleware/permissions', () => ({
requirePermission: () => (_req, _res, next) => next(),
}));
jest.doMock('../../src/middleware/ownership', () => ({
requireEventOwnership: (_req, _res, next) => next(),
}));
jest.doMock('../../src/services/imageProcessor', () => {
const actual = jest.requireActual('../../src/services/imageProcessor');
return { ...actual, generateThumbnail: jest.fn(async () => 'thumbnails/mock.jpg'), ensureThumbnail: jest.fn() };
});
jest.doMock('../../src/utils/logger', () => ({
debug: jest.fn(), info: jest.fn(), warn: jest.fn(), error: jest.fn(),
}));
({ db } = await require('./helpers/crmDb').bootCrmDb());
app = express();
app.use(express.json());
app.use('/api/admin/external-media', require('../../src/routes/adminExternalMedia'));
}, 180000);
afterAll(async () => {
if (db) await db.destroy?.();
await fs.promises.rm(tmpDir, { recursive: true, force: true }).catch(() => {});
});
async function seedEvent() {
await db('photos').del();
await db('events').del();
await fs.promises.rm(mediaRoot, { recursive: true, force: true });
await fs.promises.mkdir(mediaRoot, { recursive: true });
const [e] = await db('events').insert({
slug: `capdate-${Math.random().toString(36).slice(2, 8)}`,
event_type: 'wedding', event_name: 'capdate', event_date: '2026-01-01',
host_email: 'h@example.com', admin_email: 'a@example.com', password_hash: 'x',
share_link: `capdate-${Math.random()}`, expires_at: new Date().toISOString(),
source_mode: 'reference',
}).returning('id');
return typeof e === 'object' ? e.id : e;
}
const runImport = (eventId, external_path) => request(app)
.post(`/api/admin/external-media/events/${eventId}/import-external`)
.send({ external_path, recursive: true });
it('records the EXIF capture date on import', async () => {
const eventId = await seedEvent();
await writeJpegWithExif('trip/a.jpg', '2026-06-01T09:45:03Z');
await runImport(eventId, 'trip');
const photo = await db('photos').where({ event_id: eventId }).first();
expect(photo.captured_at).toBeTruthy();
// NOT asserted as an absolute instant. EXIF carries a naive wall-clock
// time and exifr resolves it against the HOST timezone, so the stored UTC
// value differs between a CEST developer machine and a UTC runner. What
// this fix is about is that the field is populated and orders correctly;
// that captured_at is not a true instant is a separate, pre-existing
// problem shared with managed uploads (#1172's own footnote).
expect(new Date(photo.captured_at).getUTCFullYear()).toBe(2026);
expect(new Date(photo.captured_at).getUTCMonth()).toBe(5); // June
});
it('imports a photo with no EXIF date rather than failing it', async () => {
// Plenty of sources carry none; that must stay an import, not an error.
const eventId = await seedEvent();
await writeJpegNoExif('trip/plain.jpg');
const res = await runImport(eventId, 'trip');
expect(res.body.imported).toBe(1);
const photo = await db('photos').where({ event_id: eventId }).first();
expect(photo.captured_at).toBeNull();
});
it('orders a two-batch import by capture time, not by batch', async () => {
// The reported shape: the FIRST days of the trip imported second. Sorting
// on COALESCE(captured_at, uploaded_at) put them after the last days,
// because uploaded_at is the import timestamp.
const eventId = await seedEvent();
await writeJpegWithExif('late/day12.jpg', '2026-06-12T10:00:00Z');
await runImport(eventId, 'late');
await writeJpegWithExif('early/day01.jpg', '2026-06-01T10:00:00Z');
await runImport(eventId, 'early');
const rows = await db('photos')
.where({ event_id: eventId })
.orderByRaw('COALESCE(captured_at, uploaded_at) asc')
.select('filename');
expect(rows.map((r) => r.filename)).toEqual(['day01.jpg', 'day12.jpg']);
});
});
@@ -0,0 +1,205 @@
/**
* Two overlapping external imports insert every file twice (#1162).
*
* The route checked for an existing external_relpath and then inserted, with
* an fs.stat and a `sharp().metadata()` read sitting in between. A reporter
* double-clicked a slow import of a 6012-file tree and got 8004 rows.
*
* Both halves of the fix are driven here through the real route:
*
* - the in-flight guard, which turns the second click into a 409 instead of
* a second full walk of the tree;
* - convergence when the guard cannot help (another replica, another
* process), which is the unique index from migration 186 firing and the
* loop counting a skip rather than dying or duplicating.
*
* The second is exercised by inserting a competing row from inside the mocked
* `sharp().metadata()` call — literally inside the window the bug lived in.
*/
const fs = require('fs');
const path = require('path');
const os = require('os');
const express = require('express');
const request = require('supertest');
describe('concurrent external imports (#1162)', () => {
let tmpDir; let db; let app; let mediaRoot;
// When set, the mocked sharp metadata read inserts this row first — the
// other run winning the race between our SELECT and our INSERT.
let stealDuringMetadata = null;
let thumbnailDelayMs = 0;
beforeAll(async () => {
tmpDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'picpeak-extdup-'));
mediaRoot = path.join(tmpDir, 'media');
await fs.promises.mkdir(path.join(mediaRoot, 'nas', 'individual'), { recursive: true });
for (const name of ['a.jpg', 'b.jpg', 'c.jpg']) {
await fs.promises.writeFile(path.join(mediaRoot, 'nas', 'individual', name), 'not-a-real-jpeg');
}
process.env.NODE_ENV = 'test';
process.env.TEST_DATABASE_PATH = path.join(tmpDir, 'data', 'db.sqlite');
await fs.promises.mkdir(path.dirname(process.env.TEST_DATABASE_PATH), { recursive: true });
process.env.STORAGE_PATH = path.join(tmpDir, 'storage');
process.env.EXTERNAL_MEDIA_ROOT = mediaRoot;
process.env.JWT_SECRET = process.env.JWT_SECRET || 'extdup-secret';
jest.resetModules();
jest.doMock('../../src/middleware/auth', () => ({
adminAuth: (req, _res, next) => { req.admin = { id: 1, username: 'tester', roleName: 'admin' }; next(); },
}));
jest.doMock('../../src/middleware/permissions', () => ({
requirePermission: () => (_req, _res, next) => next(),
}));
jest.doMock('../../src/middleware/ownership', () => ({
requireEventOwnership: (_req, _res, next) => next(),
}));
// The window. In production this is a real decode of a NAS-hosted file —
// hundreds of milliseconds during which the row we just proved absent can
// appear. Standing in for the other run here makes that deterministic.
jest.doMock('sharp', () => () => ({
metadata: async () => {
if (stealDuringMetadata) {
const { db: liveDb } = require('../../src/database/db');
await liveDb('photos').insert(stealDuringMetadata);
stealDuringMetadata = null;
}
return { width: 100, height: 200 };
},
}));
jest.doMock('../../src/services/imageProcessor', () => ({
generateThumbnail: jest.fn(async () => {
if (thumbnailDelayMs) await new Promise((r) => setTimeout(r, thumbnailDelayMs));
return 'thumbnails/mock.jpg';
}),
ensureThumbnail: jest.fn(),
}));
jest.doMock('../../src/utils/logger', () => ({
debug: jest.fn(), info: jest.fn(), warn: jest.fn(), error: jest.fn(),
}));
({ db } = await require('./helpers/crmDb').bootCrmDb());
app = express();
app.use(express.json());
app.use('/api/admin/external-media', require('../../src/routes/adminExternalMedia'));
}, 180000);
afterAll(async () => {
if (db) await db.destroy?.();
await fs.promises.rm(tmpDir, { recursive: true, force: true }).catch(() => {});
});
async function seedEvent() {
await db('photos').del();
await db('events').del();
stealDuringMetadata = null;
thumbnailDelayMs = 0;
const [e] = await db('events').insert({
slug: `extdup-${Math.random().toString(36).slice(2, 8)}`,
event_type: 'wedding',
event_name: 'extdup',
event_date: '2026-01-01',
host_email: 'h@example.com',
admin_email: 'a@example.com',
password_hash: 'x',
share_link: `extdup-${Math.random()}`,
expires_at: new Date().toISOString(),
source_mode: 'reference',
}).returning('id');
return typeof e === 'object' ? e.id : e;
}
const runImport = (eventId) => request(app)
.post(`/api/admin/external-media/events/${eventId}/import-external`)
.send({ external_path: 'nas', recursive: true });
async function relpathCounts(eventId) {
const rows = await db('photos').where({ event_id: eventId }).select('external_relpath');
const counts = new Map();
for (const r of rows) counts.set(r.external_relpath, (counts.get(r.external_relpath) || 0) + 1);
return counts;
}
it('rejects a second import while the first is still running', async () => {
const eventId = await seedEvent();
// Enough to keep the first request inside its loop while the second
// arrives — the "slow import looks hung, so I clicked again" case.
thumbnailDelayMs = 20;
const [first, second] = await Promise.all([runImport(eventId), runImport(eventId)]);
const statuses = [first.status, second.status].sort();
expect(statuses).toEqual([200, 409]);
const rejected = first.status === 409 ? first : second;
expect(rejected.body.error).toMatch(/already running/i);
});
it('leaves exactly one row per file after both runs', async () => {
const eventId = await seedEvent();
thumbnailDelayMs = 20;
await Promise.all([runImport(eventId), runImport(eventId)]);
const counts = await relpathCounts(eventId);
expect(counts.size).toBe(3);
expect([...counts.values()]).toEqual([1, 1, 1]);
});
it('releases the event once the import finishes, so a re-import still works', async () => {
const eventId = await seedEvent();
expect((await runImport(eventId)).status).toBe(200);
// Not 409 — the guard is per run, not a permanent lock on the event.
const second = await runImport(eventId);
expect(second.status).toBe(200);
expect(second.body.imported).toBe(0);
expect(second.body.skipped).toBe(3);
});
it('converges when another writer wins the race mid-file', async () => {
// The guard is in-process, so it cannot see a second replica. This is what
// the unique index is for: the insert bounces, and the file is counted as
// skipped rather than duplicated or lost to a 500.
const eventId = await seedEvent();
stealDuringMetadata = {
event_id: eventId,
filename: 'a.jpg',
path: 'x/a.jpg',
type: 'individual',
source_origin: 'external',
external_relpath: path.join('nas', 'individual', 'a.jpg'),
};
const res = await runImport(eventId);
expect(res.status).toBe(200);
const counts = await relpathCounts(eventId);
expect(counts.get(path.join('nas', 'individual', 'a.jpg'))).toBe(1);
// Two imported by us, one lost to the other writer and reported honestly.
expect(res.body.imported).toBe(2);
expect(res.body.skipped).toBe(1);
});
it('does not let one contended file abort the rest of the import', async () => {
const eventId = await seedEvent();
stealDuringMetadata = {
event_id: eventId,
filename: 'a.jpg',
path: 'x/a.jpg',
type: 'individual',
source_origin: 'external',
external_relpath: path.join('nas', 'individual', 'a.jpg'),
};
await runImport(eventId);
// All three files present — the contended one via the other writer's row.
expect((await relpathCounts(eventId)).size).toBe(3);
});
});
@@ -0,0 +1,175 @@
/**
* Importing a second folder must not move the photos already in the event (#1163).
*
* events.external_path is overwritten by every import, and external_relpath
* used to be stored relative to it — so a second import silently rebased every
* existing row onto the new folder. The reporter had 7547 of 8004 originals
* pointing at files that do not exist, and nothing said so: thumbnails are
* written to local storage during the import while the base path is still
* correct, so the grid carries on rendering.
*
* Driven through the real route and the real resolver, against a real
* directory tree — the failure is entirely about whether a file is where the
* app looks for it.
*/
const fs = require('fs');
const path = require('path');
const os = require('os');
const express = require('express');
const request = require('supertest');
describe('a second external import (#1163)', () => {
let tmpDir; let db; let app; let mediaRoot; let resolvePhotoFilePath;
const touch = async (rel) => {
const full = path.join(mediaRoot, rel);
await fs.promises.mkdir(path.dirname(full), { recursive: true });
await fs.promises.writeFile(full, 'not-a-real-jpeg');
};
beforeAll(async () => {
tmpDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'picpeak-ext2nd-'));
mediaRoot = path.join(tmpDir, 'media');
await fs.promises.mkdir(mediaRoot, { recursive: true });
process.env.NODE_ENV = 'test';
process.env.TEST_DATABASE_PATH = path.join(tmpDir, 'data', 'db.sqlite');
await fs.promises.mkdir(path.dirname(process.env.TEST_DATABASE_PATH), { recursive: true });
process.env.STORAGE_PATH = path.join(tmpDir, 'storage');
process.env.EXTERNAL_MEDIA_ROOT = mediaRoot;
process.env.JWT_SECRET = process.env.JWT_SECRET || 'ext2nd-secret';
jest.resetModules();
jest.doMock('../../src/middleware/auth', () => ({
adminAuth: (req, _res, next) => { req.admin = { id: 1, username: 'tester', roleName: 'admin' }; next(); },
}));
jest.doMock('../../src/middleware/permissions', () => ({
requirePermission: () => (_req, _res, next) => next(),
}));
jest.doMock('../../src/middleware/ownership', () => ({
requireEventOwnership: (_req, _res, next) => next(),
}));
jest.doMock('sharp', () => () => ({ metadata: async () => ({ width: 100, height: 200 }) }));
jest.doMock('../../src/services/imageProcessor', () => ({
generateThumbnail: jest.fn(async () => 'thumbnails/mock.jpg'),
ensureThumbnail: jest.fn(),
}));
jest.doMock('../../src/utils/logger', () => ({
debug: jest.fn(), info: jest.fn(), warn: jest.fn(), error: jest.fn(),
}));
({ db } = await require('./helpers/crmDb').bootCrmDb());
({ resolvePhotoFilePath } = require('../../src/services/photoResolver'));
app = express();
app.use(express.json());
app.use('/api/admin/external-media', require('../../src/routes/adminExternalMedia'));
}, 180000);
afterAll(async () => {
if (db) await db.destroy?.();
await fs.promises.rm(tmpDir, { recursive: true, force: true }).catch(() => {});
});
async function seedEvent() {
await db('photos').del();
await db('events').del();
await fs.promises.rm(mediaRoot, { recursive: true, force: true });
await fs.promises.mkdir(mediaRoot, { recursive: true });
const [e] = await db('events').insert({
slug: `ext2nd-${Math.random().toString(36).slice(2, 8)}`,
event_type: 'wedding',
event_name: 'ext2nd',
event_date: '2026-01-01',
host_email: 'h@example.com',
admin_email: 'a@example.com',
password_hash: 'x',
share_link: `ext2nd-${Math.random()}`,
expires_at: new Date().toISOString(),
source_mode: 'reference',
}).returning('id');
return typeof e === 'object' ? e.id : e;
}
const runImport = (eventId, external_path) => request(app)
.post(`/api/admin/external-media/events/${eventId}/import-external`)
.send({ external_path, recursive: true });
/** Where the app would go looking for this photo's original, right now. */
async function resolved(eventId, filename) {
const event = await db('events').where({ id: eventId }).first();
const photo = await db('photos').where({ event_id: eventId, filename }).first();
return resolvePhotoFilePath(event, photo);
}
it('stores paths relative to the media root, not to the imported folder', async () => {
const eventId = await seedEvent();
await touch('Trip/Leknes/old.jpg');
await runImport(eventId, 'Trip');
const photo = await db('photos').where({ event_id: eventId }).first();
expect(photo.external_relpath).toBe(path.join('Trip', 'Leknes', 'old.jpg'));
});
it('leaves the first folders originals reachable after a second import', async () => {
const eventId = await seedEvent();
await touch('Trip/Leknes/old.jpg');
await touch('Trip/Sub/new.jpg');
await runImport(eventId, 'Trip');
const before = await resolved(eventId, 'old.jpg');
await runImport(eventId, 'Trip/Sub');
const after = await resolved(eventId, 'old.jpg');
// The regression: `after` used to be <root>/Trip/Sub/Leknes/old.jpg.
expect(after).toBe(before);
expect(fs.existsSync(after)).toBe(true);
});
it('every original in the event is still on disk afterwards', async () => {
const eventId = await seedEvent();
await touch('Trip/Leknes/a.jpg');
await touch('Trip/Leknes/b.jpg');
await touch('Trip/Sub/c.jpg');
await runImport(eventId, 'Trip');
await runImport(eventId, 'Trip/Sub');
const event = await db('events').where({ id: eventId }).first();
const photos = await db('photos').where({ event_id: eventId });
expect(photos).toHaveLength(3);
for (const photo of photos) {
expect(fs.existsSync(resolvePhotoFilePath(event, photo))).toBe(true);
}
});
it('does not re-insert a file the first import already took', async () => {
// The dedupe check compares stored paths, so it has to be comparing the
// same shape the insert writes.
const eventId = await seedEvent();
await touch('Trip/Sub/c.jpg');
await runImport(eventId, 'Trip');
const second = await runImport(eventId, 'Trip/Sub');
expect(second.body.imported).toBe(0);
expect(second.body.skipped).toBe(1);
expect(await db('photos').where({ event_id: eventId }).count('* as c').first()).toEqual({ c: 1 });
});
it('resolves a subfolder that repeats its parents name', async () => {
// The old resolver stripped the relpath's first segment when it matched the
// base path's last one, which broke exactly this layout.
const eventId = await seedEvent();
await touch('Trip/Trip/x.jpg');
await runImport(eventId, 'Trip');
const event = await db('events').where({ id: eventId }).first();
const photo = await db('photos').where({ event_id: eventId }).first();
expect(resolvePhotoFilePath(event, photo)).toBe(path.join(mediaRoot, 'Trip', 'Trip', 'x.jpg'));
});
});
@@ -0,0 +1,121 @@
/**
* PostgreSQL integration test for the external-path fold (#1163).
*
* Gated the same way as picpeakRestorePg: runs only when PICPEAK_PG_TEST_URL
* points at a throwaway Postgres DB, e.g.
* PICPEAK_PG_TEST_URL="postgres://picpeak:picpeak_secure_pass_2024@127.0.0.1:7102/picpeak_fold_test" \
* npx jest __tests__/integration/externalRelpathFoldPg.test.js
*
* This exists because of a defect SQLite could not have caught. The two-pass
* rewrite parks each row on a temporary value, and that value was first written
* with a leading NUL. SQLite stores NUL in TEXT without complaint; Postgres
* rejects it outright ("invalid byte sequence for encoding UTF8"), so migration
* 187 would have rolled back on exactly the installs needing the repair — and
* only on the engine most of them run.
*
* The staging value is therefore an engine-level contract, not an
* implementation detail, and it is pinned here on the engine that constrains it.
*/
const knex = require('knex');
const fs = require('fs');
const os = require('os');
const path = require('path');
const PG_URL = process.env.PICPEAK_PG_TEST_URL;
const maybe = PG_URL ? describe : describe.skip;
maybe('external relpath fold on Postgres', () => {
let pgDb; let mediaRoot; let fold;
const touch = async (rel, bytes) => {
const full = path.join(mediaRoot, rel);
await fs.promises.mkdir(path.dirname(full), { recursive: true });
await fs.promises.writeFile(full, Buffer.alloc(bytes));
return bytes;
};
beforeAll(async () => {
mediaRoot = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'picpeak-foldpg-'));
process.env.EXTERNAL_MEDIA_ROOT = mediaRoot;
jest.resetModules();
({ foldExternalRelpaths: fold } = require('../../src/services/externalRelpathFold'));
pgDb = knex({ client: 'pg', connection: PG_URL });
}, 60000);
afterAll(async () => {
if (pgDb) await pgDb.destroy();
await fs.promises.rm(mediaRoot, { recursive: true, force: true }).catch(() => {});
delete process.env.EXTERNAL_MEDIA_ROOT;
});
beforeEach(async () => {
await pgDb.raw('DROP TABLE IF EXISTS photos, events, app_settings CASCADE');
await pgDb.schema.createTable('events', (t) => {
t.increments('id');
t.text('external_path');
});
await pgDb.schema.createTable('photos', (t) => {
t.increments('id');
t.integer('event_id');
t.text('external_relpath');
t.bigInteger('size_bytes');
t.string('source_origin').defaultTo('managed');
});
await pgDb.schema.createTable('app_settings', (t) => {
t.increments('id');
t.string('setting_key');
t.text('setting_value');
t.string('setting_type');
t.string('updated_at');
});
await fs.promises.rm(mediaRoot, { recursive: true, force: true });
await fs.promises.mkdir(mediaRoot, { recursive: true });
});
const relpaths = async () =>
(await pgDb('photos').orderBy('id').select('external_relpath')).map((r) => r.external_relpath);
it('completes the two-pass repair that a NUL staging value would abort', async () => {
// The exact shape that forces staging: `photo.jpg` repairs up to
// `Trip/photo.jpg`, while the row already holding `Trip/photo.jpg` folds
// deeper. Every final value is distinct, but a final value equals another
// row's current one, so the rewrite has to park first.
const a = await touch('Trip/photo.jpg', 11);
const b = await touch('Trip/Sub/Trip/photo.jpg', 22);
await pgDb('events').insert({ id: 1, external_path: 'Trip/Sub' });
await pgDb('photos').insert([
{ event_id: 1, external_relpath: 'photo.jpg', size_bytes: a, source_origin: 'external' },
{ event_id: 1, external_relpath: 'Trip/photo.jpg', size_bytes: b, source_origin: 'external' },
]);
await fold(pgDb);
expect(await relpaths()).toEqual(['Trip/photo.jpg', 'Trip/Sub/Trip/photo.jpg']);
});
it('leaves no staging value behind', async () => {
await touch('Trip/a.jpg', 8);
await pgDb('events').insert({ id: 1, external_path: 'Trip' });
await pgDb('photos').insert({ event_id: 1, external_relpath: 'a.jpg', size_bytes: 8, source_origin: 'external' });
await fold(pgDb);
const rows = await relpaths();
expect(rows).toEqual(['Trip/a.jpg']);
expect(rows.some((r) => r.includes('staging'))).toBe(false);
});
it('folds and marks in one transaction', async () => {
await touch('Trip/a.jpg', 8);
await pgDb('events').insert({ id: 1, external_path: 'Trip' });
await pgDb('photos').insert({ event_id: 1, external_relpath: 'a.jpg', size_bytes: 8, source_origin: 'external' });
await fold(pgDb);
// Second run is a no-op: the marker committed with the rewrites.
await fold(pgDb);
expect(await relpaths()).toEqual(['Trip/a.jpg']);
});
});
@@ -0,0 +1,216 @@
/**
* Guest filters must respect show_feedback_to_guests (#1044 follow-up).
*
* Every filter token on /photos is an OR of two halves: what THIS viewer
* marked, and what ANYONE marked. The response fields built from the second
* half — like_count, comment_count — are all gated on
* show_feedback_to_guests. The FILTER was not.
*
* So with the setting off, the numbers were hidden but `?filter=liked` still
* returned exactly the photos other people had liked: the same information as
* a set instead of a count, one token at a time. These tests pin the gate on
* every token, and pin that the viewer's own half is never gated — filtering
* by what you yourself marked is yours to do regardless.
*/
const request = require('supertest');
const express = require('express');
const cookieParser = require('cookie-parser');
const jwt = require('jsonwebtoken');
const { bootCrmDb, seedMinimal } = require('./helpers/crmDb');
process.env.JWT_SECRET = process.env.JWT_SECRET || 'filter-visibility-secret';
const SLUG = 'filter-visibility-event';
const ME = 'guest-me-identifier';
const SOMEONE_ELSE = 'guest-other-identifier';
describe('guest filters and show_feedback_to_guests (#1044)', () => {
let db;
let cleanup;
let app;
let eventId;
let mine;
let theirs;
let myGuestRowId;
const galleryToken = () => jwt.sign(
{ eventId, eventSlug: SLUG, type: 'gallery' },
process.env.JWT_SECRET,
{ expiresIn: '1h', issuer: 'picpeak-auth' }
);
const setVisibility = (visible) => db('event_feedback_settings')
.where({ event_id: eventId })
.update({ show_feedback_to_guests: visible });
// A real verified guest, which is how the viewer's own feedback is actually
// identified — NOT the `guest_id` query parameter the frontend invents.
const guestToken = () => jwt.sign(
{ type: 'guest', guestId: myGuestRowId, eventId },
process.env.JWT_SECRET,
{ expiresIn: '1h', issuer: 'picpeak-auth' }
);
const filter = async (token, { as = 'me', claimGuestId } = {}) => {
const req = request(app)
.get(`/api/gallery/${SLUG}/photos`)
.query({ filter: token, ...(claimGuestId ? { guest_id: claimGuestId } : {}) })
.set('Authorization', `Bearer ${galleryToken()}`);
if (as === 'me') req.set('x-guest-token', guestToken());
const res = await req;
expect(res.status).toBe(200);
const photos = Array.isArray(res.body) ? res.body : res.body.photos;
return (photos || []).map((p) => p.id).sort((a, b) => a - b);
};
beforeAll(async () => {
({ db, cleanup } = await bootCrmDb());
await seedMinimal(db);
const inserted = await db('events').insert({
slug: SLUG,
event_type: 'wedding',
event_name: 'Filter Visibility',
event_date: '2026-08-01',
host_email: 'host@example.com',
admin_email: 'admin@example.com',
password_hash: 'x',
share_link: `/gallery/${SLUG}/share`,
share_token: 'filter-visibility-share',
expires_at: new Date(Date.now() + 7 * 24 * 3600 * 1000).toISOString(),
is_active: 1,
is_archived: 0,
is_draft: 0,
created_at: new Date().toISOString(),
}).returning('id');
eventId = inserted[0]?.id ?? inserted[0];
const addPhoto = async (name) => {
const p = await db('photos').insert({
event_id: eventId,
filename: name,
path: `events/filter/${name}`,
type: 'individual',
uploaded_at: new Date().toISOString(),
}).returning('id');
return p[0]?.id ?? p[0];
};
mine = await addPhoto('mine.jpg');
theirs = await addPhoto('theirs.jpg');
await db('event_feedback_settings').insert({
event_id: eventId,
feedback_enabled: true,
allow_likes: true,
allow_comments: true,
allow_ratings: true,
allow_favorites: true,
moderate_comments: false,
show_feedback_to_guests: true,
});
const guestRow = await db('gallery_guests').insert({
event_id: eventId,
name: 'Me',
identifier: ME,
created_at: new Date().toISOString(),
last_seen_at: new Date().toISOString(),
is_deleted: false,
}).returning('id');
myGuestRowId = guestRow[0]?.id ?? guestRow[0];
const feedback = (photoId, who, type, extra = {}) => db('photo_feedback').insert({
photo_id: photoId,
event_id: eventId,
guest_identifier: who,
// Submission links to the per-person guest row when one is present, and
// that is the column the viewer's own half resolves through.
guest_id: who === ME ? myGuestRowId : null,
feedback_type: type,
is_approved: true,
is_hidden: false,
created_at: new Date().toISOString(),
...extra,
});
// Everything on `theirs` belongs to somebody else; `mine` is this viewer's.
await feedback(mine, ME, 'like');
await feedback(theirs, SOMEONE_ELSE, 'like');
await feedback(theirs, SOMEONE_ELSE, 'favorite');
await feedback(theirs, SOMEONE_ELSE, 'comment', { comment_text: 'lovely' });
await feedback(theirs, SOMEONE_ELSE, 'rating', { rating: 5 });
// The denormalized counters the aggregate half of the filter reads.
await db('photos').where('id', theirs).update({
like_count: 1, favorite_count: 1, comment_count: 1, average_rating: 5,
});
await db('photos').where('id', mine).update({ like_count: 1 });
app = express();
app.use(express.json());
app.use(cookieParser());
app.use('/api/gallery', require('../../src/routes/gallery'));
}, 120000);
afterAll(async () => {
if (cleanup) await cleanup();
});
describe('with feedback visible to guests', () => {
beforeAll(() => setVisibility(true));
it('shows other people\'s marks through every token, as before', async () => {
expect(await filter('liked')).toEqual([mine, theirs].sort((a, b) => a - b));
expect(await filter('favorited')).toEqual([theirs]);
expect(await filter('rated')).toEqual([theirs]);
expect(await filter('commented')).toEqual([theirs]);
});
});
describe('with feedback hidden from guests', () => {
beforeAll(() => setVisibility(false));
it('stops every token from selecting on other people\'s marks', async () => {
// `theirs` is the photo only other guests marked. It must not come back
// through any token — a filter that selects on hidden feedback reports
// that feedback just as surely as a count would.
expect(await filter('favorited')).toEqual([]);
expect(await filter('rated')).toEqual([]);
expect(await filter('commented')).toEqual([]);
});
it('still filters by what the viewer marked themselves', async () => {
// The viewer's own half is never gated: this is their own action, and
// hiding it would break "show me the ones I liked" for no privacy gain.
expect(await filter('liked')).toEqual([mine]);
});
it('drops the viewer\'s own feedback once an admin hides it', async () => {
// Moderation has to reach the filter too. getPhotoFeedback excludes
// hidden rows for the guest's OWN feedback, so a photo matching here
// would come back with nothing visible on it to explain why.
await db('photo_feedback')
.where({ photo_id: mine, guest_id: myGuestRowId, feedback_type: 'like' })
.update({ is_hidden: true });
expect(await filter('liked')).toEqual([]);
await db('photo_feedback')
.where({ photo_id: mine, guest_id: myGuestRowId, feedback_type: 'like' })
.update({ is_hidden: false });
expect(await filter('liked')).toEqual([mine]);
});
it('ignores a guest_id supplied by the caller', async () => {
// The own-half is resolved from the request identity. If it honoured the
// query string instead, anyone holding another guest's identifier could
// read that guest's hidden memberships one token at a time — straight
// back through the gate this file exists to pin.
expect(await filter('favorited', { claimGuestId: SOMEONE_ELSE })).toEqual([]);
// And an anonymous caller claiming to be me gets nothing of mine.
expect(await filter('liked', { as: 'anon', claimGuestId: ME })).toEqual([]);
});
});
});
@@ -0,0 +1,245 @@
/**
* Hidden feedback, seen from the guest who left it (#1150).
*
* Everything in the system treats a hidden row as absent: getPhotoFeedback
* drops it even for the guest's own feedback, the /photos filters drop it, and
* updatePhotoFeedbackStats does not count it. One place disagreed — the
* per-viewer `is_liked` heart — so a like the photographer had hidden still
* showed as liked on a photo whose like_count was zero. (The `my_color_label`
* badge has the same shape on main; colour labels are not on this branch.)
*
* Making those two agree exposes the second half: the duplicate check that
* powers like/favorite toggling did NOT skip hidden rows, so the now-empty
* heart, when clicked, found the hidden row and toggled it OFF. The click
* appeared to do nothing and it took two more to get back to a filled heart.
*
* Hiding a non-comment is deliberate, not an accident of the raw route: #839
* and #1044 both ship it, with tests asserting that a hidden reaction or
* colour label stops counting. So the fix is to make hidden mean absent
* consistently — not to stop admins hiding these.
*/
const request = require('supertest');
const express = require('express');
const cookieParser = require('cookie-parser');
const jwt = require('jsonwebtoken');
const { bootCrmDb, seedMinimal } = require('./helpers/crmDb');
process.env.JWT_SECRET = process.env.JWT_SECRET || 'hidden-feedback-secret';
const SLUG = 'hidden-own-feedback';
const ME = 'guest-me-identifier';
describe('a guest\'s own hidden feedback (#1150)', () => {
let db; let cleanup; let app; let feedbackService;
let eventId; let photoId; let myGuestRowId;
const galleryToken = () => jwt.sign(
{ eventId, eventSlug: SLUG, type: 'gallery' },
process.env.JWT_SECRET,
{ expiresIn: '1h', issuer: 'picpeak-auth' }
);
const guestToken = () => jwt.sign(
{ type: 'guest', guestId: myGuestRowId, eventId },
process.env.JWT_SECRET,
{ expiresIn: '1h', issuer: 'picpeak-auth' }
);
const getPhoto = async () => {
const res = await request(app)
.get(`/api/gallery/${SLUG}/photos`)
.set('Authorization', `Bearer ${galleryToken()}`)
.set('x-guest-token', guestToken());
expect(res.status).toBe(200);
const photos = Array.isArray(res.body) ? res.body : res.body.photos;
return (photos || []).find((p) => p.id === photoId);
};
beforeAll(async () => {
({ db, cleanup } = await bootCrmDb());
await seedMinimal(db);
feedbackService = require('../../src/services/feedbackService');
const [ev] = await db('events').insert({
slug: SLUG,
event_type: 'wedding',
event_name: 'Hidden Own Feedback',
event_date: '2026-08-01',
host_email: 'h@example.com',
admin_email: 'a@example.com',
password_hash: 'x',
share_link: `/gallery/${SLUG}/share`,
share_token: 'hidden-own-share',
expires_at: new Date(Date.now() + 7 * 24 * 3600 * 1000).toISOString(),
is_active: 1, is_archived: 0, is_draft: 0,
created_at: new Date().toISOString(),
}).returning('id');
eventId = typeof ev === 'object' ? ev.id : ev;
const [p] = await db('photos').insert({
event_id: eventId, filename: 'shot.jpg', path: 'events/hidden/shot.jpg',
type: 'individual', uploaded_at: new Date().toISOString(),
}).returning('id');
photoId = typeof p === 'object' ? p.id : p;
const [g] = await db('gallery_guests').insert({
event_id: eventId, name: 'Me', identifier: ME,
created_at: new Date().toISOString(), last_seen_at: new Date().toISOString(),
is_deleted: false,
}).returning('id');
myGuestRowId = typeof g === 'object' ? g.id : g;
await db('event_feedback_settings').insert({
event_id: eventId, feedback_enabled: true, allow_likes: true,
moderate_comments: false,
show_feedback_to_guests: true,
});
app = express();
app.use(express.json());
app.use(cookieParser());
app.use('/api/gallery', require('../../src/routes/gallery'));
app.use('/api/gallery', require('../../src/routes/galleryFeedback'));
}, 180000);
afterAll(async () => { if (cleanup) await cleanup(); });
const like = () => db('photo_feedback').insert({
photo_id: photoId, event_id: eventId, guest_identifier: ME,
guest_id: myGuestRowId, feedback_type: 'like',
is_approved: true, is_hidden: false, created_at: new Date().toISOString(),
});
beforeEach(async () => {
await db('photo_feedback').where({ photo_id: photoId }).del();
await db('photos').where('id', photoId).update({ like_count: 0 });
});
describe('the read surfaces agree with each other', () => {
it('un-fills the heart once the like is hidden', async () => {
await like();
await feedbackService.updatePhotoFeedbackStats(photoId);
expect((await getPhoto()).is_liked).toBe(true);
await db('photo_feedback')
.where({ photo_id: photoId, feedback_type: 'like' })
.update({ is_hidden: true });
await feedbackService.updatePhotoFeedbackStats(photoId);
const photo = await getPhoto();
// like_count already ignored hidden rows, so the heart was the only
// thing still claiming this photo was liked.
expect(photo.like_count).toBe(0);
expect(photo.is_liked).toBe(false);
});
});
describe('and every other surface agrees', () => {
it('keeps a hidden like out of /my-feedback', async () => {
await like();
await db('photo_feedback')
.where({ photo_id: photoId, feedback_type: 'like' })
.update({ is_hidden: true });
const res = await request(app)
.get(`/api/gallery/${SLUG}/my-feedback`)
.set('Authorization', `Bearer ${galleryToken()}`)
.set('x-guest-token', guestToken());
expect(res.status).toBe(200);
// In guest identity mode the Liked/Favorited/Rated chips and their
// filters are built from THIS array, not from is_liked — so a hidden
// like left an empty heart while the chip still counted it.
expect(res.body.filter((f) => f.feedback_type === 'like')).toHaveLength(0);
});
it('does not count a hidden row against the guest cap', async () => {
await db('event_feedback_settings')
.where({ event_id: eventId }).update({ max_likes_per_guest: 1 });
await like();
await db('photo_feedback')
.where({ photo_id: photoId, feedback_type: 'like' })
.update({ is_hidden: true });
// The hidden row is room, not an occupant: the guest sees an empty
// heart, and meeting that click with limit_reached leaves the control
// dead until they un-like something they can still see.
const result = await feedbackService.submitFeedback(photoId, eventId, {
feedback_type: 'like', guest_identifier: ME, guest_id: myGuestRowId,
});
expect(result.limit_reached).toBeUndefined();
await db('event_feedback_settings')
.where({ event_id: eventId }).update({ max_likes_per_guest: null });
});
it('leaves other anonymous rows alone when there is no identity to scope by', async () => {
// With neither guest_id nor guest_identifier the collapse scope degrades
// to `guest_identifier IS NULL` — every identifier-less row on the
// photo, i.e. other people's.
const anon = (extra) => ({
photo_id: photoId, event_id: eventId, feedback_type: 'like',
is_approved: true, created_at: new Date().toISOString(), ...extra,
});
const [h] = await db('photo_feedback').insert(anon({ is_hidden: true })).returning('id');
const hiddenId = typeof h === 'object' ? h.id : h;
await db('photo_feedback').insert(anon({ is_hidden: false }));
await db('photo_feedback').insert(anon({ is_hidden: false }));
await feedbackService.moderateFeedback(hiddenId, 'approve', 1);
expect(await db('photo_feedback')
.where({ photo_id: photoId, feedback_type: 'like', is_hidden: false }))
.toHaveLength(3);
});
it('collapses the replacement when an admin unhides the original', async () => {
await like();
const original = await db('photo_feedback').where({ photo_id: photoId }).first();
await db('photo_feedback').where('id', original.id).update({ is_hidden: true });
await feedbackService.submitFeedback(photoId, eventId, {
feedback_type: 'like', guest_identifier: ME, guest_id: myGuestRowId,
});
expect(await db('photo_feedback').where({ photo_id: photoId })).toHaveLength(2);
await feedbackService.moderateFeedback(original.id, 'approve', 1);
// Two visible rows for one guest would double-count in the tallies and
// need two toggles to clear, since each deletes a single row.
const visible = await db('photo_feedback')
.where({ photo_id: photoId, feedback_type: 'like', is_hidden: false });
expect(visible).toHaveLength(1);
expect(visible[0].id).toBe(original.id);
});
});
describe('and clicking still works afterwards', () => {
it('re-liking creates a fresh row instead of toggling the hidden one off', async () => {
await like();
await db('photo_feedback')
.where({ photo_id: photoId, feedback_type: 'like' })
.update({ is_hidden: true });
// What the guest sees is an empty heart, so this is an ADD.
const result = await feedbackService.submitFeedback(photoId, eventId, {
feedback_type: 'like',
guest_identifier: ME,
guest_id: myGuestRowId,
});
// Before this, the duplicate check found the hidden row and deleted it —
// `removed: true` — so the click did nothing visible and the moderation
// was silently undone.
expect(result.removed).toBeUndefined();
const visible = await db('photo_feedback')
.where({ photo_id: photoId, feedback_type: 'like', is_hidden: false });
expect(visible).toHaveLength(1);
expect((await getPhoto()).is_liked).toBe(true);
});
});
});
@@ -0,0 +1,233 @@
/**
* Shared run state for the maintenance sweeps (#1181).
*
* The behaviour that matters here cannot be observed from one process holding
* a module-level flag, which is exactly why the flag moved into the database.
* A second replica is simulated the only way that is honest in a single-process
* test: by asserting on the shared row itself, and by driving claim() twice —
* a second caller getting null is precisely what a second replica gets.
*/
const fs = require('fs');
const path = require('path');
const os = require('os');
const express = require('express');
const request = require('supertest');
describe('maintenance job state (#1181)', () => {
let tmpDir; let db; let app; let jobs;
const dimStatus = () => request(app).get('/api/admin/photos/repair-dimensions/status');
const capStatus = () => request(app).get('/api/admin/photos/repair-capture-dates/status');
beforeAll(async () => {
tmpDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'picpeak-mjs-'));
process.env.NODE_ENV = 'test';
process.env.TEST_DATABASE_PATH = path.join(tmpDir, 'data', 'db.sqlite');
await fs.promises.mkdir(path.dirname(process.env.TEST_DATABASE_PATH), { recursive: true });
process.env.STORAGE_PATH = path.join(tmpDir, 'storage');
process.env.JWT_SECRET = process.env.JWT_SECRET || 'mjs-secret';
jest.resetModules();
jest.doMock('../../src/middleware/auth', () => ({
adminAuth: (req, _res, next) => { req.admin = { id: 1, username: 'tester', roleName: 'admin' }; next(); },
}));
jest.doMock('../../src/middleware/permissions', () => ({
requirePermission: () => (_req, _res, next) => next(),
}));
jest.doMock('../../src/utils/logger', () => ({
debug: jest.fn(), info: jest.fn(), warn: jest.fn(), error: jest.fn(),
}));
({ db } = await require('./helpers/crmDb').bootCrmDb());
jobs = require('../../src/services/maintenanceJobState');
app = express();
app.use(express.json());
app.use('/api/admin/photos', require('../../src/routes/adminPhotoDimensions'));
}, 180000);
afterAll(async () => {
if (db) await db.destroy?.();
await fs.promises.rm(tmpDir, { recursive: true, force: true }).catch(() => {});
});
beforeEach(async () => {
await db('maintenance_jobs').update({
is_running: false, started_at: null, heartbeat_at: null, finished_at: null, last_result: null, owner: null, claim_token: null,
});
});
test('the lease table is kept out of .picpeak archives', () => {
// It is live state, not data. An archive taken mid-sweep would otherwise
// carry is_running = true and a claim token owned by a process on the
// SOURCE install; restored inside the staleness window, the target reports
// the job as running and refuses new POSTs with no runner to release it.
// The importer filters on this same set, so archives written before the
// exclusion are skipped on restore too.
const { EXCLUDED_TABLES } = require('../../src/services/picpeakExportService');
expect(EXCLUDED_TABLES.has('maintenance_jobs')).toBe(true);
});
test('the migration seeds a row for each job', async () => {
const names = await db('maintenance_jobs').pluck('job_name');
expect(names.sort()).toEqual(['photo_capture_date_backfill', 'photo_dimension_repair']);
});
test('a second claim is refused while the first is alive', async () => {
expect(await jobs.claim(jobs.JOB_DIMENSION_REPAIR)).toEqual(expect.any(String));
// What a second replica's POST does. Nothing about the first claim lives in
// this process, so this is the same question the other replica asks.
expect(await jobs.claim(jobs.JOB_DIMENSION_REPAIR)).toBeNull();
});
test('the two jobs claim independently', async () => {
expect(await jobs.claim(jobs.JOB_DIMENSION_REPAIR)).toEqual(expect.any(String));
expect(await jobs.claim(jobs.JOB_CAPTURE_DATE_BACKFILL)).toEqual(expect.any(String));
});
test('each claim gets a distinct token', async () => {
const first = await jobs.claim(jobs.JOB_DIMENSION_REPAIR);
await jobs.release(jobs.JOB_DIMENSION_REPAIR, first);
const second = await jobs.claim(jobs.JOB_DIMENSION_REPAIR);
// Same process, same pid — so an owner string would have collided here and
// the fencing below would be worthless.
expect(second).not.toBe(first);
});
test('a claim whose heartbeat has gone quiet can be taken over', async () => {
expect(await jobs.claim(jobs.JOB_DIMENSION_REPAIR)).toEqual(expect.any(String));
expect(await jobs.claim(jobs.JOB_DIMENSION_REPAIR)).toBeNull();
// The replica holding it was killed: no release, no further heartbeats.
const longAgo = new Date(Date.now() - jobs.DEFAULT_STALE_MS - 60000).toISOString();
await db('maintenance_jobs').where({ job_name: jobs.JOB_DIMENSION_REPAIR }).update({ heartbeat_at: longAgo });
expect(await jobs.claim(jobs.JOB_DIMENSION_REPAIR)).toEqual(expect.any(String));
});
test('a superseded runner cannot renew its lease', async () => {
const oldToken = await jobs.claim(jobs.JOB_DIMENSION_REPAIR);
const longAgo = new Date(Date.now() - jobs.DEFAULT_STALE_MS - 60000).toISOString();
await db('maintenance_jobs').where({ job_name: jobs.JOB_DIMENSION_REPAIR }).update({ heartbeat_at: longAgo });
const newToken = await jobs.claim(jobs.JOB_DIMENSION_REPAIR);
expect(newToken).toEqual(expect.any(String));
// The old runner is still alive and mid-loop. Its renewal must tell it so,
// which is what makes the route loop stop instead of running alongside the
// new owner.
expect(await jobs.heartbeat(jobs.JOB_DIMENSION_REPAIR, oldToken)).toBe(false);
expect(await jobs.heartbeat(jobs.JOB_DIMENSION_REPAIR, newToken)).toBe(true);
});
test('a superseded runner cannot release the new owner\'s claim', async () => {
const oldToken = await jobs.claim(jobs.JOB_CAPTURE_DATE_BACKFILL);
const longAgo = new Date(Date.now() - jobs.DEFAULT_STALE_MS - 60000).toISOString();
await db('maintenance_jobs').where({ job_name: jobs.JOB_CAPTURE_DATE_BACKFILL }).update({ heartbeat_at: longAgo });
const newToken = await jobs.claim(jobs.JOB_CAPTURE_DATE_BACKFILL);
// The old runner finishes late and tries to write its result. Unfenced,
// this cleared is_running under the new owner and let a THIRD sweep start.
expect(await jobs.release(jobs.JOB_CAPTURE_DATE_BACKFILL, oldToken, { success: 999, noExif: 0, failed: 0 })).toBe(false);
const state = await jobs.read(jobs.JOB_CAPTURE_DATE_BACKFILL);
expect(state.isRunning).toBe(true);
expect(state.lastResult).toBeNull();
// And the row is still the new owner's to release.
expect(await jobs.release(jobs.JOB_CAPTURE_DATE_BACKFILL, newToken, { success: 1, noExif: 0, failed: 0 })).toBe(true);
});
test('a stale run reads as not running, so the button comes back', async () => {
await jobs.claim(jobs.JOB_DIMENSION_REPAIR);
expect((await jobs.read(jobs.JOB_DIMENSION_REPAIR)).isRunning).toBe(true);
const longAgo = new Date(Date.now() - jobs.DEFAULT_STALE_MS - 60000).toISOString();
await db('maintenance_jobs').where({ job_name: jobs.JOB_DIMENSION_REPAIR }).update({ heartbeat_at: longAgo });
// is_running is still true in the row — nothing released it — but a status
// poll must not leave the operator staring at a job that cannot finish.
expect((await db('maintenance_jobs').where({ job_name: jobs.JOB_DIMENSION_REPAIR }).first()).is_running).toBeTruthy();
expect((await jobs.read(jobs.JOB_DIMENSION_REPAIR)).isRunning).toBe(false);
});
test('a heartbeat keeps a long run claimed', async () => {
const token = await jobs.claim(jobs.JOB_DIMENSION_REPAIR);
const longAgo = new Date(Date.now() - jobs.DEFAULT_STALE_MS - 60000).toISOString();
await db('maintenance_jobs').where({ job_name: jobs.JOB_DIMENSION_REPAIR }).update({ heartbeat_at: longAgo });
expect(await jobs.heartbeat(jobs.JOB_DIMENSION_REPAIR, token)).toBe(true);
expect(await jobs.claim(jobs.JOB_DIMENSION_REPAIR)).toBeNull();
expect((await jobs.read(jobs.JOB_DIMENSION_REPAIR)).isRunning).toBe(true);
});
test('release stores the result and read gives it back parsed', async () => {
const token = await jobs.claim(jobs.JOB_CAPTURE_DATE_BACKFILL);
await jobs.release(jobs.JOB_CAPTURE_DATE_BACKFILL, token, { success: 3, noExif: 2, failed: 1 });
const state = await jobs.read(jobs.JOB_CAPTURE_DATE_BACKFILL);
expect(state.isRunning).toBe(false);
expect(state.lastResult).toEqual({ success: 3, noExif: 2, failed: 1 });
});
test('releasing without a result keeps the previous run visible', async () => {
const first = await jobs.claim(jobs.JOB_CAPTURE_DATE_BACKFILL);
await jobs.release(jobs.JOB_CAPTURE_DATE_BACKFILL, first, { success: 7, noExif: 0, failed: 0 });
// The "nothing to do" path: claimed, found no candidates, released. It must
// not blank the numbers the last real run reported.
const second = await jobs.claim(jobs.JOB_CAPTURE_DATE_BACKFILL);
await jobs.release(jobs.JOB_CAPTURE_DATE_BACKFILL, second);
expect((await jobs.read(jobs.JOB_CAPTURE_DATE_BACKFILL)).lastResult).toEqual({ success: 7, noExif: 0, failed: 0 });
});
test('a malformed result does not take the status endpoint down', async () => {
await db('maintenance_jobs').where({ job_name: jobs.JOB_DIMENSION_REPAIR }).update({ last_result: 'not json' });
const state = await jobs.read(jobs.JOB_DIMENSION_REPAIR);
expect(state.lastResult).toBeNull();
expect(state.isRunning).toBe(false);
});
test('both status endpoints report the shared row, not process memory', async () => {
await jobs.claim(jobs.JOB_DIMENSION_REPAIR);
const capToken = await jobs.claim(jobs.JOB_CAPTURE_DATE_BACKFILL);
await jobs.release(jobs.JOB_CAPTURE_DATE_BACKFILL, capToken, { success: 1, noExif: 0, failed: 0 });
// Written straight to the row, exactly as another replica would have.
const dim = await dimStatus();
expect(dim.status).toBe(200);
expect(dim.body.isRunning).toBe(true);
const cap = await capStatus();
expect(cap.status).toBe(200);
expect(cap.body.isRunning).toBe(false);
expect(cap.body.lastResult).toEqual({ success: 1, noExif: 0, failed: 0 });
});
test('a POST is refused while another replica holds the claim', async () => {
// The claim was taken by "another replica" — this process knows nothing
// about it beyond the row.
await jobs.claim(jobs.JOB_CAPTURE_DATE_BACKFILL);
const res = await request(app).post('/api/admin/photos/repair-capture-dates');
expect(res.status).toBe(409);
const dimRes = await request(app).post('/api/admin/photos/repair-dimensions');
// The other job is untouched by that claim, so it is free to start.
expect(dimRes.status).toBe(200);
});
test('the no-op path releases the claim it took', async () => {
// No photos at all, so both endpoints take their "nothing to do" exit.
await db('photos').del();
const res = await request(app).post('/api/admin/photos/repair-capture-dates');
expect(res.body.count).toBe(0);
const row = await db('maintenance_jobs').where({ job_name: jobs.JOB_CAPTURE_DATE_BACKFILL }).first();
expect(row.is_running).toBeFalsy();
// ...and a second POST is therefore accepted rather than 409ing forever.
expect((await request(app).post('/api/admin/photos/repair-capture-dates')).status).toBe(200);
});
});
@@ -0,0 +1,102 @@
/**
* PostgreSQL checks for the shared maintenance-job state (#1181).
* Gated: runs only when PICPEAK_PG_TEST_URL points at a throwaway database, e.g.
* PICPEAK_PG_TEST_URL="postgres://picpeak:picpeak_secure_pass_2024@127.0.0.1:7102/picpeak_mjs_test" \
* npx jest __tests__/integration/maintenanceJobStatePg.test.js
*
* What SQLite cannot answer: the claim leans on comparing a `timestamp` column
* against an ISO-8601 string, and on an UPDATE ... WHERE guard being atomic
* under real concurrent connections. SQLite compares those strings
* lexicographically and serialises writes anyway, so it would pass either way —
* exactly the shape of divergence that has bitten this repo before.
*/
const knex = require('knex');
const PG_URL = process.env.PICPEAK_PG_TEST_URL;
const maybe = PG_URL ? describe : describe.skip;
maybe('maintenance job state on Postgres', () => {
let pgDb;
let jobs;
const JOB = 'photo_dimension_repair';
beforeAll(async () => {
pgDb = knex({ client: 'pg', connection: PG_URL, pool: { min: 0, max: 10 } });
await pgDb.raw('DROP TABLE IF EXISTS maintenance_jobs');
await require('../../migrations/core/179_maintenance_job_state').up(pgDb);
jest.resetModules();
jest.doMock('../../src/database/db', () => ({ db: pgDb }));
jest.doMock('../../src/utils/logger', () => ({
debug: jest.fn(), info: jest.fn(), warn: jest.fn(), error: jest.fn(),
}));
jobs = require('../../src/services/maintenanceJobState');
}, 60000);
afterAll(async () => {
jest.dontMock('../../src/database/db');
if (pgDb) await pgDb.destroy();
});
beforeEach(async () => {
await pgDb('maintenance_jobs').update({
is_running: false, started_at: null, heartbeat_at: null, finished_at: null, last_result: null, owner: null, claim_token: null,
});
});
test('the ISO-string cutoff really compares as a timestamp, not as text', async () => {
expect(await jobs.claim(JOB)).toEqual(expect.any(String));
expect(await jobs.claim(JOB)).toBeNull();
await pgDb('maintenance_jobs').where({ job_name: JOB })
.update({ heartbeat_at: new Date(Date.now() - jobs.DEFAULT_STALE_MS - 60000).toISOString() });
// If Postgres had rejected or mis-cast the ISO string this would either
// throw or never match.
expect(await jobs.claim(JOB)).toEqual(expect.any(String));
const row = await pgDb('maintenance_jobs').where({ job_name: JOB }).first();
expect(row.heartbeat_at).toBeInstanceOf(Date);
});
test('concurrent claims on real connections produce exactly one winner', async () => {
// The whole point of the conditional UPDATE. Ten connections race; nine
// must lose. SQLite cannot demonstrate this — it serialises writers.
const results = await Promise.all(Array.from({ length: 10 }, () => jobs.claim(JOB)));
expect(results.filter(Boolean)).toHaveLength(1);
// ...and the winner holds a token nobody else can forge.
expect(results.find(Boolean)).toEqual(expect.any(String));
});
test('a released job can be re-claimed exactly once again', async () => {
const token = await jobs.claim(JOB);
await jobs.release(JOB, token, { success: 2, failed: 0 });
const results = await Promise.all(Array.from({ length: 5 }, () => jobs.claim(JOB)));
expect(results.filter(Boolean)).toHaveLength(1);
expect((await jobs.read(JOB)).lastResult).toEqual({ success: 2, failed: 0 });
});
test('a superseded runner is fenced out on real Postgres', async () => {
const oldToken = await jobs.claim(JOB);
await pgDb('maintenance_jobs').where({ job_name: JOB })
.update({ heartbeat_at: new Date(Date.now() - jobs.DEFAULT_STALE_MS - 60000).toISOString() });
const newToken = await jobs.claim(JOB);
expect(await jobs.heartbeat(JOB, oldToken)).toBe(false);
expect(await jobs.release(JOB, oldToken, { success: 999, failed: 0 })).toBe(false);
// The new owner still holds it, with its result unwritten.
expect((await jobs.read(JOB)).isRunning).toBe(true);
expect(await jobs.release(JOB, newToken, { success: 4, failed: 0 })).toBe(true);
});
test('read() reports a live claim as running and a stale one as not', async () => {
await jobs.claim(JOB);
expect((await jobs.read(JOB)).isRunning).toBe(true);
await pgDb('maintenance_jobs').where({ job_name: JOB })
.update({ heartbeat_at: new Date(Date.now() - jobs.DEFAULT_STALE_MS - 1000).toISOString() });
expect((await jobs.read(JOB)).isRunning).toBe(false);
});
});
@@ -0,0 +1,114 @@
/**
* Publishing must not be a way around the configured gallery password policy.
*
* `POST /:id/publish` (#627) re-hashes `password_hash` from a plaintext the
* admin re-types in the publish dialog, and validated it with nothing but
* express-validator's `isLength({ min: 6 })`. So the configured complexity —
* moderate by default, meaning 8 characters plus upper, lower and a digit —
* governed event creation and password reset, while this door accepted
* `aaaaaa` and made it the live gallery password.
*
* Not an escalation: it needs admin auth plus events.edit, and such an admin
* could already set a weak password elsewhere. It is a policy gap — the admin
* UI advertises a complexity level this write path did not enforce.
*/
const path = require('path');
const fs = require('fs');
const os = require('os');
process.env.NODE_ENV = 'test';
process.env.TEST_DATABASE_PATH = path.join(
fs.mkdtempSync(path.join(os.tmpdir(), 'picpeak-pubpolicy-')), 'db.sqlite',
);
process.env.JWT_SECRET = process.env.JWT_SECRET || 'publish-policy-test-secret';
process.env.STORAGE_PATH = fs.mkdtempSync(path.join(os.tmpdir(), 'picpeak-pubpolicy-storage-'));
const request = require('supertest');
const {
bootCrmDb, seedMinimal, assignAdminRole, mintAdminToken, buildRouteApp,
} = require('./helpers/crmDb');
describe('publish enforces the gallery password policy', () => {
let db; let cleanup; let app; let token;
beforeAll(async () => {
({ db, cleanup } = await bootCrmDb());
const { adminId } = await seedMinimal(db);
await assignAdminRole(db, adminId, 'admin');
token = mintAdminToken(adminId);
app = buildRouteApp('/admin/events', require('../../src/routes/adminEvents'));
}, 120000);
afterAll(async () => { if (cleanup) await cleanup(); });
async function seedDraft(slug) {
const [row] = await db('events').insert({
slug,
event_type: 'wedding',
event_name: `Event ${slug}`,
event_date: '2026-09-01',
host_email: 'client@example.com',
admin_email: 'admin@example.com',
password_hash: 'original-hash',
require_password: 1,
share_link: `/gallery/${slug}/share`,
share_token: `${slug}-token`,
expires_at: new Date(Date.now() + 7 * 24 * 3600 * 1000).toISOString(),
is_active: 1,
is_archived: 0,
is_draft: 1,
created_at: new Date().toISOString(),
}).returning('id');
return typeof row === 'object' ? row.id : row;
}
it('refuses a password that misses the configured complexity', async () => {
const id = await seedDraft('weak-publish');
const res = await request(app)
.post(`/admin/events/${id}/publish`)
.set('Authorization', `Bearer ${token}`)
.send({ password: 'aaaaaa' });
expect(res.status).toBe(400);
expect(res.body.error).toMatch(/security requirements/i);
// Rejected BEFORE the write, not after — the gallery must be untouched,
// and still a draft.
const after = await db('events').where({ id }).first();
expect(after.password_hash).toBe('original-hash');
expect(after.is_draft === 1 || after.is_draft === true).toBe(true);
});
it('still accepts a password that meets it', async () => {
const id = await seedDraft('strong-publish');
const res = await request(app)
.post(`/admin/events/${id}/publish`)
.set('Authorization', `Bearer ${token}`)
.send({ password: 'Sup3r-Secret' });
expect(res.status).toBe(200);
const bcrypt = require('bcrypt');
const after = await db('events').where({ id }).first();
expect(after.password_hash).not.toBe('original-hash');
expect(await bcrypt.compare('Sup3r-Secret', after.password_hash)).toBe(true);
});
it('leaves a publish without a password alone', async () => {
// The legacy sentinel path: no password in the body means no rehash, so
// the policy has nothing to check and must not block the publish.
const id = await seedDraft('no-password-publish');
const res = await request(app)
.post(`/admin/events/${id}/publish`)
.set('Authorization', `Bearer ${token}`)
.send({});
expect(res.status).toBe(200);
const after = await db('events').where({ id }).first();
expect(after.password_hash).toBe('original-hash');
});
});
@@ -0,0 +1,260 @@
/**
* scripts/regenerate-thumbnails.js against external photos (#1148).
*
* The same defect #1129 fixed in the admin route, still standing in the CLI
* fallback: the script resolved every source as
* `storage/events/active/<photo.path>` and fs.access'd it. External and
* reference rows do not live there — their originals sit under
* `events.external_path` — so every one failed the check and was counted as an
* error. On an install where all photos are external the script did nothing at
* all, while reporting one error per photo.
*
* Driven against a REAL file on a REAL external mount with the real
* imageProcessor, not a mock: the whole point is that the source resolves off
* the mount, and a mocked ensureThumbnail would assert nothing about that.
*
* Responsive tiers (#1095/#1109) do not exist on this branch, so the tier
* backfill in the main twin has nothing to port. Everything else does.
*/
const fs = require('fs');
const path = require('path');
const os = require('os');
const sharp = require('sharp');
const { execFile } = require('child_process');
describe('regenerate-thumbnails script (#1148)', () => {
let tmpDir; let db; let cleanup; let regenerateThumbnails;
let eventId; let externalPhotoId; let videoPhotoId; let watcherVideoId; let repairPhotoId;
let vanishingPhotoId;
let externalRoot;
beforeAll(async () => {
tmpDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'picpeak-regen-script-'));
process.env.NODE_ENV = 'test';
process.env.TEST_DATABASE_PATH = path.join(tmpDir, 'data', 'test.db');
process.env.STORAGE_PATH = path.join(tmpDir, 'storage');
// External sources are sandboxed under EXTERNAL_MEDIA_ROOT. Rows carry a
// path relative to that root (#1163), so the 'wedding/' prefix on each
// external_relpath below is the event folder, not decoration.
process.env.EXTERNAL_MEDIA_ROOT = path.join(tmpDir, 'media');
externalRoot = path.join(process.env.EXTERNAL_MEDIA_ROOT, 'wedding');
await fs.promises.mkdir(path.dirname(process.env.TEST_DATABASE_PATH), { recursive: true });
await fs.promises.mkdir(process.env.STORAGE_PATH, { recursive: true });
await fs.promises.mkdir(externalRoot, { recursive: true });
jest.resetModules();
({ db, cleanup } = await require('./helpers/crmDb').bootCrmDb());
// A real image on the external mount — never under events/active.
await sharp({
create: { width: 1200, height: 800, channels: 3, background: { r: 10, g: 90, b: 160 } },
}).jpeg().toFile(path.join(externalRoot, 'shot.jpg'));
const [ev] = await db('events').insert({
slug: 'regen-script-event',
event_type: 'wedding',
event_name: 'Regen Script',
event_date: '2026-08-01',
host_email: 'h@example.com',
admin_email: 'a@example.com',
password_hash: 'x',
share_link: '/gallery/regen-script-event/share',
expires_at: new Date(Date.now() + 7 * 24 * 3600 * 1000).toISOString(),
source_mode: 'reference',
external_path: 'wedding',
created_at: new Date().toISOString(),
}).returning('id');
eventId = typeof ev === 'object' ? ev.id : ev;
const [p] = await db('photos').insert({
event_id: eventId,
filename: 'shot.jpg',
// `path` is what the old script joined onto events/active. Left
// populated on purpose: the fix must ignore it for an external row.
path: 'regen-script-event/shot.jpg',
type: 'individual',
source_origin: 'external',
external_relpath: 'wedding/shot.jpg',
uploaded_at: new Date().toISOString(),
}).returning('id');
externalPhotoId = typeof p === 'object' ? p.id : p;
const [v] = await db('photos').insert({
event_id: eventId,
filename: 'clip.mp4',
path: 'regen-script-event/clip.mp4',
type: 'individual',
media_type: 'video',
mime_type: 'video/mp4',
source_origin: 'external',
external_relpath: 'wedding/clip.mp4',
uploaded_at: new Date().toISOString(),
}).returning('id');
videoPhotoId = typeof v === 'object' ? v.id : v;
// How fileWatcher.processNewPhoto actually writes a video: `type` and
// `mime_type` set, media_type left to its 'image' default. A media_type-only
// filter lets this through and hands the container to Sharp.
//
// The file has to EXIST, otherwise the row fails resolution and looks
// skipped for the wrong reason — the bug is Sharp being handed a video, not
// a missing source. Real MP4 header bytes, no image in sight.
await fs.promises.writeFile(
path.join(externalRoot, 'watched.mp4'),
Buffer.from('00000018667479706d70343200000000', 'hex')
);
const [wv] = await db('photos').insert({
event_id: eventId,
filename: 'watched.mp4',
path: 'regen-script-event/watched.mp4',
type: 'video',
mime_type: 'video/mp4',
source_origin: 'external',
external_relpath: 'wedding/watched.mp4',
uploaded_at: new Date().toISOString(),
}).returning('id');
watcherVideoId = typeof wv === 'object' ? wv.id : wv;
expect((await db('photos').where('id', watcherVideoId).first()).media_type).not.toBe('video');
// A photo whose thumbnail_path points at something that is no longer there.
await sharp({
create: { width: 900, height: 600, channels: 3, background: { r: 200, g: 40, b: 40 } },
}).jpeg().toFile(path.join(externalRoot, 'repair.jpg'));
const [rp] = await db('photos').insert({
event_id: eventId,
filename: 'repair.jpg',
path: 'regen-script-event/repair.jpg',
type: 'individual',
thumbnail_path: 'thumbnails/thumb_ext_missing_repair.jpg',
source_origin: 'external',
external_relpath: 'wedding/repair.jpg',
uploaded_at: new Date().toISOString(),
}).returning('id');
repairPhotoId = typeof rp === 'object' ? rp.id : rp;
// A photo whose source is not on the mount at all — an unavailable mount,
// which is the failure an operator most needs to hear about.
const [vp] = await db('photos').insert({
event_id: eventId,
filename: 'missing.jpg',
path: 'regen-script-event/missing.jpg',
type: 'individual',
source_origin: 'external',
external_relpath: 'missing.jpg',
uploaded_at: new Date().toISOString(),
}).returning('id');
vanishingPhotoId = typeof vp === 'object' ? vp.id : vp;
({ regenerateThumbnails } = require('../../scripts/regenerate-thumbnails'));
}, 180000);
afterAll(async () => {
if (cleanup) await cleanup();
await fs.promises.rm(tmpDir, { recursive: true, force: true }).catch(() => {});
});
it('builds a thumbnail for an external photo instead of erroring on events/active', async () => {
// The location the old script computed and fs.access'd. Nothing is there,
// which is the whole defect — it is not where an external original lives.
// (The old script cannot be driven from a test directly: it had no export
// and ran on require, calling process.exit. Making it importable is part
// of this fix.)
const legacyPath = path.join(process.env.STORAGE_PATH, 'events/active', 'regen-script-event/shot.jpg');
expect(fs.existsSync(legacyPath)).toBe(false);
const result = await regenerateThumbnails(eventId);
// The old script reported an error for this photo and wrote nothing.
// The unresolvable row fails; the external photo and the repair row build.
expect(result.errorCount).toBe(1);
expect(result.successCount).toBe(2);
const row = await db('photos').where('id', externalPhotoId).first();
expect(row.thumbnail_path).toBeTruthy();
const onDisk = path.join(process.env.STORAGE_PATH, row.thumbnail_path);
expect(fs.existsSync(onDisk)).toBe(true);
// Named per-photo so two events referencing one NAS basename cannot
// clobber each other — the property ensureThumbnail owns and the reason
// the script must not build this name itself.
expect(path.basename(row.thumbnail_path)).toContain(`ext${externalPhotoId}_`);
});
it('leaves videos alone', async () => {
// A video thumbnail is a poster frame from videoProcessor; handing the
// container to Sharp produced one error per video row.
const row = await db('photos').where('id', videoPhotoId).first();
expect(row.thumbnail_path).toBeFalsy();
});
it('leaves a watcher-imported video alone, which carries no media_type', async () => {
// fileWatcher writes type + mime_type and lets media_type default to
// 'image', so filtering on media_type alone still fed these to Sharp. The
// signal is errorCount: the images are already done by now, so the only
// NEW thing that could fail this run is a video reaching Sharp. One error
// is the deliberately unresolvable row; two would be the video.
const result = await regenerateThumbnails(eventId);
expect(result.errorCount).toBe(1);
const row = await db('photos').where('id', watcherVideoId).first();
expect(row.thumbnail_path).toBeFalsy();
});
it('is idempotent — a second run skips instead of rebuilding', async () => {
const before = await db('photos').where('id', externalPhotoId).first();
const result = await regenerateThumbnails(eventId);
expect(result.errorCount).toBe(1);
expect(result.successCount).toBe(0);
expect(result.skipCount).toBe(2);
const after = await db('photos').where('id', externalPhotoId).first();
expect(after.thumbnail_path).toBe(before.thumbnail_path);
});
it('counts a repaired thumbnail as generated, not skipped', async () => {
// Both images are valid at this point. Destroy ONE thumbnail object while
// leaving thumbnail_path pointing at it — the corrupt/missing case.
const row = await db('photos').where('id', repairPhotoId).first();
const onDisk = path.join(process.env.STORAGE_PATH, row.thumbnail_path);
await fs.promises.rm(onDisk);
const result = await regenerateThumbnails(eventId);
// On local and external storage the rebuilt key is identical, so inferring
// "skipped" from an unchanged path reports this repair as already valid —
// the one number an operator running this is actually reading.
expect(result.successCount).toBe(1);
expect(result.skipCount).toBe(1);
expect(result.errorCount).toBe(1);
expect(fs.existsSync(onDisk)).toBe(true);
});
/** Run the CLI the way cron does, and hand back its exit status. */
const runCli = (args = []) => new Promise((resolve) => {
execFile(
process.execPath,
[path.join(__dirname, '..', '..', 'scripts', 'regenerate-thumbnails.js'), ...args],
{ env: { ...process.env }, cwd: path.join(__dirname, '..', '..') },
(error, stdout, stderr) => resolve({ code: error?.code ?? 0, stdout, stderr })
);
});
it('exits nonzero when a photo could not be built', async () => {
// Exit status is the only thing a cron job reads, and `missing.jpg` has no
// source on the mount.
const failed = await runCli([String(eventId)]);
expect(failed.code).toBe(1);
expect(failed.stderr).toContain('completed with failures');
}, 120000);
it('exits zero when every photo resolves', async () => {
// Drop the unresolvable row: a clean run must not cry wolf at automation.
await db('photos').where('id', vanishingPhotoId).del();
const ok = await runCli([String(eventId)]);
expect(ok.code).toBe(0);
expect(ok.stdout).toContain('Script completed successfully');
}, 120000);
});
@@ -0,0 +1,107 @@
/**
* The admin photo list's category filter, and the value it answers to (#1211).
*
* The frontend used to send `category_id=0` for "Uncategorized". This route
* skips `'0'` outright — the guard reads `category_id !== '0'` — so no
* condition was applied and the whole event came back. Four lines below that
* guard sits the branch that does the work, keyed on the literal
* `uncategorized`, which nothing was sending.
*
* Reported in #1209 by someone trying to isolate a few thousand uncategorised
* imports. The frontend half is fixed in PhotoFilters; this pins the backend
* half of the same contract, because the failure mode was the two ends
* disagreeing about a string and neither one being wrong on its own.
*/
const request = require('supertest');
const express = require('express');
const { bootCrmDb, seedMinimal } = require('./helpers/crmDb');
describe('admin photo list — uncategorized filter (#1211)', () => {
let db; let cleanup; let app;
let eventId; let categoryId;
let uncategorisedIds; let categorisedId;
const list = async (query = '') => {
const res = await request(app).get(`/api/admin/events/${eventId}/photos${query}`);
expect(res.status).toBe(200);
const photos = Array.isArray(res.body) ? res.body : res.body.photos;
return (photos || []).map((p) => p.id).sort((a, b) => a - b);
};
beforeAll(async () => {
jest.resetModules();
jest.doMock('../../src/middleware/auth', () => ({
adminAuth: (req, _res, next) => { req.admin = { id: 1, username: 'tester' }; next(); },
}));
jest.doMock('../../src/middleware/permissions', () => ({
requirePermission: () => (_req, _res, next) => next(),
}));
jest.doMock('../../src/middleware/ownership', () => ({
requireEventOwnership: (_req, _res, next) => next(),
}));
jest.doMock('../../src/utils/logger', () => ({
debug: jest.fn(), info: jest.fn(), warn: jest.fn(), error: jest.fn(),
}));
({ db, cleanup } = await bootCrmDb());
await seedMinimal(db);
const [ev] = await db('events').insert({
slug: 'uncat-filter', event_type: 'wedding', event_name: 'Uncat Filter',
event_date: '2026-08-01', host_email: 'h@example.com', admin_email: 'a@example.com',
password_hash: 'x', share_link: '/gallery/uncat-filter/share',
expires_at: new Date(Date.now() + 7 * 24 * 3600 * 1000).toISOString(),
is_active: 1, is_archived: 0, is_draft: 0, created_at: new Date().toISOString(),
}).returning('id');
eventId = typeof ev === 'object' ? ev.id : ev;
const [cat] = await db('photo_categories')
.insert({ name: 'Ceremony', slug: 'ceremony', event_id: eventId })
.returning('id');
categoryId = typeof cat === 'object' ? cat.id : cat;
const insertPhoto = async (filename, category) => {
const [p] = await db('photos').insert({
event_id: eventId, filename, path: `events/uncat/${filename}`,
type: 'individual', category_id: category,
uploaded_at: new Date().toISOString(),
}).returning('id');
return typeof p === 'object' ? p.id : p;
};
// Two with no category — the shape a plugin upload leaves behind — and one
// filed properly, so a filter that does nothing is visibly different from
// a filter that works.
uncategorisedIds = [await insertPhoto('a.jpg', null), await insertPhoto('b.jpg', null)];
categorisedId = await insertPhoto('c.jpg', categoryId);
uncategorisedIds.sort((a, b) => a - b);
app = express();
app.use(express.json());
app.use('/api/admin/events', require('../../src/routes/adminPhotos'));
}, 180000);
afterAll(async () => { if (cleanup) await cleanup(); });
it('returns only the photos with no category', async () => {
expect(await list('?category_id=uncategorized')).toEqual(uncategorisedIds);
});
it('returns everything when no category filter is given', async () => {
expect(await list()).toEqual([...uncategorisedIds, categorisedId].sort((a, b) => a - b));
});
it('still filters by a real category id', async () => {
expect(await list(`?category_id=${categoryId}`)).toEqual([categorisedId]);
});
it('treats 0 as no filter at all', async () => {
// Pinning the behaviour that made the bug silent rather than loud: '0' is
// not "uncategorized" and never was, it simply falls through the guard. A
// future change that made 0 mean uncategorized here would be fine too —
// but it must be a decision, not an accident, and this test forces it.
expect(await list('?category_id=0')).toEqual([...uncategorisedIds, categorisedId].sort((a, b) => a - b));
});
});
@@ -1,103 +0,0 @@
/**
* Regression test for the cross-event thumbnail enumeration leak.
*
* Thumbnails are served flat from /thumbnails/thumb_<name> with
* deterministic, enumerable filenames. photoAuth previously granted any
* holder of a gallery token for ANY active event access to ANY thumbnail
* (it set eventSlug=null and returned next() as long as the token's event
* existed), so a visitor to one gallery could pull another (password-
* protected) gallery's entire thumbnail set. The fix scopes thumbnail
* access to the token's event by matching the requested file against
* photos.thumbnail_path for that event_id.
*/
process.env.JWT_SECRET = 'test-secret-thumbnail-scope-000000000000';
const jwt = require('jsonwebtoken');
// Two events, each owning one thumbnail. The photos mock resolves a row
// only when BOTH event_id and thumbnail_path match — i.e. it models the
// real ownership query.
const EVENTS = [
{ id: 10, slug: 'event-a', is_active: 1 },
{ id: 20, slug: 'event-b', is_active: 1 },
];
const PHOTOS = [
{ id: 1, event_id: 10, thumbnail_path: 'thumbnails/thumb_event-a_ceremony_0001.jpg' },
{ id: 2, event_id: 20, thumbnail_path: 'thumbnails/thumb_event-b_ceremony_0001.jpg' },
];
jest.mock('../../src/database/db', () => ({
db: (table) => ({
_cond: null,
where(cond) { this._cond = cond; return this; },
first() {
if (table === 'events') {
return Promise.resolve(EVENTS.find((e) => e.id === this._cond.id) || null);
}
if (table === 'photos') {
return Promise.resolve(
PHOTOS.find((p) => p.event_id === this._cond.event_id
&& p.thumbnail_path === this._cond.thumbnail_path) || null
);
}
return Promise.resolve(null);
},
}),
}));
jest.mock('../../src/utils/logger', () => ({
info: jest.fn(), warn: jest.fn(), error: jest.fn(), debug: jest.fn(),
}));
const photoAuth = require('../../src/middleware/photoAuth');
function galleryToken(eventId) {
return jwt.sign({ type: 'gallery', eventId }, process.env.JWT_SECRET, { issuer: 'picpeak-auth' });
}
function makeReqRes(token, thumbPath) {
const req = { path: thumbPath, headers: { authorization: `Bearer ${token}` }, cookies: {} };
const res = {
statusCode: null,
body: null,
status(code) { this.statusCode = code; return this; },
json(payload) { this.body = payload; return this; },
};
return { req, res };
}
describe('photoAuth — thumbnail ownership scoping', () => {
it('denies a gallery token for event A fetching event B\'s thumbnail', async () => {
const { req, res } = makeReqRes(galleryToken(10), '/thumb_event-b_ceremony_0001.jpg');
const next = jest.fn();
await photoAuth(req, res, next);
// Access denied: middleware must not pass the request through.
expect(next).not.toHaveBeenCalled();
expect(res.statusCode).toBeGreaterThanOrEqual(400);
expect(req.event).toBeUndefined();
});
it('allows a gallery token to fetch its own event\'s thumbnail', async () => {
const { req, res } = makeReqRes(galleryToken(20), '/thumb_event-b_ceremony_0001.jpg');
const next = jest.fn();
await photoAuth(req, res, next);
expect(next).toHaveBeenCalled();
expect(req.event).toMatchObject({ id: 20 });
});
it('denies a traversal / foreign filename that matches no owned thumbnail', async () => {
const { req, res } = makeReqRes(galleryToken(10), '/thumb_../../etc/passwd');
const next = jest.fn();
await photoAuth(req, res, next);
expect(next).not.toHaveBeenCalled();
expect(res.statusCode).toBeGreaterThanOrEqual(400);
expect(req.event).toBeUndefined();
});
});
@@ -0,0 +1,70 @@
/**
* Second security sweep on the same branch as the password-strength DoS fix
* (stable port: the maintenance and admin-preview cases do not apply here).
* Each block pins one gap the audit found:
*
* - the general rate limiter skipped anyone holding ANY verified JWT,
* including a gallery token minted for free on password-less galleries
* - the multipart branch of the CSRF Content-Type gate accepted cross-site
* form posts
*/
const jwt = require('jsonwebtoken');
process.env.JWT_SECRET = 'hardening-batch2-secret';
const fake = { maintenance: 'true', revoked: false, beforeCutoff: false, admin: { id: 1, password_changed_at: null } };
jest.mock('../../src/database/db', () => {
const db = jest.fn((table) => {
const q = {
where: jest.fn().mockReturnThis(),
select: jest.fn().mockReturnThis(),
first: jest.fn(async () => {
if (table === 'app_settings') {
return { setting_key: 'general_maintenance_mode', setting_value: fake.maintenance };
}
if (table === 'admin_users') return fake.admin;
return null;
}),
};
return q;
});
return { db, withRetry: (fn) => fn() };
});
jest.mock('../../src/utils/logger', () => ({ error: jest.fn(), warn: jest.fn(), info: jest.fn(), debug: jest.fn() }));
process.env.FRONTEND_URL = 'https://photos.example.com';
const { isAuthenticated } = require('../../src/services/rateLimitService');
const { multipartOriginAllowed } = require('../../src/utils/requestOrigin');
const iat = Math.floor(Date.now() / 1000) - 10;
const adminToken = (extra = {}) => jwt.sign({ type: 'admin', id: 1, iat, ...extra }, process.env.JWT_SECRET, { issuer: 'picpeak-auth' });
const galleryToken = () => jwt.sign({ type: 'gallery', eventId: 1, iat }, process.env.JWT_SECRET, { issuer: 'picpeak-auth' });
describe('general rate limiter skip', () => {
const req = (token) => ({ path: '/api/gallery/x/photos', headers: { authorization: `Bearer ${token}` }, cookies: {} });
it('is granted to an admin session', () => {
expect(isAuthenticated(req(adminToken()))).toBe(true);
});
it('is NOT granted to a gallery token', () => {
expect(isAuthenticated(req(galleryToken()))).toBe(false);
});
});
describe('multipart origin gate', () => {
const req = (headers) => ({ headers: { host: 'photos.example.com', ...headers } });
it('accepts same-origin, same-site and non-browser requests', () => {
expect(multipartOriginAllowed(req({ 'sec-fetch-site': 'same-origin' }))).toBe(true);
expect(multipartOriginAllowed(req({ 'sec-fetch-site': 'same-site' }))).toBe(true);
expect(multipartOriginAllowed(req({ 'sec-fetch-site': 'none' }))).toBe(true);
expect(multipartOriginAllowed(req({}))).toBe(true);
expect(multipartOriginAllowed(req({ origin: 'https://photos.example.com' }))).toBe(true);
// Same-origin install without FRONTEND_URL: Origin matches the Host.
expect(multipartOriginAllowed({ headers: { host: 'gallery.local', origin: 'http://gallery.local' } })).toBe(true);
});
it('rejects cross-site form posts', () => {
expect(multipartOriginAllowed(req({ 'sec-fetch-site': 'cross-site' }))).toBe(false);
expect(multipartOriginAllowed(req({ origin: 'https://evil.example' }))).toBe(false);
expect(multipartOriginAllowed(req({ origin: 'null' }))).toBe(false);
});
});
@@ -0,0 +1,242 @@
/**
* Repairing the bundled templates' fixed image height (#1131).
*
* The risk in a migration that rewrites user-visible CSS is doing too much,
* so most of what is pinned here is what it must NOT touch: the other pixel
* heights inside the very same templates (a 1px divider, an 8px scrollbar),
* and any rule a user wrote themselves.
*/
const path = require('path');
const fs = require('fs');
const os = require('os');
const migration = require('../../migrations/core/175_fix_css_template_photo_height');
const ELEGANT_DARK = `
.photo-card {
border-radius: 12px;
}
.photo-card img {
width: 100%;
height: 200px;
object-fit: cover;
transition: transform 0.3s ease;
}
`;
const LIQUID_GLASS_DARK = `
.gallery-page::after {
content: '';
height: 1px;
background: linear-gradient(90deg, transparent, #fff, transparent);
}
.photo-card img {
width: 100%;
height: 240px;
object-fit: cover;
filter: brightness(0.9);
}
.gallery-page ::-webkit-scrollbar {
width: 8px;
height: 8px;
}
@media (max-width: 640px) {
.photo-card img {
height: 180px;
}
}
`;
describe('migration 175 — CSS template image height (#1131)', () => {
let knex; let tmpDir;
beforeAll(async () => {
tmpDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'picpeak-mig175-'));
knex = require('knex')({
client: 'sqlite3',
connection: { filename: path.join(tmpDir, 'db.sqlite') },
useNullAsDefault: true,
});
await knex.schema.createTable('css_templates', (t) => {
t.increments('id').primary();
t.string('name');
t.text('css_content');
});
});
afterAll(async () => {
if (knex) await knex.destroy();
await fs.promises.rm(tmpDir, { recursive: true, force: true }).catch(() => {});
});
beforeEach(async () => { await knex('css_templates').del(); });
const contentOf = async (name) =>
(await knex('css_templates').where({ name }).first()).css_content;
it('relaxes the default template so the layouts h-full can win', async () => {
await knex('css_templates').insert({ name: 'Elegant Dark', css_content: ELEGANT_DARK });
await migration.up(knex);
const css = await contentOf('Elegant Dark');
expect(css).toContain('height: 100%');
expect(css).not.toContain('height: 200px');
// Everything else about the rule survives.
expect(css).toContain('object-fit: cover');
expect(css).toContain('transition: transform 0.3s ease');
});
it('fixes both the base rule and the mobile override of the dark glass template', async () => {
await knex('css_templates').insert({ name: 'Liquid Glass Dark', css_content: LIQUID_GLASS_DARK });
await migration.up(knex);
const css = await contentOf('Liquid Glass Dark');
expect(css).not.toContain('height: 240px');
expect(css).not.toContain('height: 180px');
expect(css.match(/height: 100%/g)).toHaveLength(2);
});
it('leaves the divider and the scrollbar alone', async () => {
await knex('css_templates').insert({ name: 'Liquid Glass Dark', css_content: LIQUID_GLASS_DARK });
await migration.up(knex);
// The whole reason this matches full rule bodies rather than every
// `height: <n>px`: these are in the same stylesheet and are correct.
const css = await contentOf('Liquid Glass Dark');
expect(css).toContain('height: 1px');
expect(css).toContain('width: 8px');
expect(css).toContain('height: 8px');
});
/**
* The case that forced the scope wider. `sanitizeCSS` strips control
* characters, so any template ever saved through the editor — including a
* save that only changed its name — has had every newline REMOVED. An
* exact-text migration finds nothing on those installs, is recorded as
* applied, and leaves them broken permanently.
*/
it('fixes a template that has been through the editor, newlines and all', async () => {
const { sanitizeCSS } = require('../../src/utils/cssSanitizer');
const { sanitized } = sanitizeCSS(ELEGANT_DARK);
// Precondition: the sanitizer really did flatten it.
expect(sanitized).not.toContain('\n');
expect(sanitized).toContain('height: 200px');
await knex('css_templates').insert({ name: 'Saved Once', css_content: sanitized });
await migration.up(knex);
const css = await contentOf('Saved Once');
expect(css).not.toContain('200px');
expect(css).toContain('height: 100%');
});
it('relaxes a user-authored fixed height too, but only on .photo-card img', async () => {
// Deliberately broader than the seeded text — see the migration header. A
// pixel height on the image cannot be right under any of the seven
// layouts, whoever wrote it; a height anywhere else is none of our
// business.
const mine = '.photo-card img {\n height: 220px;\n}\n.hero { height: 400px; }';
await knex('css_templates').insert({ name: 'My Own', css_content: mine });
await migration.up(knex);
const css = await contentOf('My Own');
expect(css).toContain('height: 100%');
expect(css).not.toContain('220px');
expect(css).toContain('.hero { height: 400px; }');
});
it('does not rewrite other properties that merely end in -height', async () => {
// `line-height: 200px` contains `height: 200px` as a substring, so an
// unanchored pattern silently rewrites it — in a migration that cannot be
// undone.
const mine = [
'.photo-card img {',
' line-height: 200px;',
' max-height: 300px;',
' min-height: 14px;',
' --tile-height: 220px;',
' height: 200px;',
'}',
].join('\n');
await knex('css_templates').insert({ name: 'Adjacent Props', css_content: mine });
await migration.up(knex);
const css = await contentOf('Adjacent Props');
expect(css).toContain('line-height: 200px');
expect(css).toContain('max-height: 300px');
expect(css).toContain('min-height: 14px');
expect(css).toContain('--tile-height: 220px');
// Only the real one moved.
expect(css).toContain('height: 100%');
expect(css).not.toMatch(/(?<![\w-])height:\s*200px/);
});
it('handles a grouped selector list', async () => {
// Requiring `{` straight after `img` skipped these entirely — and the
// migration is still recorded as applied, so the template kept the bug.
const mine = '.photo-card img, .thumbnail img {\n height: 200px;\n}';
await knex('css_templates').insert({ name: 'Grouped', css_content: mine });
await migration.up(knex);
const css = await contentOf('Grouped');
expect(css).toContain('.photo-card img, .thumbnail img {');
expect(css).toContain('height: 100%');
expect(css).not.toContain('200px');
});
it('skips a nested rule rather than rewriting the wrong declaration', async () => {
// Valid nested CSS that passes the validator. A brace-greedy body would
// capture the inner block and rewrite the CAPTION's height, which cannot
// be undone. Leaving it untouched is the lesser evil.
const mine = '.photo-card img {\n & + .caption { height: 200px; }\n}';
await knex('css_templates').insert({ name: 'Nested', css_content: mine });
await migration.up(knex);
expect(await contentOf('Nested')).toBe(mine);
});
it('leaves non-pixel heights on the image alone', async () => {
const mine = '.photo-card img { height: 50vh; }\n.photo-card img { height: auto; }';
await knex('css_templates').insert({ name: 'Relative', css_content: mine });
await migration.up(knex);
expect(await contentOf('Relative')).toBe(mine);
});
it('is idempotent and safe on a row with no CSS', async () => {
await knex('css_templates').insert([
{ name: 'Elegant Dark', css_content: ELEGANT_DARK },
{ name: 'Empty', css_content: null },
]);
await migration.up(knex);
const once = await contentOf('Elegant Dark');
await migration.up(knex);
expect(await contentOf('Elegant Dark')).toBe(once);
expect(await contentOf('Empty')).toBeNull();
});
it('no-ops when the table does not exist yet', async () => {
await knex.schema.dropTable('css_templates');
await expect(migration.up(knex)).resolves.toBeUndefined();
await knex.schema.createTable('css_templates', (t) => {
t.increments('id').primary();
t.string('name');
t.text('css_content');
});
});
});
@@ -0,0 +1,546 @@
/**
* One row per external file per event (#1162).
*
* The migration has two halves and they fail differently: the cleanup can take
* out the wrong row of a pair (losing a thumbnail, orphaning an event's hero),
* and the index can fail to be created at all — leaving an install that looks
* migrated and is still racing. Both are pinned here.
*/
const path = require('path');
const fs = require('fs');
const os = require('os');
const migration = require('../../migrations/core/176_external_relpath_unique');
describe('migration 176 — unique (event_id, external_relpath) (#1162)', () => {
let knex; let tmpDir;
beforeAll(async () => {
tmpDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'picpeak-mig186-'));
knex = require('knex')({
client: 'sqlite3',
connection: { filename: path.join(tmpDir, 'db.sqlite') },
useNullAsDefault: true,
});
});
afterAll(async () => {
if (knex) await knex.destroy();
await fs.promises.rm(tmpDir, { recursive: true, force: true }).catch(() => {});
});
beforeEach(async () => {
for (const table of [
'photos', 'events', 'photo_categories', 'photo_feedback',
'photo_admin_marks', 'photo_faces', 'image_access_logs', 'transfer_files',
]) {
await knex.schema.dropTableIfExists(table);
}
await knex.schema.createTable('events', (t) => {
t.increments('id').primary();
t.integer('hero_photo_id');
t.string('download_zip_path');
t.string('download_zip_generated_at');
});
await knex.schema.createTable('photo_categories', (t) => {
t.increments('id').primary();
t.integer('hero_photo_id');
});
await knex.schema.createTable('photos', (t) => {
t.increments('id').primary();
t.integer('event_id');
t.string('external_relpath');
t.string('thumbnail_path');
t.string('source_origin').defaultTo('managed');
t.integer('feedback_count').defaultTo(0);
t.integer('like_count').defaultTo(0);
t.decimal('average_rating', 3, 2).defaultTo(0);
t.integer('favorite_count').defaultTo(0);
t.integer('reaction_count').defaultTo(0);
t.integer('color_label_count').defaultTo(0);
t.string('face_status');
t.integer('view_count').defaultTo(0);
t.integer('download_count').defaultTo(0);
t.integer('face_count');
t.string('face_started_at');
t.text('face_error');
});
// Declared exactly as the real schema declares them — CASCADE and all.
// The point of these tables here is that SQLite does NOT enforce any of
// it (PicPeak never sets `PRAGMA foreign_keys = ON`), so a bare delete of
// the photo row leaves every one of them dangling.
await knex.schema.createTable('photo_feedback', (t) => {
t.increments('id').primary();
t.integer('photo_id').references('id').inTable('photos').onDelete('CASCADE');
t.integer('event_id');
t.string('feedback_type');
t.text('comment_text');
t.string('guest_identifier');
// Per-person guest identity (migration 078). Nullable: galleries without
// guest identity leave it NULL and fall back to guest_identifier.
t.integer('guest_id');
t.integer('rating');
t.boolean('is_hidden').defaultTo(false);
t.boolean('is_approved').defaultTo(true);
});
await knex.schema.createTable('photo_admin_marks', (t) => {
t.increments('id').primary();
t.integer('photo_id').notNullable().references('id').inTable('photos').onDelete('CASCADE');
t.integer('event_id');
t.integer('admin_id');
t.integer('rating');
// Independently writable alongside rating, per photoAdminMarksService.
t.string('color_label', 16);
t.unique(['photo_id', 'admin_id'], 'photo_admin_marks_photo_admin_uniq');
});
await knex.schema.createTable('photo_faces', (t) => {
t.increments('id').primary();
t.integer('photo_id').references('id').inTable('photos').onDelete('CASCADE');
t.integer('event_id');
// purgePhotoFaces rebuilds the people that lose members, so the cluster
// link and the vectors recomputeCentroid reads have to be here for this
// to exercise the real path rather than a stub.
t.integer('person_id');
t.binary('embedding');
t.float('det_score');
});
await knex.schema.createTable('image_access_logs', (t) => {
t.increments('id').primary();
t.integer('photo_id');
});
await knex.schema.createTable('transfer_files', (t) => {
t.increments('id').primary();
t.integer('transfer_id');
t.integer('photo_id');
t.unique(['transfer_id', 'photo_id'], 'transfer_files_unique');
});
});
/** Two duplicate rows for the same file: id 1 survives, id 2 is doomed. */
const seedPair = async () => {
await knex('photos').insert([
{ event_id: 1, external_relpath: 'a/x.jpg', thumbnail_path: 't', source_origin: 'external' },
{ event_id: 1, external_relpath: 'a/x.jpg', thumbnail_path: 't', source_origin: 'external' },
]);
};
const rows = () => knex('photos').orderBy('id', 'asc').select('*');
it('collapses a duplicated pair to one row and leaves distinct paths alone', async () => {
await knex('photos').insert([
{ event_id: 1, external_relpath: 'a/x.jpg', thumbnail_path: 't1', source_origin: 'external' },
{ event_id: 1, external_relpath: 'a/x.jpg', thumbnail_path: 't2', source_origin: 'external' },
{ event_id: 1, external_relpath: 'a/y.jpg', thumbnail_path: 't3', source_origin: 'external' },
]);
await migration.up(knex);
const after = await rows();
expect(after.map((r) => r.external_relpath)).toEqual(['a/x.jpg', 'a/y.jpg']);
// Lowest id survives when both sides are equally complete.
expect(after[0].id).toBe(1);
});
it('does not collapse the same path across different events', async () => {
// The constraint is per event. Two events referencing the same NAS folder
// is a supported setup, and treating those as duplicates would delete one
// event's entire library.
await knex('photos').insert([
{ event_id: 1, external_relpath: 'a/x.jpg', source_origin: 'external' },
{ event_id: 2, external_relpath: 'a/x.jpg', source_origin: 'external' },
]);
await migration.up(knex);
expect(await knex('photos').count('* as c').first()).toEqual({ c: 2 });
});
it('never touches managed rows, however many carry NULL', async () => {
// Every managed photo has external_relpath NULL. Grouping on it without
// the NOT NULL filter would make them all one enormous "duplicate" group
// and delete the entire library bar one row.
await knex('photos').insert([
{ event_id: 1, external_relpath: null, source_origin: 'managed' },
{ event_id: 1, external_relpath: null, source_origin: 'managed' },
{ event_id: 1, external_relpath: null, source_origin: 'managed' },
]);
await migration.up(knex);
expect(await knex('photos').count('* as c').first()).toEqual({ c: 3 });
});
it('keeps the row that has a thumbnail, not merely the lowest id', async () => {
// An import killed mid-flight leaves rows without a thumbnail. Dropping
// the completed one would blank a tile in the grid for no reason.
await knex('photos').insert([
{ event_id: 1, external_relpath: 'a/x.jpg', thumbnail_path: null, source_origin: 'external' },
{ event_id: 1, external_relpath: 'a/x.jpg', thumbnail_path: 'thumb.jpg', source_origin: 'external' },
]);
await migration.up(knex);
const after = await rows();
expect(after).toHaveLength(1);
expect(after[0].thumbnail_path).toBe('thumb.jpg');
});
it('repoints a hero that pointed at the row being removed', async () => {
// events.hero_photo_id is ON DELETE SET NULL, so without this the cleanup
// silently strips the event's hero image — a visible regression caused
// entirely by the fix.
await knex('photos').insert([
{ event_id: 1, external_relpath: 'a/x.jpg', thumbnail_path: 't', source_origin: 'external' },
{ event_id: 1, external_relpath: 'a/x.jpg', thumbnail_path: 't', source_origin: 'external' },
]);
await knex('events').insert({ id: 1, hero_photo_id: 2 });
await knex('photo_categories').insert({ id: 1, hero_photo_id: 2 });
await migration.up(knex);
expect((await knex('events').where({ id: 1 }).first()).hero_photo_id).toBe(1);
expect((await knex('photo_categories').where({ id: 1 }).first()).hero_photo_id).toBe(1);
});
it('leaves a hero that pointed at the survivor untouched', async () => {
await knex('photos').insert([
{ event_id: 1, external_relpath: 'a/x.jpg', thumbnail_path: 't', source_origin: 'external' },
{ event_id: 1, external_relpath: 'a/x.jpg', thumbnail_path: 't', source_origin: 'external' },
]);
await knex('events').insert({ id: 1, hero_photo_id: 1 });
await migration.up(knex);
expect((await knex('events').where({ id: 1 }).first()).hero_photo_id).toBe(1);
});
it('makes a second insert of the same path impossible afterwards', async () => {
// The whole point. Without this the route is still racing, and the
// migration is recorded as applied.
await knex('photos').insert({ event_id: 1, external_relpath: 'a/x.jpg', source_origin: 'external' });
await migration.up(knex);
await expect(
knex('photos').insert({ event_id: 1, external_relpath: 'a/x.jpg', source_origin: 'external' })
).rejects.toThrow(/unique/i);
});
it('still admits managed rows once the index exists', async () => {
await migration.up(knex);
await knex('photos').insert([
{ event_id: 1, external_relpath: null, source_origin: 'managed' },
{ event_id: 1, external_relpath: null, source_origin: 'managed' },
]);
expect(await knex('photos').count('* as c').first()).toEqual({ c: 2 });
});
it('leaves nothing dangling behind the deleted row', async () => {
// SQLite never enforces the ON DELETE CASCADE these tables declare, so a
// bare delete strands biometric embeddings, feedback and marks pointing at
// a photo id that no longer exists — on every SQLite install.
await seedPair();
await knex('photo_faces').insert({ photo_id: 2, event_id: 1 });
await knex('image_access_logs').insert({ photo_id: 2 });
await migration.up(knex);
expect(await knex('photo_faces').where('photo_id', 2).first()).toBeUndefined();
expect(await knex('image_access_logs').where('photo_id', 2).first()).toBeUndefined();
});
it('does not carry the duplicate\'s faces over to the survivor', async () => {
// Both rows were scanned independently, so the survivor already holds its
// own embeddings. Moving these would fabricate a second copy of every face
// and split the person clusters built from them.
await seedPair();
await knex('photo_faces').insert([{ photo_id: 1, event_id: 1 }, { photo_id: 2, event_id: 1 }]);
await migration.up(knex);
expect(await knex('photo_faces').count('* as c').first()).toEqual({ c: 1 });
});
it('moves a guest comment to the survivor rather than deleting it', async () => {
// The duplicates were separate tiles in the grid, so a guest could have
// commented on either. Silently dropping that inside a fix for silent data
// loss would be its own bug.
await seedPair();
await knex('photo_feedback').insert({
photo_id: 2, event_id: 1, feedback_type: 'comment',
comment_text: 'lovely shot', guest_identifier: 'guest-a',
});
await migration.up(knex);
const rows = await knex('photo_feedback');
expect(rows).toHaveLength(1);
expect(rows[0].photo_id).toBe(1);
expect(rows[0].comment_text).toBe('lovely shot');
});
it('keeps both comments when the same guest commented on both tiles', async () => {
await seedPair();
await knex('photo_feedback').insert([
{ photo_id: 1, event_id: 1, feedback_type: 'comment', comment_text: 'one', guest_identifier: 'g' },
{ photo_id: 2, event_id: 1, feedback_type: 'comment', comment_text: 'two', guest_identifier: 'g' },
]);
await migration.up(knex);
const rows = await knex('photo_feedback').orderBy('id');
expect(rows.map((r) => r.comment_text)).toEqual(['one', 'two']);
expect(rows.every((r) => r.photo_id === 1)).toBe(true);
});
it('does not double-count a like the same guest left on both tiles', async () => {
// Unlike comments, a like is a per-guest toggle: moving it would show two
// likes from one person.
await seedPair();
await knex('photo_feedback').insert([
{ photo_id: 1, event_id: 1, feedback_type: 'like', guest_identifier: 'g' },
{ photo_id: 2, event_id: 1, feedback_type: 'like', guest_identifier: 'g' },
]);
await migration.up(knex);
expect(await knex('photo_feedback').count('* as c').first()).toEqual({ c: 1 });
});
it('moves a like from a guest the survivor has never seen', async () => {
await seedPair();
await knex('photo_feedback').insert({
photo_id: 2, event_id: 1, feedback_type: 'like', guest_identifier: 'other',
});
await migration.up(knex);
const rows = await knex('photo_feedback');
expect(rows).toHaveLength(1);
expect(rows[0].photo_id).toBe(1);
});
it('moves an admin mark, and drops it when that admin already marked the survivor', async () => {
// photo_admin_marks is UNIQUE(photo_id, admin_id), so a blind move would
// throw and abort the migration.
await seedPair();
await knex('photo_admin_marks').insert([
{ photo_id: 1, event_id: 1, admin_id: 7, rating: 5 },
{ photo_id: 2, event_id: 1, admin_id: 7, rating: 2 },
{ photo_id: 2, event_id: 1, admin_id: 9, rating: 4 },
]);
await migration.up(knex);
const rows = await knex('photo_admin_marks').orderBy('admin_id');
expect(rows.map((r) => [r.admin_id, r.rating])).toEqual([[7, 5], [9, 4]]);
expect(rows.every((r) => r.photo_id === 1)).toBe(true);
});
it('respects the transfer_files uniqueness when moving membership', async () => {
await seedPair();
await knex('transfer_files').insert([
{ transfer_id: 3, photo_id: 1 },
{ transfer_id: 3, photo_id: 2 },
{ transfer_id: 4, photo_id: 2 },
]);
await migration.up(knex);
const rows = await knex('transfer_files').orderBy('transfer_id');
expect(rows.map((r) => r.transfer_id)).toEqual([3, 4]);
expect(rows.every((r) => r.photo_id === 1)).toBe(true);
});
it('recomputes the survivor\'s feedback totals after reparenting rows', async () => {
// photos carries denormalized counters (migration 033). A survivor that
// now OWNS the feedback but still renders zero is the visible half of
// getting this wrong.
await seedPair();
await knex('photo_feedback').insert([
{ photo_id: 2, event_id: 1, feedback_type: 'like', guest_identifier: 'g1' },
{ photo_id: 2, event_id: 1, feedback_type: 'rating', rating: 4, guest_identifier: 'g1' },
]);
await migration.up(knex);
const survivor = await knex('photos').where('id', 1).first();
expect(survivor.like_count).toBe(1);
expect(Number(survivor.average_rating)).toBe(4);
expect(survivor.feedback_count).toBe(1);
});
it('keeps two people who share a device apart', async () => {
// guest_identifier is per-device; guest_id is per-person (migration 078),
// and feedbackService scopes by guest_id when it is present. Keying on the
// identifier alone would read these as one person and delete a rating.
await seedPair();
await knex('photo_feedback').insert([
{ photo_id: 1, event_id: 1, feedback_type: 'rating', rating: 5, guest_identifier: 'shared', guest_id: 10 },
{ photo_id: 2, event_id: 1, feedback_type: 'rating', rating: 2, guest_identifier: 'shared', guest_id: 11 },
]);
await migration.up(knex);
const rows = await knex('photo_feedback').orderBy('guest_id');
expect(rows.map((r) => [r.guest_id, r.rating])).toEqual([[10, 5], [11, 2]]);
});
it('still dedupes one person voting on both tiles', async () => {
await seedPair();
await knex('photo_feedback').insert([
{ photo_id: 1, event_id: 1, feedback_type: 'like', guest_identifier: 'shared', guest_id: 10 },
{ photo_id: 2, event_id: 1, feedback_type: 'like', guest_identifier: 'shared', guest_id: 10 },
]);
await migration.up(knex);
expect(await knex('photo_feedback').count('* as c').first()).toEqual({ c: 1 });
});
it('still clears face rows on a branch that has no face feature', async () => {
// DIVERGES FROM MAIN, deliberately. Face recognition (#1090) is main-only:
// there is no faceProcessor on this branch, so purgePhotoFaces cannot be
// called and there are no event_people counts or centroids to reconcile.
// What still matters is the half that is not optional — the rows must not
// dangle, because SQLite never enforces the CASCADE that would remove
// them. The service reaches for purgePhotoFaces, finds nothing, and falls
// back to a plain delete; this pins that fallback.
//
// If faces are ever backported, main's version of this test comes with
// them.
await seedPair();
await knex('photo_faces').insert({ photo_id: 2, event_id: 1, person_id: 5 });
await migration.up(knex);
expect(await knex('photo_faces').count('* as c').first()).toEqual({ c: 0 });
});
it('keeps a hidden moderation record from swallowing the visible replacement', async () => {
// feedbackService lets both coexist and counts only the visible one.
await seedPair();
await knex('photo_feedback').insert([
{ photo_id: 1, event_id: 1, feedback_type: 'like', guest_identifier: 'g', is_hidden: true },
{ photo_id: 2, event_id: 1, feedback_type: 'like', guest_identifier: 'g', is_hidden: false },
]);
await migration.up(knex);
expect(await knex('photo_feedback').count('* as c').first()).toEqual({ c: 2 });
});
it('merges the independent halves of one admin\'s mark', async () => {
// rating and color_label are written independently, so the same admin can
// have rated one tile and coloured the other.
await seedPair();
await knex('photo_admin_marks').insert([
{ photo_id: 1, event_id: 1, admin_id: 7, rating: 5, color_label: null },
{ photo_id: 2, event_id: 1, admin_id: 7, rating: null, color_label: 'red' },
]);
await migration.up(knex);
const rows = await knex('photo_admin_marks');
expect(rows).toHaveLength(1);
expect([rows[0].rating, rows[0].color_label]).toEqual([5, 'red']);
});
it('requeues the survivor when the duplicate held the only scan', async () => {
// Otherwise the sole embeddings go with the purge and nothing re-queues:
// the photo just silently stops having a face.
await seedPair();
await knex('photo_faces').insert({ photo_id: 2, event_id: 1 });
await migration.up(knex);
expect((await knex('photos').where('id', 1).first()).face_status).toBe('pending');
});
it('carries the duplicate\'s views and downloads over', async () => {
await seedPair();
await knex('photos').where('id', 1).update({ view_count: 2, download_count: 1 });
await knex('photos').where('id', 2).update({ view_count: 5, download_count: 3 });
await migration.up(knex);
const survivor = await knex('photos').where('id', 1).first();
expect([survivor.view_count, survivor.download_count]).toEqual([7, 4]);
});
it('fails loudly rather than recording itself applied without the index', async () => {
// Swallowing a failed CREATE INDEX would leave the install permanently
// racy — the in-flight guard only covers one process — with nothing to
// trigger a retry. Driven through the helper the migration calls, against
// a table that still holds duplicates — i.e. what it would face if the
// dedupe above had not achieved uniqueness.
await seedPair();
const { createExternalRelpathIndex } = require('../../src/services/externalPhotoDedupe');
await expect(createExternalRelpathIndex(knex)).rejects.toThrow(/unique/i);
});
it('invalidates the pre-built download zip for the affected event', async () => {
// The cached archive still contains the rows just removed, and every
// ordinary photo-deletion path invalidates it for exactly that reason.
// getZipInfo treats a cleared record as a miss and rebuilds on request.
await seedPair();
await knex('events').insert({
id: 1, download_zip_path: 'events/active/x/.download-cache/all.zip',
download_zip_generated_at: '2026-01-01',
});
await migration.up(knex);
const ev = await knex('events').where('id', 1).first();
expect(ev.download_zip_path).toBeNull();
expect(ev.download_zip_generated_at).toBeNull();
});
it('leaves an untouched event\'s zip alone', async () => {
await seedPair();
await knex('events').insert([
{ id: 1, download_zip_path: 'a.zip', download_zip_generated_at: '2026-01-01' },
{ id: 2, download_zip_path: 'b.zip', download_zip_generated_at: '2026-01-01' },
]);
await migration.up(knex);
expect((await knex('events').where('id', 2).first()).download_zip_path).toBe('b.zip');
});
it('is idempotent', async () => {
await knex('photos').insert([
{ event_id: 1, external_relpath: 'a/x.jpg', source_origin: 'external' },
{ event_id: 1, external_relpath: 'a/x.jpg', source_origin: 'external' },
]);
await migration.up(knex);
const once = await rows();
await migration.up(knex);
expect(await rows()).toEqual(once);
});
it('rolls back to an unconstrained table', async () => {
await migration.up(knex);
await migration.down(knex);
await knex('photos').insert([
{ event_id: 1, external_relpath: 'a/x.jpg', source_origin: 'external' },
{ event_id: 1, external_relpath: 'a/x.jpg', source_origin: 'external' },
]);
expect(await knex('photos').count('* as c').first()).toEqual({ c: 2 });
});
it('no-ops before 041 has added the column', async () => {
await knex.schema.dropTableIfExists('photos');
await knex.schema.createTable('photos', (t) => { t.increments('id').primary(); });
await expect(migration.up(knex)).resolves.toBeUndefined();
});
});
@@ -0,0 +1,364 @@
/**
* Folding the event's base path into every external row (#1163).
*
* Two things can go wrong and both are silent, which is why they are pinned
* here rather than left to review: folding a path that was ALREADY folded
* (every original moves), and "repairing" a healthy install because the media
* root happened to be unmounted when the migration ran (every original moves).
*
* The repair itself is driven against a real temp directory tree, because the
* whole mechanism is "is this file actually there" and a mocked fs would only
* be testing the mock.
*/
const path = require('path');
const fs = require('fs');
const os = require('os');
describe('migration 177 — external_relpath from the media root (#1163)', () => {
let knex; let tmpDir; let mediaRoot; let migration;
/** Writes `bytes` bytes and returns the size, so fixtures can record it the
* way an import would have. */
const touch = async (rel, bytes = 8) => {
const full = path.join(mediaRoot, rel);
await fs.promises.mkdir(path.dirname(full), { recursive: true });
await fs.promises.writeFile(full, Buffer.alloc(bytes));
return bytes;
};
beforeAll(async () => {
tmpDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'picpeak-mig187-'));
mediaRoot = path.join(tmpDir, 'media');
await fs.promises.mkdir(mediaRoot, { recursive: true });
process.env.EXTERNAL_MEDIA_ROOT = mediaRoot;
// The service caches the root on first call, so it must not have been
// resolved before EXTERNAL_MEDIA_ROOT was set above.
jest.resetModules();
migration = require('../../migrations/core/177_external_relpath_from_root');
knex = require('knex')({
client: 'sqlite3',
connection: { filename: path.join(tmpDir, 'db.sqlite') },
useNullAsDefault: true,
});
});
afterAll(async () => {
if (knex) await knex.destroy();
await fs.promises.rm(tmpDir, { recursive: true, force: true }).catch(() => {});
delete process.env.EXTERNAL_MEDIA_ROOT;
});
beforeEach(async () => {
await knex.schema.dropTableIfExists('photos');
await knex.schema.dropTableIfExists('events');
await knex.schema.dropTableIfExists('app_settings');
await knex.schema.createTable('events', (t) => {
t.increments('id').primary();
t.string('external_path');
});
await knex.schema.createTable('photos', (t) => {
t.increments('id').primary();
t.integer('event_id');
t.string('external_relpath');
t.integer('size_bytes');
t.string('source_origin').defaultTo('managed');
});
await knex.schema.createTable('app_settings', (t) => {
t.increments('id').primary();
t.string('setting_key');
t.text('setting_value');
t.string('setting_type');
t.string('updated_at');
});
await fs.promises.rm(mediaRoot, { recursive: true, force: true });
await fs.promises.mkdir(mediaRoot, { recursive: true });
});
const relpaths = async () =>
(await knex('photos').orderBy('id', 'asc').select('external_relpath'))
.map((r) => r.external_relpath);
it('folds the base path into every row of a healthy event', async () => {
await touch('Trip/Leknes/a.jpg');
await touch('Trip/Leknes/b.jpg');
await knex('events').insert({ id: 1, external_path: 'Trip' });
await knex('photos').insert([
{ event_id: 1, external_relpath: 'Leknes/a.jpg', source_origin: 'external' },
{ event_id: 1, external_relpath: 'Leknes/b.jpg', source_origin: 'external' },
]);
await migration.up(knex);
expect(await relpaths()).toEqual(['Trip/Leknes/a.jpg', 'Trip/Leknes/b.jpg']);
});
it('repairs rows an earlier import had rebased', async () => {
// The reported shape: a parent imported first, a child imported second, so
// events.external_path is the child and the parent's rows resolve into a
// path that does not exist.
const oldSize = await touch('Trip/Leknes/old.jpg', 11); // from the first import
const newSize = await touch('Trip/Sub/new.jpg', 22); // from the second
await knex('events').insert({ id: 1, external_path: 'Trip/Sub' });
await knex('photos').insert([
{ event_id: 1, external_relpath: 'Leknes/old.jpg', size_bytes: oldSize, source_origin: 'external' },
{ event_id: 1, external_relpath: 'new.jpg', size_bytes: newSize, source_origin: 'external' },
]);
await migration.up(knex);
// The old row is placed where the file actually is; the new one keeps
// resolving exactly where it resolved before.
expect(await relpaths()).toEqual(['Trip/Leknes/old.jpg', 'Trip/Sub/new.jpg']);
});
it('refuses an ancestor whose file is a different size', async () => {
// The dangerous case: the row's own file was simply deleted, and an
// UNRELATED file one directory up happens to share its name. Adopting it
// would make downloads serve the wrong original — worse than a dead link.
await touch('Trip/photo.jpg', 999);
await knex('events').insert({ id: 1, external_path: 'Trip/Sub' });
await knex('photos').insert({
event_id: 1, external_relpath: 'photo.jpg', size_bytes: 42, source_origin: 'external',
});
await migration.up(knex);
expect(await relpaths()).toEqual(['Trip/Sub/photo.jpg']);
});
it('refuses an ancestor when the row records no size to check against', async () => {
// Nothing to verify provenance with, so the row stays where it resolves
// today rather than adopting a same-named stranger.
await touch('Trip/photo.jpg', 100);
await knex('events').insert({ id: 1, external_path: 'Trip/Sub' });
await knex('photos').insert({
event_id: 1, external_relpath: 'photo.jpg', size_bytes: null, source_origin: 'external',
});
await migration.up(knex);
expect(await relpaths()).toEqual(['Trip/Sub/photo.jpg']);
});
it('leaves nothing folded when a rewrite fails partway', async () => {
// Without a transaction, a crash between the first event's UPDATE and the
// marker leaves mixed formats behind — and the next run folds the already
// folded rows a second time, putting every original one directory deeper.
await touch('A/one.jpg');
await touch('B/two.jpg');
await knex('events').insert([
{ id: 1, external_path: 'A' },
{ id: 2, external_path: 'B' },
]);
await knex('photos').insert([
{ event_id: 1, external_relpath: 'one.jpg', source_origin: 'external' },
{ event_id: 2, external_relpath: 'two.jpg', source_origin: 'external' },
]);
// app_settings is written last, in the same transaction as the rewrites.
await knex.schema.dropTableIfExists('app_settings_backup');
await knex.raw('CREATE TRIGGER fail_marker BEFORE INSERT ON app_settings '
+ "BEGIN SELECT RAISE(ABORT, 'boom'); END");
await expect(migration.up(knex)).rejects.toThrow(/boom/);
await knex.raw('DROP TRIGGER fail_marker');
// Every row still base-relative, and no marker — so a retry is correct.
expect(await relpaths()).toEqual(['one.jpg', 'two.jpg']);
expect(await knex('app_settings').where('setting_key', 'external_relpath_root_relative').first())
.toBeUndefined();
});
it('removes the losing row when two paths converge, instead of stranding it', async () => {
// Trip/Sub/c.jpg imported once via `Trip` (as `Sub/c.jpg`) and once via
// `Trip/Sub` (as `c.jpg`). Both fold to the same path. Skipping the loser
// would leave it base-relative under a root-only resolver — pointing at
// <root>/c.jpg — with the marker claiming the conversion is complete.
const size = await touch('Trip/Sub/c.jpg', 33);
await knex('events').insert({ id: 1, external_path: 'Trip/Sub' });
await knex('photos').insert([
{ event_id: 1, external_relpath: 'Sub/c.jpg', size_bytes: size, source_origin: 'external' },
{ event_id: 1, external_relpath: 'c.jpg', size_bytes: size, source_origin: 'external' },
]);
await migration.up(knex);
const rows = await knex('photos').select('external_relpath');
expect(rows).toHaveLength(1);
expect(rows[0].external_relpath).toBe('Trip/Sub/c.jpg');
});
it('survives a final path that equals another row\'s current path', async () => {
// `photo.jpg` repairs to `Trip/photo.jpg` while the row already holding
// `Trip/photo.jpg` folds to `Trip/Sub/Trip/photo.jpg`. Every FINAL value is
// distinct, but a one-pass rewrite collides halfway through — and on
// Postgres that 23505 is misread by the migration runner as "already
// applied", leaving everything unconverted.
const a = await touch('Trip/photo.jpg', 11);
const b = await touch('Trip/Sub/Trip/photo.jpg', 22);
await knex('events').insert({ id: 1, external_path: 'Trip/Sub' });
await knex('photos').insert([
{ event_id: 1, external_relpath: 'photo.jpg', size_bytes: a, source_origin: 'external' },
{ event_id: 1, external_relpath: 'Trip/photo.jpg', size_bytes: b, source_origin: 'external' },
]);
await migration.up(knex);
expect(await relpaths()).toEqual(['Trip/photo.jpg', 'Trip/Sub/Trip/photo.jpg']);
});
it('does not re-prefix a row inserted while the probe was running', async () => {
// Phase 1 runs outside the transaction and can take minutes on a cold
// mount. An import finishing in that window writes an already
// root-relative row, which a `where event_id` bulk update would prefix a
// second time with the stale base.
await touch('Trip/a.jpg');
await knex('events').insert({ id: 1, external_path: 'Trip' });
await knex('photos').insert({ event_id: 1, external_relpath: 'a.jpg', source_origin: 'external' });
const { foldExternalRelpaths } = require('../../src/services/externalRelpathFold');
const realStat = fs.promises.stat;
let injected = false;
jest.spyOn(fs.promises, 'access').mockImplementation(async (...args) => {
if (!injected) {
injected = true;
await knex('photos').insert({
event_id: 1, external_relpath: 'Trip/late.jpg', source_origin: 'external',
});
}
return realStat(args[0]).then(() => undefined);
});
await foldExternalRelpaths(knex);
fs.promises.access.mockRestore();
expect((await relpaths()).sort()).toEqual(['Trip/a.jpg', 'Trip/late.jpg']);
});
it('leaves a row it cannot place resolving where it resolves today', async () => {
// Never guess below current behaviour: a file that is genuinely gone must
// not have its path rewritten to some other file that happens to exist.
await touch('Trip/Sub/present.jpg');
await knex('events').insert({ id: 1, external_path: 'Trip/Sub' });
await knex('photos').insert([
{ event_id: 1, external_relpath: 'present.jpg', source_origin: 'external' },
{ event_id: 1, external_relpath: 'vanished.jpg', source_origin: 'external' },
]);
await migration.up(knex);
expect(await relpaths()).toEqual(['Trip/Sub/present.jpg', 'Trip/Sub/vanished.jpg']);
});
it('folds without repairing when the media root is unmounted', async () => {
// An unmounted share leaves the mountpoint as an empty directory, so every
// file looks missing. Repairing off that signal would move every original
// on a perfectly healthy install.
await knex('events').insert({ id: 1, external_path: 'Trip/Sub' });
await knex('photos').insert([
{ event_id: 1, external_relpath: 'Leknes/a.jpg', source_origin: 'external' },
]);
// mediaRoot is empty — see beforeEach.
await migration.up(knex);
expect(await relpaths()).toEqual(['Trip/Sub/Leknes/a.jpg']);
});
it('leaves managed rows alone', async () => {
await touch('Trip/a.jpg');
await knex('events').insert({ id: 1, external_path: 'Trip' });
await knex('photos').insert([
{ event_id: 1, external_relpath: null, source_origin: 'managed' },
{ event_id: 1, external_relpath: 'a.jpg', source_origin: 'external' },
]);
await migration.up(knex);
expect(await relpaths()).toEqual([null, 'Trip/a.jpg']);
});
it('leaves an event with no base path alone — its rows are already root-relative', async () => {
await touch('a.jpg');
await knex('events').insert({ id: 1, external_path: null });
await knex('photos').insert({ event_id: 1, external_relpath: 'a.jpg', source_origin: 'external' });
await migration.up(knex);
expect(await relpaths()).toEqual(['a.jpg']);
});
it('folds each event with its own base', async () => {
await touch('A/one.jpg');
await touch('B/two.jpg');
await knex('events').insert([
{ id: 1, external_path: 'A' },
{ id: 2, external_path: 'B' },
]);
await knex('photos').insert([
{ event_id: 1, external_relpath: 'one.jpg', source_origin: 'external' },
{ event_id: 2, external_relpath: 'two.jpg', source_origin: 'external' },
]);
await migration.up(knex);
expect(await relpaths()).toEqual(['A/one.jpg', 'B/two.jpg']);
});
it('tolerates a base path with stray slashes', async () => {
await touch('Trip/a.jpg');
await knex('events').insert({ id: 1, external_path: '/Trip/' });
await knex('photos').insert({ event_id: 1, external_relpath: 'a.jpg', source_origin: 'external' });
await migration.up(knex);
expect(await relpaths()).toEqual(['Trip/a.jpg']);
});
it('does not fold twice when run again', async () => {
// The failure this guards is total: every original on the install moves one
// directory deeper, and there is no undo.
await touch('Trip/a.jpg');
await knex('events').insert({ id: 1, external_path: 'Trip' });
await knex('photos').insert({ event_id: 1, external_relpath: 'a.jpg', source_origin: 'external' });
await migration.up(knex);
await migration.up(knex);
expect(await relpaths()).toEqual(['Trip/a.jpg']);
});
it('does not fold twice when the base repeats in the relpath', async () => {
// The inference this migration deliberately does NOT use: `Trip/x.jpg`
// under base `Trip` already "starts with the base", but has not been
// folded — it is a subfolder that shares its parent's name.
await touch('Trip/Trip/x.jpg');
await knex('events').insert({ id: 1, external_path: 'Trip' });
await knex('photos').insert({ event_id: 1, external_relpath: 'Trip/x.jpg', source_origin: 'external' });
await migration.up(knex);
expect(await relpaths()).toEqual(['Trip/Trip/x.jpg']);
});
it('rollback does not clear the marker, so a re-run cannot double-fold', async () => {
await touch('Trip/a.jpg');
await knex('events').insert({ id: 1, external_path: 'Trip' });
await knex('photos').insert({ event_id: 1, external_relpath: 'a.jpg', source_origin: 'external' });
await migration.up(knex);
await migration.down(knex);
await migration.up(knex);
expect(await relpaths()).toEqual(['Trip/a.jpg']);
});
it('no-ops before 041 has added the column', async () => {
await knex.schema.dropTableIfExists('photos');
await knex.schema.createTable('photos', (t) => { t.increments('id').primary(); });
await expect(migration.up(knex)).resolves.toBeUndefined();
});
});
@@ -0,0 +1,83 @@
/**
* Legacy preview keys must not survive the encoder change.
*
* The old generator kept the SOURCE basename verbatim while always writing
* JPEG, so a `.webp` upload produced `preview_shot.webp` holding a JPEG. The
* route now derives Content-Type from the key, and sets `nosniff` — so that
* legacy object would be announced as image/webp and render as a broken image.
*/
const path = require('path');
const fs = require('fs');
const os = require('os');
const migration = require('../../migrations/core/178_reset_legacy_preview_paths');
describe('migration 178 — legacy preview keys (#1166 follow-up)', () => {
let knex; let tmpDir;
beforeAll(async () => {
tmpDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'picpeak-mig188-'));
knex = require('knex')({
client: 'sqlite3',
connection: { filename: path.join(tmpDir, 'db.sqlite') },
useNullAsDefault: true,
});
});
afterAll(async () => {
if (knex) await knex.destroy();
await fs.promises.rm(tmpDir, { recursive: true, force: true }).catch(() => {});
});
beforeEach(async () => {
await knex.schema.dropTableIfExists('photos');
await knex.schema.createTable('photos', (t) => {
t.increments('id').primary();
t.string('preview_path');
t.string('thumbnail_path');
});
});
it('clears the mislabelled .webp keys that would render broken', async () => {
await knex('photos').insert({ preview_path: 'previews/preview_shot.webp' });
await migration.up(knex);
expect((await knex('photos').first()).preview_path).toBeNull();
});
it('clears .jpg keys too, because a byte-correct one can still be flattened', async () => {
// A legacy .jpg key is valid JPEG, but it may be a flattened rendition of a
// transparent or animated source, and nothing in the key says so. One lazy
// regeneration is cheaper than reasoning about which of them lied.
await knex('photos').insert([
{ preview_path: 'previews/preview_a.jpg' },
{ preview_path: 'previews/preview_b.png' },
]);
await migration.up(knex);
expect(await knex('photos').whereNotNull('preview_path').count('* as c').first()).toEqual({ c: 0 });
});
it('leaves thumbnails alone — they are a different cache', async () => {
await knex('photos').insert({ preview_path: 'previews/p.jpg', thumbnail_path: 'thumbnails/t.jpg' });
await migration.up(knex);
expect((await knex('photos').first()).thumbnail_path).toBe('thumbnails/t.jpg');
});
it('is idempotent and safe with nothing to clear', async () => {
await migration.up(knex);
await expect(migration.up(knex)).resolves.toBeUndefined();
});
it('no-ops before 104 has added the column', async () => {
await knex.schema.dropTableIfExists('photos');
await knex.schema.createTable('photos', (t) => { t.increments('id').primary(); });
await expect(migration.up(knex)).resolves.toBeUndefined();
});
});
@@ -0,0 +1,223 @@
/**
* Regression test: deleting an event must remove its stored objects.
*
* deleteEventCascade() cleaned up the local filesystem only (#608). On an
* S3/R2 storage backend that cleanup is a no-op, so every deleted gallery
* left its originals and derived tiers in the bucket — unreferenced,
* invisible in the UI, and billed forever. Measured on a v3.45.16 install
* against Cloudflare R2: deleting a 403-photo event changed the bucket
* object count by exactly zero.
*
* The keys must be collected BEFORE the transaction deletes the photo
* rows, because afterwards nothing knows which objects were this event's.
*/
const os = require('os');
const path = require('path');
// The cascade runs a real `fs.rm(..., { recursive: true })` over
// {STORAGE_PATH}/events/{active,archived}/{slug}. Point that at a throwaway
// directory before requiring the module under test — the default resolves
// into the working tree.
process.env.STORAGE_PATH = path.join(os.tmpdir(), 'picpeak-cascade-storage-test');
const mockStorage = { delete: jest.fn().mockResolvedValue(undefined) };
const mockEvent = {
id: 42,
slug: 'other-demo-2026-01-01',
event_name: 'Demo',
source_mode: 'managed',
// Written through the backend by archiveService, so it is a bucket object
// and the fs.unlink in the cascade never touched it on S3.
archive_path: 'archives/other-demo-2026-01-01.zip',
// The pre-built "Download All" zip. Lives under the event prefix, so the
// recursive fs.rm covers it on local disk and nothing covers it on S3.
download_zip_path: 'events/active/other-demo-2026-01-01/.download-cache/all.zip',
};
const mockPhotos = [
{
id: 1,
path: 'other-demo-2026-01-01/photo_one.jpg',
thumbnail_path: 'thumbnails/thumb_aaa_photo_one.jpg',
hero_path: null,
preview_path: 'previews/prev_aaa_photo_one.jpg',
watermark_path: 'watermarked/wm_aaa_photo_one.jpg',
source_origin: 'managed',
},
{
id: 2,
path: 'other-demo-2026-01-01/photo_two.jpg',
thumbnail_path: 'thumbnails/thumb_bbb_photo_two.jpg',
hero_path: null,
preview_path: null,
watermark_path: null,
source_origin: 'managed',
},
{
// External photos live outside the managed backend and must be left alone.
id: 3,
path: 'ignored.jpg',
thumbnail_path: null,
hero_path: null,
preview_path: null,
watermark_path: null,
source_origin: 'external',
},
];
let mockPhotoRowsDeleted = false;
let mockJobRowsDeleted = false;
// Photos in OTHER events that share a canonical derivative key with this one.
let mockSharedDerivatives = [];
// The shared-derivative probe: db('photos').whereNot(...).where(cb).select(...)
const sharedProbe = {
where: () => sharedProbe,
whereIn: () => sharedProbe,
orWhereIn: () => sharedProbe,
select: async () => mockSharedDerivatives,
};
function mockMakeDb() {
const table = (name) => {
const chain = {
where: () => chain,
first: async () => (name === 'events' ? mockEvent : undefined),
whereNotNull: () => chain,
whereNot: () => sharedProbe,
orWhereIn: () => chain,
whereIn: () => chain,
select: async () => {
if (name === 'photos') {
// The whole point: if this runs after the transaction, the rows
// are gone and we would collect nothing.
return mockPhotoRowsDeleted ? [] : mockPhotos;
}
return [];
},
del: async () => {
if (name === 'photos') mockPhotoRowsDeleted = true;
if (name === 'download_jobs') mockJobRowsDeleted = true;
return 1;
},
};
return chain;
};
// #1132 guards the merge-dismissals delete behind a hasTable check.
table.schema = { hasTable: async () => false };
table.transaction = async (cb) => cb(table);
return table;
}
jest.mock('../../src/database/db', () => ({
db: mockMakeDb(),
logActivity: jest.fn().mockResolvedValue(undefined),
}));
jest.mock('../../src/services/storage', () => ({
getStorage: () => mockStorage,
}));
const { deleteEventCascade } = require('../../src/routes/adminEvents/helpers');
describe('deleteEventCascade — storage cleanup', () => {
beforeEach(() => {
mockStorage.delete.mockClear();
mockPhotoRowsDeleted = false;
mockJobRowsDeleted = false;
mockSharedDerivatives = [];
});
it('deletes originals and every derived tier from the storage backend', async () => {
await deleteEventCascade(42, { id: 1, username: 'admin' });
const deleted = mockStorage.delete.mock.calls.map(([key]) => key);
expect(deleted).toEqual(expect.arrayContaining([
'events/active/other-demo-2026-01-01/photo_one.jpg',
'events/active/other-demo-2026-01-01/photo_two.jpg',
'thumbnails/thumb_aaa_photo_one.jpg',
'thumbnails/thumb_bbb_photo_two.jpg',
'previews/prev_aaa_photo_one.jpg',
]));
});
it('deletes pre-generated watermarks and the archive zip', async () => {
await deleteEventCascade(42, { id: 1, username: 'admin' });
const deleted = mockStorage.delete.mock.calls.map(([key]) => key);
// Both are storage-backend objects that only fs.unlink ever touched, so
// both survived an event delete on S3.
expect(deleted).toEqual(expect.arrayContaining([
'watermarked/wm_aaa_photo_one.jpg',
'archives/other-demo-2026-01-01.zip',
]));
});
it('deletes the Download All cache, which only fs.rm ever covered', async () => {
await deleteEventCascade(42, { id: 1, username: 'admin' });
const deleted = mockStorage.delete.mock.calls.map(([key]) => key);
// Sits under events/active/{slug}/.download-cache/ — swept by the
// recursive fs.rm on local disk, invisible to it on S3 where the prefix
// is not a directory. Gallery-sized. (download_jobs is main-only, so the
// per-job archives main also sweeps have no counterpart here.)
expect(deleted).toContain(
'events/active/other-demo-2026-01-01/.download-cache/all.zip'
);
});
it('leaves a derivative alone when another event still points at it', async () => {
// Canonical thumbnail/hero/preview keys are not event-scoped — the
// basename is the photo's filename, and filenames are not unique across
// events. Deleting one a surviving gallery still references would blank
// its tile.
mockSharedDerivatives = [{
thumbnail_path: 'thumbnails/thumb_aaa_photo_one.jpg',
hero_path: null,
preview_path: null,
watermark_path: null,
}];
await deleteEventCascade(42, { id: 1, username: 'admin' });
const deleted = mockStorage.delete.mock.calls.map(([key]) => key);
expect(deleted).not.toContain('thumbnails/thumb_aaa_photo_one.jpg');
// The originals are slug-scoped and must still go.
expect(deleted).toContain('events/active/other-demo-2026-01-01/photo_one.jpg');
// So must a derivative nobody else claims.
expect(deleted).toContain('thumbnails/thumb_bbb_photo_two.jpg');
});
it('never asks the backend to delete the same key twice', async () => {
await deleteEventCascade(42, { id: 1, username: 'admin' });
const managed = mockStorage.delete.mock.calls
.map(([key]) => key)
.filter((key) => !key.startsWith('thumbnails/thumb_w') && !key.startsWith('previews/preview_w'));
expect(managed).toEqual([...new Set(managed)]);
});
it('leaves external/reference photos in place', async () => {
await deleteEventCascade(42, { id: 1, username: 'admin' });
const deleted = mockStorage.delete.mock.calls.map(([key]) => key);
expect(deleted).not.toEqual(expect.arrayContaining(['ignored.jpg']));
expect(deleted).not.toEqual(expect.arrayContaining(['events/active/ignored.jpg']));
});
it('still completes the delete when the storage backend throws', async () => {
mockStorage.delete.mockRejectedValue(new Error('bucket unreachable'));
await expect(deleteEventCascade(42, { id: 1, username: 'admin' }))
.resolves.toEqual({ id: 42, name: 'Demo' });
mockStorage.delete.mockResolvedValue(undefined);
});
});
@@ -135,9 +135,9 @@ function signAdminToken({ id = 1, username = 'admin', iat, exp }) {
);
}
function signGalleryToken({ eventId = 100, eventSlug = 'wedding' } = {}) {
function signGalleryToken({ eventId = 100, eventSlug = 'wedding', ...extra } = {}) {
return jwt.sign(
{ eventId, eventSlug, type: 'gallery' },
{ eventId, eventSlug, type: 'gallery', ...extra },
process.env.JWT_SECRET,
{ expiresIn: '1h', issuer: 'picpeak-auth' }
);
@@ -288,6 +288,56 @@ describe('GET /auth/session — symmetry with protected middleware', () => {
expect(res.body.valid).toBe(true);
});
/**
* What KIND of gallery session this is (#1149).
*
* The frontend used to keep this in sessionStorage, which is per-TAB while
* the cookie is per-browser: a gallery reopened in a second tab lost
* 'client' even though the backend still served it as one, and the UI hid
* the only control that clears the privileged cookie. Reported from the
* token so a restored session knows what it actually is.
*/
describe('gallery session kind', () => {
beforeEach(() => {
fakeDb.events.push({
id: 100,
slug: 'wedding',
is_active: true,
is_archived: false,
expires_at: new Date(Date.now() + 86400_000),
});
});
it('reports a PIN-client session as client', async () => {
const res = await request(makeApp())
.get('/auth/session?slug=wedding')
.set('Authorization', `Bearer ${signGalleryToken({ accessLevel: 'client' })}`);
expect(res.body.valid).toBe(true);
expect(res.body.accessLevel).toBe('client');
expect(res.body.viaCustomer).toBe(false);
});
it('reports a customer-portal session, which looks like a guest', async () => {
const res = await request(makeApp())
.get('/auth/session?slug=wedding')
.set('Authorization', `Bearer ${signGalleryToken({ via: 'customer', customerId: 7 })}`);
expect(res.body.valid).toBe(true);
expect(res.body.accessLevel).toBe('guest');
expect(res.body.viaCustomer).toBe(true);
});
it('reports a plain guest as neither', async () => {
// The flags have to discriminate, or they would just hand every visitor
// a Logout button back.
const res = await request(makeApp())
.get('/auth/session?slug=wedding')
.set('Authorization', `Bearer ${signGalleryToken()}`);
expect(res.body.valid).toBe(true);
expect(res.body.accessLevel).toBe('guest');
expect(res.body.viaCustomer).toBe(false);
});
});
it('returns valid:false when the token is revoked', async () => {
fakeDb.adminUsers.push({
id: 1,
@@ -127,7 +127,44 @@ describe('dashboard scoping (GHSA-c2jj / gqx7 / jhcf)', () => {
expect(res.status).toBe(200);
expect(Number(res.body.totalEvents)).toBe(1);
expect(Number(res.body.totalPhotos)).toBe(1);
expect(Number(res.body.storageUsed)).toBe(1000);
// The catalogued original bytes — this is what carries the per-event
// scoping, and what `storageUsed` reported before #1164.
expect(Number(res.body.catalogedBytes)).toBe(1000);
});
it('/stats reports disk usage unscoped, because disk is not per-event', async () => {
// storageUsed is a measurement of the storage root (#1164), so it is the
// same number for every admin by design. Pinned so a future reviewer
// reading "everything on this endpoint is scoped" does not turn it into a
// sum of this editor's photos again — which is the bug that was fixed.
const res = await request(app)
.get('/api/admin/dashboard/stats')
.set('Authorization', `Bearer ${editorToken}`);
expect(res.status).toBe(200);
expect(res.body.storageUsed).not.toBe(1000);
expect(res.body).toHaveProperty('storageBreakdown');
});
it('/stats reports the catalogued figure on an S3 backend, not a near-zero disk walk', async () => {
// STORAGE_PATH holds only incidental local files when objects live in a
// bucket, so walking it would report near-zero and drag the soft-limit
// recommendation with it.
const prev = process.env.STORAGE_BACKEND;
process.env.STORAGE_BACKEND = 's3';
try {
const res = await request(app)
.get('/api/admin/dashboard/stats')
.set('Authorization', `Bearer ${editorToken}`);
expect(res.status).toBe(200);
expect(res.body.storageUsed).toBeNull();
expect(res.body.storageMeasurement).toBe('catalog');
expect(Number(res.body.catalogedBytes)).toBe(1000);
} finally {
if (prev === undefined) delete process.env.STORAGE_BACKEND;
else process.env.STORAGE_BACKEND = prev;
}
});
it('/analytics does not expose a foreign gallery name or slug', async () => {
@@ -0,0 +1,278 @@
/**
* Single-photo gallery downloads must go through the storage backend (#1048).
*
* `GET /api/gallery/:slug/download/:photoId` resolved a LOCAL filesystem path
* unconditionally and handed it to res.sendFile. On an S3/R2 deployment
* managed photos never exist on local disk, so every per-photo download 404'd
* with ENOENT — while download-all and secure-images worked fine, because they
* already went through getStorage(). The gallery looks healthy until a guest
* clicks the download button on a single photo.
*
* The local branch is pinned just as hard: sendFile emits Content-Length,
* Accept-Ranges, ETag and Last-Modified and answers Range with a 206. Routing
* local installs through a bare stream.pipe(res) to share one code path would
* silently drop all of that, and a resumed download would append a second full
* body onto the partial file.
*/
const path = require('path');
const fs = require('fs');
const os = require('os');
process.env.NODE_ENV = 'test';
process.env.TEST_DATABASE_PATH = path.join(
fs.mkdtempSync(path.join(os.tmpdir(), 'picpeak-dl-')), 'db.sqlite',
);
process.env.JWT_SECRET = process.env.JWT_SECRET || 'download-test-secret';
process.env.STORAGE_PATH = fs.mkdtempSync(path.join(os.tmpdir(), 'picpeak-dl-storage-'));
const { Readable } = require('stream');
const SLUG = 'download-gallery';
const FILENAME = 'original.jpg';
// Deliberately not written to disk anywhere: if the route reads the
// filesystem instead of the backend, it cannot produce these bytes.
const mockObjectBody = Buffer.from('S3-ONLY-ORIGINAL-BYTES-not-on-local-disk');
const mockBackendKind = { value: 's3' };
const mockStorage = {
kind: () => mockBackendKind.value,
stat: jest.fn(async () => ({ size: mockObjectBody.length, mtime: new Date('2026-08-20T10:00:00Z') })),
get: jest.fn(async () => Readable.from([mockObjectBody])),
getRange: jest.fn(async (key, start, end) => Readable.from([mockObjectBody.subarray(start, end + 1)])),
delete: jest.fn(async () => undefined),
exists: jest.fn(async () => true),
};
jest.mock('../../src/services/storage', () => ({
getStorage: () => mockStorage,
initStorage: async () => mockStorage,
}));
const request = require('supertest');
const express = require('express');
const cookieParser = require('cookie-parser');
const { bootCrmDb, seedMinimal } = require('../integration/helpers/crmDb');
describe('single-photo download through the storage backend (#1048)', () => {
let db; let cleanup; let app; let eventId; let photoId;
beforeAll(async () => {
({ db, cleanup } = await bootCrmDb());
await seedMinimal(db);
const ev = await db('events').insert({
slug: SLUG,
event_type: 'wedding',
event_name: 'Downloads',
event_date: '2026-08-01',
host_email: 'h@example.com',
admin_email: 'a@example.com',
password_hash: 'x',
share_link: `/gallery/${SLUG}/s`,
share_token: 'download-share',
expires_at: new Date(Date.now() + 7 * 864e5).toISOString(),
is_active: 1,
is_archived: 0,
is_draft: 0,
require_password: 0,
allow_downloads: 1,
created_at: new Date().toISOString(),
}).returning('id');
eventId = ev[0]?.id ?? ev[0];
const row = await db('photos').insert({
event_id: eventId,
filename: FILENAME,
path: `${SLUG}/${FILENAME}`,
type: 'individual',
source_origin: 'managed',
mime_type: 'image/jpeg',
uploaded_at: new Date().toISOString(),
}).returning('id');
photoId = row[0]?.id ?? row[0];
app = express();
app.use(express.json());
app.use(cookieParser());
app.use('/api/gallery', require('../../src/routes/gallery'));
}, 120000);
afterAll(async () => { if (cleanup) await cleanup(); });
beforeEach(() => {
mockBackendKind.value = 's3';
mockStorage.get.mockClear();
mockStorage.getRange.mockClear();
});
it('streams the stored object instead of 404ing on a local path', async () => {
const res = await request(app)
.get(`/api/gallery/${SLUG}/download/${photoId}`)
.buffer(true)
.parse((response, cb) => {
const chunks = [];
response.on('data', (c) => chunks.push(c));
response.on('end', () => cb(null, Buffer.concat(chunks)));
});
expect(res.status).toBe(200);
// The bytes only exist in the backend — proof it did not read the disk.
expect(res.body.equals(mockObjectBody)).toBe(true);
expect(mockStorage.get).toHaveBeenCalledWith(`events/active/${SLUG}/${FILENAME}`);
// Never written locally, so a filesystem read could not have served this.
expect(fs.existsSync(path.join(process.env.STORAGE_PATH, 'events/active', SLUG, FILENAME))).toBe(false);
});
it('sends Content-Length so the browser can show download progress', async () => {
const res = await request(app).get(`/api/gallery/${SLUG}/download/${photoId}`);
expect(res.headers['content-length']).toBe(String(mockObjectBody.length));
expect(res.headers['accept-ranges']).toBe('bytes');
expect(res.headers['content-disposition']).toContain(FILENAME);
});
it('answers a Range request with 206 and only the requested bytes', async () => {
const res = await request(app)
.get(`/api/gallery/${SLUG}/download/${photoId}`)
.set('Range', 'bytes=0-9')
.buffer(true)
.parse((response, cb) => {
const chunks = [];
response.on('data', (c) => chunks.push(c));
response.on('end', () => cb(null, Buffer.concat(chunks)));
});
expect(res.status).toBe(206);
expect(res.headers['content-range']).toBe(`bytes 0-9/${mockObjectBody.length}`);
expect(res.headers['content-length']).toBe('10');
expect(res.body.equals(mockObjectBody.subarray(0, 10))).toBe(true);
expect(mockStorage.getRange).toHaveBeenCalledWith(`events/active/${SLUG}/${FILENAME}`, 0, 9);
});
it('ignores a malformed Range rather than emitting a nonsense 206', async () => {
const res = await request(app)
.get(`/api/gallery/${SLUG}/download/${photoId}`)
.set('Range', 'bytes=abc-def');
expect(res.status).toBe(200);
expect(res.headers['content-range']).toBeUndefined();
});
it('404s cleanly when the object is missing from the backend', async () => {
mockStorage.stat.mockResolvedValueOnce(null);
const res = await request(app).get(`/api/gallery/${SLUG}/download/${photoId}`);
expect(res.status).toBe(404);
// The error must not inherit the image headers staged for a successful
// download, or the browser saves a .jpg containing JSON.
expect(res.headers['content-type']).toMatch(/json/);
expect(res.headers['content-disposition']).toBeUndefined();
});
it('keeps res.sendFile on a local backend rather than a bare pipe', async () => {
mockBackendKind.value = 'local';
const abs = path.join(process.env.STORAGE_PATH, 'events/active', SLUG, FILENAME);
fs.mkdirSync(path.dirname(abs), { recursive: true });
fs.writeFileSync(abs, 'local-disk-bytes');
const res = await request(app).get(`/api/gallery/${SLUG}/download/${photoId}`);
expect(res.status).toBe(200);
expect(mockStorage.get).not.toHaveBeenCalled();
// sendFile's signature: conditional-request headers a raw pipe never sets.
expect(res.headers.etag).toBeDefined();
expect(res.headers['last-modified']).toBeDefined();
fs.rmSync(abs, { force: true });
});
it('does not serve a partial body when the If-Range validator is stale', async () => {
// The object was replaced since the client's last attempt. Answering 206
// from the new bytes would let it splice two versions into one file.
const res = await request(app)
.get(`/api/gallery/${SLUG}/download/${photoId}`)
.set('Range', 'bytes=0-9')
.set('If-Range', new Date('2020-01-01T00:00:00Z').toUTCString());
expect(res.status).toBe(200);
expect(res.headers['content-range']).toBeUndefined();
expect(res.headers['content-length']).toBe(String(mockObjectBody.length));
});
it('still serves 206 when the If-Range validator matches', async () => {
const res = await request(app)
.get(`/api/gallery/${SLUG}/download/${photoId}`)
.set('Range', 'bytes=0-9')
.set('If-Range', new Date('2026-08-20T10:00:00Z').toUTCString());
expect(res.status).toBe(206);
expect(res.headers['content-range']).toBe(`bytes 0-9/${mockObjectBody.length}`);
});
it('errors cleanly when the object vanishes between stat and get', async () => {
// HeadObject succeeding does not mean GetObject will — a concurrent
// delete lands here. The staged image headers must not escape with it.
const gone = new Error('NoSuchKey');
gone.name = 'NoSuchKey';
mockStorage.get.mockRejectedValueOnce(gone);
const res = await request(app).get(`/api/gallery/${SLUG}/download/${photoId}`);
expect(res.status).toBe(404);
expect(res.headers['content-type']).toMatch(/json/);
expect(res.headers['content-disposition']).toBeUndefined();
});
it('does not send 206 headers before the range fetch can fail', async () => {
// writeHead(206) before the await would make this ERR_HTTP_HEADERS_SENT.
mockStorage.getRange.mockRejectedValueOnce(new Error('connection reset'));
const res = await request(app)
.get(`/api/gallery/${SLUG}/download/${photoId}`)
.set('Range', 'bytes=0-9');
expect(res.status).toBe(500);
expect(res.headers['content-type']).toMatch(/json/);
expect(res.headers['content-range']).toBeUndefined();
});
it('answers HEAD from stat instead of draining the object out of S3', async () => {
const before = (await db('photos').where('id', photoId).first()).download_count || 0;
const logsBefore = (await db('access_logs').where({ photo_id: photoId, action: 'download' })).length;
const res = await request(app).head(`/api/gallery/${SLUG}/download/${photoId}`);
expect(res.status).toBe(200);
expect(res.headers['content-length']).toBe(String(mockObjectBody.length));
expect(res.headers['accept-ranges']).toBe('bytes');
// The whole point: no egress for a metadata probe.
expect(mockStorage.get).not.toHaveBeenCalled();
expect(mockStorage.getRange).not.toHaveBeenCalled();
// And no side effects: a probe is not a download.
const after = (await db('photos').where('id', photoId).first()).download_count || 0;
expect(after).toBe(before);
const logsAfter = (await db('access_logs').where({ photo_id: photoId, action: 'download' })).length;
expect(logsAfter).toBe(logsBefore);
});
it('returns a clean error when the range stream dies before its first chunk', async () => {
// Resolves, then errors — writeHead would already have committed the 206,
// leaving a connection reset as the only possible outcome.
const { Readable: R } = require('stream');
mockStorage.getRange.mockImplementationOnce(async () => {
const dead = new R({ read() { this.destroy(new Error('socket hang up')); } });
return dead;
});
const res = await request(app)
.get(`/api/gallery/${SLUG}/download/${photoId}`)
.set('Range', 'bytes=0-9');
expect(res.status).toBe(500);
expect(res.headers['content-type']).toMatch(/json/);
expect(res.headers['content-range']).toBeUndefined();
});
});
@@ -0,0 +1,170 @@
/**
* "Date Taken" ordering across SQLite's storage classes (#1172).
*
* photos.captured_at does not hold one type on SQLite. Three writers put three
* different things in it:
*
* integer managed uploads — photoProcessor.js:441 hands knex a Date, which
* the sqlite3 binding stores as epoch milliseconds
* text external imports and the capture-date backfill, which write
* ISO-8601 ('2026-06-03T01:15:00.000Z')
* null no capture date, so the sort falls through to uploaded_at —
* itself text, in knex's 'YYYY-MM-DD HH:MM:SS' shape
*
* A plain COALESCE over that mixture is not an ordering. SQLite sorts INTEGER
* before TEXT unconditionally, so every managed photo carrying EXIF came back
* ahead of every photo that did not, whatever the dates said. And among the
* text values 'T' (0x54) outranks the space (0x20), so a same-day ISO 01:15
* sorted behind a fallback 23:00.
*
* Both failures predate #1172 — the first needs only two managed photos — but
* the sort is what that issue is about, so they are fixed and pinned here.
* Every test below fails on the unfixed ORDER BY.
*/
const path = require('path');
const fs = require('fs');
const os = require('os');
process.env.NODE_ENV = 'test';
process.env.TEST_DATABASE_PATH = path.join(
fs.mkdtempSync(path.join(os.tmpdir(), 'picpeak-capsort-')), 'db.sqlite',
);
process.env.JWT_SECRET = process.env.JWT_SECRET || 'capsort-test-secret';
process.env.STORAGE_PATH = fs.mkdtempSync(path.join(os.tmpdir(), 'picpeak-capsort-storage-'));
const request = require('supertest');
const express = require('express');
const cookieParser = require('cookie-parser');
const { bootCrmDb, seedMinimal } = require('../integration/helpers/crmDb');
const SLUG = 'capsort-gallery';
describe('capture-date ordering on SQLite (#1172)', () => {
let db; let cleanup; let app; let eventId;
// Managed uploads store an epoch-millisecond INTEGER, because
// photoProcessor.js:441 hands knex a Date and the sqlite3 binding converts
// it. That conversion cannot be reproduced from inside jest — there the
// binding's type dispatch misses sandbox-created Dates and writes the string
// "[object Object]" instead (CLAUDE.md). Verified outside jest: a Date lands
// as {"c":1830211200000,"ty":"integer"}. So these tests write the integer
// production would have written, rather than a Date that jest mangles.
const managed = (iso) => new Date(iso).getTime();
const addPhoto = async (filename, capturedAt, uploadedAt) => {
const row = await db('photos').insert({
event_id: eventId,
filename,
path: `${SLUG}/${filename}`,
type: 'individual',
captured_at: capturedAt,
uploaded_at: uploadedAt,
}).returning('id');
return row[0]?.id ?? row[0];
};
const orderedFilenames = async (order = 'asc') => {
const res = await request(app).get(`/api/gallery/${SLUG}/photos?sort=capture_date&order=${order}`);
expect(res.status).toBe(200);
return res.body.photos.map((p) => p.filename);
};
beforeAll(async () => {
({ db, cleanup } = await bootCrmDb());
await seedMinimal(db);
const ev = await db('events').insert({
slug: SLUG,
event_type: 'wedding',
event_name: 'Capture Sort',
event_date: '2026-08-01',
host_email: 'h@example.com',
admin_email: 'a@example.com',
password_hash: 'x',
share_link: `/gallery/${SLUG}/s`,
share_token: 'capsort-share',
expires_at: new Date(Date.now() + 7 * 864e5).toISOString(),
is_active: 1,
is_archived: 0,
is_draft: 0,
require_password: 0,
created_at: new Date().toISOString(),
}).returning('id');
eventId = ev[0]?.id ?? ev[0];
app = express();
app.use(express.json());
app.use(cookieParser());
app.use('/api/gallery', require('../../src/routes/gallery'));
}, 120000);
afterAll(async () => { if (cleanup) await cleanup(); });
beforeEach(async () => { await db('photos').where({ event_id: eventId }).del(); });
test('the fixture really does put three storage classes in one column', async () => {
expect(['sqlite3', 'better-sqlite3']).toContain(db.client.config.client);
await addPhoto('m.jpg', managed('2026-06-03T01:15:00Z'), '2026-01-01 00:00:00');
await addPhoto('e.jpg', '2020-01-01T00:00:00.000Z', '2026-01-01 00:00:00');
await addPhoto('n.jpg', null, '2026-01-01 00:00:00');
const rows = await db.raw('select filename, typeof(captured_at) as t from photos order by filename');
const byName = Object.fromEntries((rows.rows || rows).map((r) => [r.filename, r.t]));
// Exactly the mixture that made COALESCE meaningless.
expect(byName).toEqual({ 'm.jpg': 'integer', 'e.jpg': 'text', 'n.jpg': 'null' });
});
test('a managed EXIF date does not outrank an earlier one stored as text', async () => {
// The pre-existing failure, reachable with managed photos alone: integer
// beat text regardless of the dates, so this came back exactly reversed.
await addPhoto('managed-2027.jpg', managed('2027-12-31T00:00:00Z'), '2026-01-01 00:00:00');
await addPhoto('external-2020.jpg', '2020-01-01T00:00:00.000Z', '2026-01-01 00:00:00');
expect(await orderedFilenames('asc')).toEqual(['external-2020.jpg', 'managed-2027.jpg']);
expect(await orderedFilenames('desc')).toEqual(['managed-2027.jpg', 'external-2020.jpg']);
});
test('a photo with no capture date sorts by its upload time, not ahead of everything', async () => {
await addPhoto('has-exif-2027.jpg', managed('2027-12-31T00:00:00Z'), '2027-12-31 00:00:00');
await addPhoto('no-exif-2020.jpg', null, '2020-01-01 00:00:00');
expect(await orderedFilenames('asc')).toEqual(['no-exif-2020.jpg', 'has-exif-2027.jpg']);
});
test('an ISO capture time and a fallback upload time compare by clock, not by separator', async () => {
// Same day: 'T' vs ' ' decided this before, so 01:15 sorted after 23:00.
await addPhoto('iso-0115.jpg', '2026-06-03T01:15:00.000Z', '2026-06-03 05:00:00');
await addPhoto('fallback-2300.jpg', null, '2026-06-03 23:00:00');
expect(await orderedFilenames('asc')).toEqual(['iso-0115.jpg', 'fallback-2300.jpg']);
});
test('an epoch-integer uploaded_at is compared as a date, not as its digits', async () => {
// uploaded_at is not always text either: a legacy archive restore leaves
// epoch milliseconds in it (a .picpeak restore from an install that stored them that way).
// Reading that with substr() would have compared the string '1830297600000'
// against '2020-01-01 00:00:00', putting the 2028 row first.
await addPhoto('epoch-upload-2028.jpg', null, new Date('2028-01-01T00:00:00Z').getTime());
await addPhoto('captured-2020.jpg', managed('2020-01-01T00:00:00Z'), '2020-01-01 00:00:00');
const [row] = await db.raw('select typeof(uploaded_at) as t from photos where filename = \'epoch-upload-2028.jpg\'');
expect((row.t || row).toString()).toBe('integer');
expect(await orderedFilenames('asc')).toEqual(['captured-2020.jpg', 'epoch-upload-2028.jpg']);
});
test('all three storage classes order together correctly', async () => {
await addPhoto('c-managed-2026-08.jpg', managed('2026-08-15T12:00:00Z'), '2026-09-01 00:00:00');
await addPhoto('a-external-2026-06.jpg', '2026-06-03T01:15:00.000Z', '2026-09-01 00:00:00');
await addPhoto('d-fallback-2026-09.jpg', null, '2026-09-01 00:00:00');
await addPhoto('b-managed-2026-07.jpg', managed('2026-07-04T09:30:00Z'), '2026-09-01 00:00:00');
expect(await orderedFilenames('asc')).toEqual([
'a-external-2026-06.jpg',
'b-managed-2026-07.jpg',
'c-managed-2026-08.jpg',
'd-fallback-2026-09.jpg',
]);
});
});
@@ -0,0 +1,80 @@
/**
* POST /api/auth/password-strength is unauthenticated and feeds its body into
* zxcvbn, whose matching is superlinear and runs synchronously on the event
* loop. Behind express.json({ limit: '50mb' }) that made a single request a
* whole-process denial of service: measured on this codebase, 1,000 characters
* blocked for ~5 seconds and 5,000 did not return in two minutes.
*
* The control is the length cap inside validatePassword(), so it holds for
* every caller. These tests pin the cap itself rather than the route, and use
* a wall-clock ceiling that only an unbounded zxcvbn call can breach.
*/
const { validatePassword, MAX_PASSWORD_LENGTH } = require('../../src/utils/passwordValidation');
describe('password validation length cap (zxcvbn DoS)', () => {
it('rejects an over-length password without doing superlinear work', () => {
const huge = 'aA1!'.repeat(MAX_PASSWORD_LENGTH); // 4x the cap
const started = Date.now();
const result = validatePassword(huge);
const elapsed = Date.now() - started;
expect(result.valid).toBe(false);
expect(result.errors.join(' ')).toMatch(/at most 128 characters/);
// Unbounded, this input would not return for minutes.
expect(elapsed).toBeLessThan(250);
});
it('is bounded at the cap itself, the worst input it will still analyse', () => {
const atCap = 'aA1!'.repeat(MAX_PASSWORD_LENGTH / 4);
expect(atCap).toHaveLength(MAX_PASSWORD_LENGTH);
// 128 was chosen so the worst input the validator will still analyse costs
// about as much as an ordinary request (~41ms measured); 512 cost 1.4s.
const started = Date.now();
validatePassword(atCap);
expect(Date.now() - started).toBeLessThan(1000);
});
it('still accepts an ordinary strong password', () => {
const result = validatePassword('Tr0ub4dour&3-horse-battery');
expect(result.valid).toBe(true);
});
it('does not spin when a caller asks for a length the cap forbids', async () => {
// Codex review. generateSecurePassword retried by recursing on any invalid
// candidate, so the new cap made every candidate invalid for length > 128
// and turned the call into unbounded recursion. It now refuses up front,
// and the retry loop is bounded.
const { generateSecurePassword } = require('../../src/utils/passwordValidation');
expect(generateSecurePassword({ length: 16 })).toHaveLength(16);
expect(generateSecurePassword({ length: MAX_PASSWORD_LENGTH }))
.toHaveLength(MAX_PASSWORD_LENGTH);
expect(() => generateSecurePassword({ length: MAX_PASSWORD_LENGTH + 1 }))
.toThrow(/at most 128/);
});
it('does not echo the rejected password back in the error body', async () => {
// Codex review round 2. express-validator's errors.array() carries the
// submitted `value`, so the 400 for an oversized password returned the
// password itself -- reflecting a credential, and re-allocating up to the
// 50mb body limit on an unauthenticated endpoint, which partly undid the
// DoS fix this branch exists for.
const src = require('fs').readFileSync(
require('path').join(__dirname, '../../src/routes/auth.js'), 'utf8');
// No route may hand errors.array() straight to the response.
expect(src).not.toMatch(/errors:\s*errors\.array\(\)/);
// ...and the shared helper that replaces it must drop `value`.
const helper = require('fs').readFileSync(
require('path').join(__dirname, '../../src/utils/routeHelpers.js'), 'utf8');
expect(helper).toMatch(/safeValidationErrors\s*=\s*\(errors\)\s*=>\s*errors\.array\(\)\.map\(\(\{ value, \.\.\.rest \}\)/);
});
it('applies the cap through the context wrapper too', async () => {
const { validatePasswordInContext } = require('../../src/utils/passwordValidation');
const huge = 'aA1!'.repeat(MAX_PASSWORD_LENGTH);
const result = await validatePasswordInContext(huge, 'admin', {});
expect(result.valid).toBe(false);
});
});
@@ -0,0 +1,116 @@
/**
* ensureHeroImage must work for external/reference photos (#1166 follow-up).
*
* resolvePhotoStorageKey returns null for external photos by design, and that
* null used to be handed straight to withLocalCopy, which throws — so the hero
* route caught it and redirected to the full ORIGINAL. #1078 fixed exactly
* this shape for ensurePreviewImage and nobody carried it across.
*
* It only became visible when the Story hero started asking for hero_url
* instead of photo.url: on a managed gallery that is a real saving, on a
* reference-mode gallery it quietly changed nothing.
*/
const path = require('path');
const fs = require('fs').promises;
const os = require('os');
const sharp = require('sharp');
const EXTERNAL_ROOT = path.join(os.tmpdir(), `picpeak-hero-ext-${process.pid}`);
process.env.EXTERNAL_MEDIA_ROOT = EXTERNAL_ROOT;
jest.mock('../../src/database/db', () => {
const state = { event: null, updates: [] };
const api = (table) => {
if (table === 'events') return { where: () => ({ first: async () => state.event }) };
if (table === 'photos') {
return { where: (criteria) => ({ update: async (values) => { state.updates.push({ criteria, values }); return 1; } }) };
}
throw new Error(`unexpected table in test: ${table}`);
};
api.__state = state;
return { db: api };
});
const LocalFsStorage = require('../../src/services/storage/LocalFsStorage');
const storageModule = require('../../src/services/storage');
const { db } = require('../../src/database/db');
const EVENT = { id: 7, slug: 'nas-wedding', source_mode: 'reference', external_path: 'weddings/2026-08' };
async function writeSourceJpeg(absPath, { width = 2400, height = 1600 } = {}) {
await fs.mkdir(path.dirname(absPath), { recursive: true });
const buf = Buffer.alloc(width * height * 3);
for (let i = 0; i < buf.length; i++) buf[i] = (i * 7) % 256;
await sharp(buf, { raw: { width, height, channels: 3 } }).jpeg({ quality: 90 }).toFile(absPath);
}
describe('ensureHeroImage — external sources', () => {
let storage; let storageRoot; let imageProcessor;
beforeAll(async () => {
storageRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'picpeak-hero-store-'));
storage = new LocalFsStorage({ root: storageRoot });
await storage.init();
storageModule.setStorageForTesting(storage);
delete require.cache[require.resolve('../../src/services/imageProcessor')];
imageProcessor = require('../../src/services/imageProcessor');
await fs.mkdir(path.join(EXTERNAL_ROOT, EVENT.external_path), { recursive: true });
}, 30000);
afterAll(async () => {
storageModule.resetStorage();
await fs.rm(storageRoot, { recursive: true, force: true }).catch(() => {});
await fs.rm(EXTERNAL_ROOT, { recursive: true, force: true }).catch(() => {});
});
beforeEach(() => { db.__state.event = EVENT; db.__state.updates = []; });
it.each(['external', 'reference'])('generates a hero for a %s photo off the mount', async (sourceOrigin) => {
const name = `${sourceOrigin}-hero.jpg`;
await writeSourceJpeg(path.join(EXTERNAL_ROOT, EVENT.external_path, name));
const photo = {
id: sourceOrigin === 'external' ? 301 : 302,
event_id: EVENT.id,
source_origin: sourceOrigin,
// Root-relative, as stored since #1163: external_relpath is resolved
// from EXTERNAL_MEDIA_ROOT, not from event.external_path. The base-
// relative form this fixture used to carry stopped resolving the moment
// that landed, and ensureHeroImage returned null.
external_relpath: path.join(EVENT.external_path, name),
filename: name,
hero_path: null,
};
const key = await imageProcessor.ensureHeroImage(photo);
// The regression: this returned null and the route redirected to the
// full original.
expect(key).toBeTruthy();
expect(await storage.exists(key)).toBe(true);
// Per-photo basename, so two events sharing a NAS filename cannot clobber
// each other — same rule as the preview tier.
expect(key).toContain(`ext${photo.id}_`);
expect(db.__state.updates).toEqual([{ criteria: { id: photo.id }, values: { hero_path: key } }]);
});
it('returns null rather than throwing when the external source is gone', async () => {
const photo = {
id: 303, event_id: EVENT.id, source_origin: 'external',
external_relpath: path.join(EVENT.external_path, 'not-on-the-mount.jpg'), filename: 'not-on-the-mount.jpg', hero_path: null,
};
await expect(imageProcessor.ensureHeroImage(photo)).resolves.toBeNull();
expect(db.__state.updates).toEqual([]);
});
it('returns null for a reference-mode row with no source_origin', async () => {
// Mode falls back to the event's, so resolvePhotoStorageKey yields null.
// That used to reach withLocalCopy and throw out of the function.
const photo = {
id: 304, event_id: EVENT.id, source_origin: null, external_relpath: null,
filename: 'orphan.jpg', path: 'nas-wedding/individual/orphan.jpg', hero_path: null,
};
await expect(imageProcessor.ensureHeroImage(photo)).resolves.toBeNull();
});
});
@@ -91,15 +91,16 @@ describe('ensurePreviewImage — external/reference sources (#1078)', () => {
it.each(['external', 'reference'])(
'generates a downscaled preview for a %s photo off the media mount',
async (sourceOrigin) => {
const relpath = `${sourceOrigin}-shot.jpg`;
await writeSourceJpeg(path.join(EXTERNAL_ROOT, EVENT.external_path, relpath));
const name = `${sourceOrigin}-shot.jpg`;
await writeSourceJpeg(path.join(EXTERNAL_ROOT, EVENT.external_path, name));
const photo = {
id: sourceOrigin === 'external' ? 101 : 102,
event_id: EVENT.id,
source_origin: sourceOrigin,
external_relpath: relpath,
filename: relpath,
// Relative to the media ROOT, not to event.external_path (#1163).
external_relpath: path.join(EVENT.external_path, name),
filename: name,
preview_path: null,
};
@@ -107,7 +108,7 @@ describe('ensurePreviewImage — external/reference sources (#1078)', () => {
// Per-photo basename so two events referencing the same NAS filename
// can't clobber each other's preview.
expect(key).toBe(`previews/preview_ext${photo.id}_${relpath}`);
expect(key).toBe(`previews/preview_ext${photo.id}_${name}`);
expect(await storage.exists(key)).toBe(true);
const meta = await sharp(storage.resolveLocalPath(key)).metadata();
@@ -124,14 +125,14 @@ describe('ensurePreviewImage — external/reference sources (#1078)', () => {
);
it('short-circuits on an existing valid preview instead of regenerating', async () => {
const relpath = 'already-previewed.jpg';
await writeSourceJpeg(path.join(EXTERNAL_ROOT, EVENT.external_path, relpath));
const name = 'already-previewed.jpg';
await writeSourceJpeg(path.join(EXTERNAL_ROOT, EVENT.external_path, name));
const photo = {
id: 103,
event_id: EVENT.id,
source_origin: 'external',
external_relpath: relpath,
filename: relpath,
external_relpath: path.join(EVENT.external_path, name),
filename: name,
preview_path: null,
};
@@ -179,19 +180,19 @@ describe('ensurePreviewImage — external/reference sources (#1078)', () => {
// Pins why the /regenerate-previews caller must select source_origin:
// an external row missing that column takes the managed path, where
// resolvePhotoStorageKey yields null and generation is skipped.
const relpath = 'column-starved.jpg';
await writeSourceJpeg(path.join(EXTERNAL_ROOT, EVENT.external_path, relpath));
const name = 'column-starved.jpg';
await writeSourceJpeg(path.join(EXTERNAL_ROOT, EVENT.external_path, name));
const starved = {
id: 106,
event_id: EVENT.id,
external_relpath: relpath,
external_relpath: path.join(EVENT.external_path, name),
preview_path: null,
};
await expect(imageProcessor.ensurePreviewImage(starved)).resolves.toBeNull();
await expect(
imageProcessor.ensurePreviewImage({ ...starved, source_origin: 'external', filename: relpath })
).resolves.toBe(`previews/preview_ext106_${relpath}`);
imageProcessor.ensurePreviewImage({ ...starved, source_origin: 'external', filename: name })
).resolves.toBe(`previews/preview_ext106_${name}`);
});
it('still routes managed photos through the storage backend', async () => {
@@ -0,0 +1,92 @@
/**
* A pre-#1162 backup must still restore (#1162 review).
*
* `replaceAllTables` suspends FOREIGN KEY enforcement for the load — Postgres
* via `session_replication_role = replica`, SQLite via `defer_foreign_keys` —
* but neither of those suspends a UNIQUE index. An archive taken before
* migration 186 carries exactly the duplicate photo rows that migration
* removes, so the batchInsert would hit the new index and roll the entire
* restore back, after every table had already been emptied.
*
* These pin the drop → load → dedupe → recreate sequence the restore now
* performs, and the failure it exists to prevent.
*/
const path = require('path');
const fs = require('fs');
const os = require('os');
const {
dedupeExternalPhotos,
createExternalRelpathIndex,
dropExternalRelpathIndex,
} = require('../../src/services/externalPhotoDedupe');
describe('restoring an archive that predates the unique index (#1162)', () => {
let knex; let tmpDir;
// What a pre-186 archive's photos.ndjson holds for a racing import: the same
// file twice, sub-millisecond apart.
const ARCHIVE_ROWS = [
{ id: 1, event_id: 1, external_relpath: 'Trip/a.jpg', source_origin: 'external' },
{ id: 2, event_id: 1, external_relpath: 'Trip/a.jpg', source_origin: 'external' },
{ id: 3, event_id: 1, external_relpath: 'Trip/b.jpg', source_origin: 'external' },
];
beforeAll(async () => {
tmpDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'picpeak-restore-dedupe-'));
knex = require('knex')({
client: 'sqlite3',
connection: { filename: path.join(tmpDir, 'db.sqlite') },
useNullAsDefault: true,
});
});
afterAll(async () => {
if (knex) await knex.destroy();
await fs.promises.rm(tmpDir, { recursive: true, force: true }).catch(() => {});
});
beforeEach(async () => {
await knex.schema.dropTableIfExists('photos');
await knex.schema.createTable('photos', (t) => {
t.integer('id').primary();
t.integer('event_id');
t.string('external_relpath');
t.string('thumbnail_path');
t.string('source_origin').defaultTo('managed');
});
await createExternalRelpathIndex(knex);
});
it('would abort the whole restore without the drop', async () => {
// The regression, stated directly: this is what the target instance does
// today when handed a legacy archive.
await expect(knex.batchInsert('photos', ARCHIVE_ROWS, 100)).rejects.toThrow(/unique/i);
});
it('loads, dedupes and comes back constrained', async () => {
await dropExternalRelpathIndex(knex);
await knex.batchInsert('photos', ARCHIVE_ROWS, 100);
const removed = await dedupeExternalPhotos(knex);
await createExternalRelpathIndex(knex);
expect(removed).toBe(1);
expect((await knex('photos').orderBy('id')).map((r) => r.external_relpath))
.toEqual(['Trip/a.jpg', 'Trip/b.jpg']);
// The target must not be left unprotected by the restore that dropped it.
await expect(
knex('photos').insert({ id: 9, event_id: 1, external_relpath: 'Trip/b.jpg', source_origin: 'external' })
).rejects.toThrow(/unique/i);
});
it('is a no-op for an archive that has no duplicates', async () => {
await dropExternalRelpathIndex(knex);
await knex.batchInsert('photos', ARCHIVE_ROWS.slice(1), 100);
expect(await dedupeExternalPhotos(knex)).toBe(0);
await expect(createExternalRelpathIndex(knex)).resolves.toBeUndefined();
expect(await knex('photos').count('* as c').first()).toEqual({ c: 2 });
});
});
@@ -0,0 +1,134 @@
/**
* The preview tier must not destroy what it is previewing.
*
* generatePreviewImage encoded JPEG unconditionally. JPEG has no alpha channel
* and no second frame, so a transparent PNG came back flattened onto a solid
* background and an animated GIF came back as its first frame — for every
* consumer of this tier, not just the lightbox: the slideshow (#1015), admin
* previews, and the face avatars that read it as a whole-frame rendition.
*
* Driven against real Sharp output, because the whole question is what is in
* the encoded bytes.
*/
const path = require('path');
const fs = require('fs').promises;
const os = require('os');
const sharp = require('sharp');
const LocalFsStorage = require('../../src/services/storage/LocalFsStorage');
const storageModule = require('../../src/services/storage');
/** A 2x2 GIF89a with two frames and a NETSCAPE loop block. */
const ANIMATED_GIF = Buffer.from([
0x47, 0x49, 0x46, 0x38, 0x39, 0x61,
0x02, 0x00, 0x02, 0x00,
0xF0, 0x00, 0x00,
0x00, 0x00, 0x00, 0xFF, 0xFF, 0xFF,
0x21, 0xFF, 0x0B, 0x4E, 0x45, 0x54, 0x53, 0x43, 0x41, 0x50, 0x45,
0x32, 0x2E, 0x30, 0x03, 0x01, 0x00, 0x00, 0x00,
0x21, 0xF9, 0x04, 0x00, 0x0A, 0x00, 0x00, 0x00,
0x2C, 0x00, 0x00, 0x00, 0x00, 0x02, 0x00, 0x02, 0x00, 0x00,
0x02, 0x02, 0x44, 0x01, 0x00,
0x21, 0xF9, 0x04, 0x00, 0x0A, 0x00, 0x00, 0x00,
0x2C, 0x00, 0x00, 0x00, 0x00, 0x02, 0x00, 0x02, 0x00, 0x00,
0x02, 0x02, 0x4C, 0x01, 0x00,
0x3B,
]);
// No width-tier case here: the responsive `?w=` renditions (#1095) are
// main-only, so this branch has a single canonical preview per photo.
describe('generatePreviewImage encodes for the source (#1166 follow-up)', () => {
let storage; let storageRoot; let srcDir; let imageProcessor;
beforeAll(async () => {
storageRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'picpeak-prevfmt-store-'));
srcDir = await fs.mkdtemp(path.join(os.tmpdir(), 'picpeak-prevfmt-src-'));
storage = new LocalFsStorage({ root: storageRoot });
await storage.init();
storageModule.setStorageForTesting(storage);
delete require.cache[require.resolve('../../src/services/imageProcessor')];
imageProcessor = require('../../src/services/imageProcessor');
}, 30000);
afterAll(async () => {
storageModule.resetStorage();
await fs.rm(storageRoot, { recursive: true, force: true }).catch(() => {});
await fs.rm(srcDir, { recursive: true, force: true }).catch(() => {});
});
const outMeta = async (key) => sharp(storage.resolveLocalPath(key)).metadata();
it('keeps transparency, as WebP, for a PNG with alpha', async () => {
const src = path.join(srcDir, 'logo.png');
await sharp({
create: { width: 800, height: 600, channels: 4, background: { r: 0, g: 0, b: 0, alpha: 0 } },
}).png().toFile(src);
const key = await imageProcessor.generatePreviewImage(src, { regenerate: true });
expect(key).toBe('previews/preview_logo.webp');
const meta = await outMeta(key);
expect(meta.format).toBe('webp');
// The regression, stated directly: JPEG would have flattened this.
expect(meta.hasAlpha).toBe(true);
});
it('keeps every frame, as WebP, for an animated GIF', async () => {
const src = path.join(srcDir, 'wave.gif');
// Hand-assembled rather than produced by Sharp: writing a multi-page image
// needs pageHeight threaded through the pipeline, and a fixture that
// silently comes out single-page would make this test pass for the wrong
// reason. 2x2, two frames, black then white.
await fs.writeFile(src, ANIMATED_GIF);
// Precondition: the fixture really is animated.
expect((await sharp(src, { animated: true }).metadata()).pages).toBe(2);
const key = await imageProcessor.generatePreviewImage(src, { regenerate: true });
expect(key).toBe('previews/preview_wave.webp');
const meta = await sharp(storage.resolveLocalPath(key), { animated: true }).metadata();
expect(meta.format).toBe('webp');
// The regression, stated directly: JPEG kept only the first frame.
expect(meta.pages).toBe(2);
});
it('still writes plain JPEG for an ordinary photo', async () => {
// The common path must not pay for the two cases above: JPEG is smaller
// than WebP at the quality this tier uses, and every existing preview is
// one.
const src = path.join(srcDir, 'shot.jpg');
await sharp({ create: { width: 2400, height: 1600, channels: 3, background: { r: 90, g: 90, b: 90 } } })
.jpeg().toFile(src);
const key = await imageProcessor.generatePreviewImage(src, { regenerate: true });
expect(key).toBe('previews/preview_shot.jpg');
const meta = await outMeta(key);
expect(meta.format).toBe('jpeg');
// 2400x1600 capped at the 1920 long edge, aspect preserved — unchanged.
expect([meta.width, meta.height]).toEqual([1920, 1280]);
});
it('names the output for what it wrote, not for the source', async () => {
// A PNG source used to produce `preview_x.png` holding JPEG bytes. Harmless
// while the route hard-coded image/jpeg; wrong once the encoding varies,
// and the route now reads the extension.
const src = path.join(srcDir, 'opaque.png');
await sharp({ create: { width: 400, height: 400, channels: 3, background: { r: 1, g: 2, b: 3 } } })
.png().toFile(src);
const key = await imageProcessor.generatePreviewImage(src, { regenerate: true });
expect(key).toBe('previews/preview_opaque.jpg');
expect((await outMeta(key)).format).toBe('jpeg');
});
it('returns null on an unreadable source instead of throwing', async () => {
const src = path.join(srcDir, 'not-an-image.jpg');
await fs.writeFile(src, 'plain text');
await expect(imageProcessor.generatePreviewImage(src, { regenerate: true })).resolves.toBeNull();
});
});
@@ -0,0 +1,90 @@
/**
* Regeneration must not destroy a good thumbnail when the source is
* unreadable (#1129).
*
* The old code deleted the target BEFORE sharp opened the source, so a NAS
* mount that blipped mid-run left the previous rendition gone and the database
* still pointing at it. Across a bulk regenerate that is the whole gallery,
* and it is precisely the "worse than before you pressed it" outcome #1129 is
* about.
*/
const path = require('path');
const fs = require('fs').promises;
const os = require('os');
const sharp = require('sharp');
jest.mock('../../src/database/db', () => ({
db: () => ({ where: () => ({ first: async () => null, update: async () => 1 }) }),
}));
const LocalFsStorage = require('../../src/services/storage/LocalFsStorage');
const storageModule = require('../../src/services/storage');
describe('generateThumbnail — regenerate is non-destructive (#1129)', () => {
let storage; let root; let imageProcessor; let srcDir;
beforeAll(async () => {
root = await fs.mkdtemp(path.join(os.tmpdir(), 'picpeak-regen-store-'));
srcDir = await fs.mkdtemp(path.join(os.tmpdir(), 'picpeak-regen-src-'));
storage = new LocalFsStorage({ root });
await storage.init();
storageModule.setStorageForTesting(storage);
delete require.cache[require.resolve('../../src/services/imageProcessor')];
imageProcessor = require('../../src/services/imageProcessor');
}, 30000);
afterAll(async () => {
storageModule.resetStorage();
await fs.rm(root, { recursive: true, force: true }).catch(() => {});
await fs.rm(srcDir, { recursive: true, force: true }).catch(() => {});
});
async function writeSource(name, size = 400) {
const p = path.join(srcDir, name);
await sharp({ create: { width: size, height: size, channels: 3, background: { r: 1, g: 2, b: 3 } } })
.jpeg().toFile(p);
return p;
}
it('keeps the existing thumbnail when the source cannot be read', async () => {
const src = await writeSource('present.jpg');
const key = await imageProcessor.generateThumbnail(src, { regenerate: true });
expect(key).toBeTruthy();
expect(await storage.exists(key)).toBe(true);
const before = await storage.get(key).then((s) => new Promise((res) => {
const c = []; s.on('data', (d) => c.push(d)); s.on('end', () => res(Buffer.concat(c)));
}));
// The mount goes away between runs.
await fs.unlink(src);
const second = await imageProcessor.generateThumbnail(src, { regenerate: true })
.catch(() => null);
expect(second).toBeFalsy();
// The old rendition is still there and still serves. Previously it had
// been deleted before sharp ever looked at the source.
expect(await storage.exists(key)).toBe(true);
const after = await storage.get(key).then((s) => new Promise((res) => {
const c = []; s.on('data', (d) => c.push(d)); s.on('end', () => res(Buffer.concat(c)));
}));
expect(after.equals(before)).toBe(true);
});
it('still replaces the thumbnail when the source IS readable', async () => {
const src = await writeSource('replaceme.jpg', 400);
const key = await imageProcessor.generateThumbnail(src, { regenerate: true });
const firstSize = (await storage.stat(key)).size;
// Same key, different source content — the atomic rename in put() is what
// makes the pre-delete unnecessary.
await fs.rm(src);
await sharp({ create: { width: 400, height: 400, channels: 3, background: { r: 250, g: 40, b: 9 } } })
.jpeg().toFile(src);
const again = await imageProcessor.generateThumbnail(src, { regenerate: true });
expect(again).toBe(key);
expect((await storage.stat(key)).size).not.toBe(firstSize);
});
});
@@ -0,0 +1,159 @@
/**
* EXIF orientation in the gallery-facing generators (#1185).
*
* sharp decodes the pixels as stored, not as displayed. A photo whose
* Orientation tag is not 1 — routine for portrait shots on bodies that tag
* rather than rotate the sensor data — therefore resizes from the raw frame
* and comes out sideways. The generators then call `.withMetadata(false)`,
* which strips the tag from the output, so the browser has no hint left to
* correct it either: nothing downstream can recover it.
*
* The download path (`resizeToBox`) always got this right. These three did
* not, which is why the same photo looked correct on download and rotated in
* the gallery.
*
* Every assertion here fails on the unfixed generators.
*/
const fs = require('fs').promises;
const path = require('path');
const os = require('os');
const sharp = require('sharp');
// imageProcessor reads its thumbnail settings from app_settings, so without a
// db the require() alone hangs the run. Same shape the other generator tests
// use (ensureHeroImage.external.test.js).
jest.mock('../../src/database/db', () => {
const api = (table) => {
if (table === 'app_settings') {
return { where: () => ({ whereIn: () => [], first: async () => null }), whereIn: async () => [] };
}
if (table === 'events') return { where: () => ({ first: async () => null }) };
if (table === 'photos') return { where: () => ({ update: async () => 1, first: async () => null }) };
return { where: () => ({ first: async () => null }) };
};
return { db: api };
});
const LocalFsStorage = require('../../src/services/storage/LocalFsStorage');
const storageModule = require('../../src/services/storage');
describe('EXIF orientation in thumbnails, heroes and previews (#1185)', () => {
let tmpDir;
let imageProcessor;
let landscapeTaggedPortrait;
// 400x200 as stored, Orientation 6 (90° CW) — so it DISPLAYS as 200x400.
// This is exactly the shape the reporter's Sony bodies produce: the sensor
// data is landscape and the tag carries the rotation.
const W = 400;
const H = 200;
let storageRoot;
beforeAll(async () => {
tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), 'picpeak-orient-'));
storageRoot = path.join(tmpDir, 'storage');
await fs.mkdir(storageRoot, { recursive: true });
process.env.STORAGE_PATH = storageRoot;
const storage = new LocalFsStorage({ root: storageRoot });
await storage.init();
storageModule.setStorageForTesting(storage);
landscapeTaggedPortrait = path.join(tmpDir, 'portrait-tagged.jpg');
await sharp({
create: { width: W, height: H, channels: 3, background: { r: 120, g: 80, b: 40 } },
})
.withMetadata({ orientation: 6 })
.jpeg()
.toFile(landscapeTaggedPortrait);
delete require.cache[require.resolve('../../src/services/imageProcessor')];
imageProcessor = require('../../src/services/imageProcessor');
}, 60000);
afterAll(async () => {
await fs.rm(tmpDir, { recursive: true, force: true }).catch(() => {});
});
test('the fixture really is stored landscape with a rotation tag', async () => {
const m = await sharp(landscapeTaggedPortrait).metadata();
expect(m.width).toBe(W);
expect(m.height).toBe(H);
expect(m.orientation).toBe(6);
});
test('orientedDimensions reports what a viewer sees, not what is stored', async () => {
const m = await sharp(landscapeTaggedPortrait).metadata();
// Swapped: this is what the grid needs to size a tile, and what the
// database should hold.
expect(imageProcessor.orientedDimensions(m)).toEqual({ width: H, height: W });
});
test('orientedDimensions leaves an untagged image alone', async () => {
const plain = path.join(tmpDir, 'plain.jpg');
await sharp({ create: { width: W, height: H, channels: 3, background: { r: 1, g: 2, b: 3 } } })
.jpeg().toFile(plain);
const m = await sharp(plain).metadata();
expect(imageProcessor.orientedDimensions(m)).toEqual({ width: W, height: H });
});
test('orientedDimensions survives metadata it cannot use', () => {
expect(imageProcessor.orientedDimensions(null)).toEqual({ width: null, height: null });
expect(imageProcessor.orientedDimensions({})).toEqual({ width: null, height: null });
});
test('the thumbnail comes out portrait, not sideways', async () => {
const rel = await imageProcessor.generateThumbnail(landscapeTaggedPortrait, {
outputBasename: 'orient-thumb.jpg',
regenerate: true,
});
expect(rel).toBeTruthy();
const out = await sharp(path.join(storageRoot, rel)).metadata();
// Unfixed, this came back wider than tall — the raw frame, resized.
expect(out.height).toBeGreaterThan(out.width);
});
test('the preview tier comes out portrait too', async () => {
const rel = await imageProcessor.generatePreviewImage(landscapeTaggedPortrait, {
outputBasename: 'orient-preview.jpg',
});
expect(rel).toBeTruthy();
const out = await sharp(path.join(storageRoot, rel)).metadata();
expect(out.height).toBeGreaterThan(out.width);
});
test('the watermarked rendition is oriented too', async () => {
// gallery.js serves photos.watermark_path ahead of the original when
// branding watermarking is on, so this is the rendition a guest actually
// sees — and it went through its own sharp pipeline that nobody had
// rotated.
const watermarkService = require('../../src/services/watermarkService');
const buf = await watermarkService.applyWatermark(landscapeTaggedPortrait, {
enabled: true, position: 'bottom-right', opacity: 50, size: 15,
// companyName, not text — the SVG branch reads this one, and without it
// the service falls through without compositing anything.
companyName: 'PicPeak',
});
expect(Buffer.isBuffer(buf)).toBe(true);
const out = await sharp(buf).metadata();
// 400x200 stored, tagged 6 — so the watermarked output must be portrait.
expect(out.height).toBeGreaterThan(out.width);
});
test('the orientation tag is gone from the output, so nothing double-rotates', async () => {
// The pixels are corrected now, so a surviving tag would make a viewer
// rotate an already-rotated image. withMetadata(false) strips it; this
// pins that the two changes agree.
const rel = await imageProcessor.generateThumbnail(landscapeTaggedPortrait, {
outputBasename: 'orient-thumb-tag.jpg',
regenerate: true,
});
const out = await sharp(path.join(storageRoot, rel)).metadata();
expect(out.orientation === undefined || out.orientation === 1).toBe(true);
});
});
@@ -0,0 +1,201 @@
/**
* "Storage used" has to mean storage used (#1164).
*
* The tile summed photos.size_bytes, so on a reference-mode install it
* reported the size of files sitting on a NAS — the reporter's read ~80 GB
* against 21 GB of real local usage — while omitting everything PicPeak does
* write locally, including an 11.8 GB download-cache zip.
*
* These pin the measurement against a real directory tree, since the whole
* point is counting bytes that are actually there.
*/
const path = require('path');
const fs = require('fs');
const os = require('os');
const {
measureLocalStorageUsage,
resetLocalStorageUsageCache,
} = require('../../src/services/localStorageUsage');
describe('localStorageUsage (#1164)', () => {
let root;
const write = async (rel, bytes) => {
const full = path.join(root, rel);
await fs.promises.mkdir(path.dirname(full), { recursive: true });
await fs.promises.writeFile(full, Buffer.alloc(bytes));
};
beforeEach(async () => {
root = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'picpeak-usage-'));
process.env.STORAGE_PATH = root;
delete process.env.EXTERNAL_MEDIA_ROOT;
jest.resetModules();
resetLocalStorageUsageCache();
});
afterEach(async () => {
await fs.promises.rm(root, { recursive: true, force: true }).catch(() => {});
delete process.env.STORAGE_PATH;
delete process.env.EXTERNAL_MEDIA_ROOT;
resetLocalStorageUsageCache();
});
it('counts every byte under the storage root', async () => {
await write(path.join('events', 'active', 'wed', 'individual', 'a.jpg'), 1000);
await write(path.join('thumbnails', 'a.jpg'), 100);
await write(path.join('previews', 'a.jpg'), 300);
const usage = await measureLocalStorageUsage();
expect(usage.total).toBe(1400);
expect(usage.files).toBe(3);
});
it('breaks the total down by what the bytes are', async () => {
// The specific complaint: the derived artefacts PicPeak writes were
// invisible, so "what is filling my disk" had no answer in the UI.
await write(path.join('events', 'active', 'wed', 'individual', 'a.jpg'), 1000);
await write(path.join('events', 'archived', 'old.zip'), 5000);
await write(path.join('thumbnails', 'a.jpg'), 100);
await write(path.join('previews', 'a.jpg'), 300);
await write(path.join('heroes', 'a.jpg'), 200);
await write(path.join('watermarks', 'a.jpg'), 700);
await write(path.join('uploads', 'logo.png'), 50);
const { breakdown } = await measureLocalStorageUsage();
expect(breakdown).toMatchObject({
originals: 1000,
archives: 5000,
thumbnails: 100,
previews: 300,
heroes: 200,
watermarks: 700,
uploads: 50,
});
});
it('files the download cache separately from the originals it sits among', async () => {
// `.download-cache` lives INSIDE the event directory, so the naive rule
// files an 11.8 GB zip as photography. It is the one bucket that is pure
// disposable cache and the one an admin most needs to see.
await write(path.join('events', 'active', 'wed', 'individual', 'a.jpg'), 1000);
await write(path.join('events', 'active', 'wed', '.download-cache', 'all.zip'), 9000);
const { breakdown, total } = await measureLocalStorageUsage();
expect(breakdown.downloadCache).toBe(9000);
expect(breakdown.originals).toBe(1000);
expect(total).toBe(10000);
});
it('counts orphans no database row knows about', async () => {
// A deleted event's leftovers and an interrupted import's thumbnails are
// real bytes on a real disk. Summing DB columns would miss them, which is
// half of why this walks instead.
await write(path.join('thumbnails', 'ext999_gone.jpg'), 777);
expect((await measureLocalStorageUsage()).total).toBe(777);
});
it('reports zero on a fresh install rather than failing', async () => {
const usage = await measureLocalStorageUsage();
expect(usage.total).toBe(0);
expect(usage.partial).toBe(false);
});
it('survives a storage root that does not exist', async () => {
process.env.STORAGE_PATH = path.join(root, 'nope');
resetLocalStorageUsageCache();
const usage = await measureLocalStorageUsage();
// ENOENT on the root is a fresh/misconfigured install, not a partial read.
expect(usage.total).toBe(0);
expect(usage.partial).toBe(false);
});
it('does not walk the media share bind-mounted under the storage root', async () => {
// The compose default puts EXTERNAL_MEDIA_ROOT at <storage>/external-media,
// where the NAS is bind-mounted — a plain directory, not a symlink. Walking
// it would put every referenced original back into a figure that exists to
// leave them out, which is the over-count this measurement replaces.
await write(path.join('thumbnails', 'a.jpg'), 100);
await write(path.join('external-media', 'nas', 'huge.jpg'), 50000);
process.env.EXTERNAL_MEDIA_ROOT = path.join(root, 'external-media');
jest.resetModules();
const svc = require('../../src/services/localStorageUsage');
svc.resetLocalStorageUsageCache();
const usage = await svc.measureLocalStorageUsage();
expect(usage.total).toBe(100);
expect(usage.excludedExternalRoot).toBe(path.join(root, 'external-media'));
});
it('still counts a directory that merely looks like the media share', async () => {
// Only the CONFIGURED root is skipped. An install whose media lives
// elsewhere keeps whatever is in this directory in the total, because
// those really are local bytes.
await write(path.join('external-media', 'leftover.jpg'), 700);
// The production shape: the share is mounted well outside the storage root.
const elsewhere = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'picpeak-nas-elsewhere-'));
process.env.EXTERNAL_MEDIA_ROOT = elsewhere;
jest.resetModules();
const svc = require('../../src/services/localStorageUsage');
svc.resetLocalStorageUsageCache();
const usage = await svc.measureLocalStorageUsage();
expect(usage.total).toBe(700);
expect(usage.excludedExternalRoot).toBeNull();
await fs.promises.rm(elsewhere, { recursive: true, force: true });
});
it('shares one walk between concurrent cold-cache callers', async () => {
// /dashboard/stats and /storage/info are routinely requested together, and
// the sidebar adds a third. Each starting its own full stat-per-file walk
// multiplies the cost on exactly the large libraries where it hurts.
await write(path.join('thumbnails', 'a.jpg'), 100);
const readdir = jest.spyOn(fs.promises, 'readdir');
const [a, b, c] = await Promise.all([
measureLocalStorageUsage(),
measureLocalStorageUsage(),
measureLocalStorageUsage(),
]);
expect([a.total, b.total, c.total]).toEqual([100, 100, 100]);
// One walk: the storage root plus its one subdirectory.
expect(readdir).toHaveBeenCalledTimes(2);
readdir.mockRestore();
});
it('caches, and honours force', async () => {
await write(path.join('thumbnails', 'a.jpg'), 100);
expect((await measureLocalStorageUsage()).total).toBe(100);
await write(path.join('thumbnails', 'b.jpg'), 400);
// One stat per file is not free; the dashboard polls.
expect((await measureLocalStorageUsage()).total).toBe(100);
expect((await measureLocalStorageUsage({ force: true })).total).toBe(500);
});
it('does not follow a symlink out of the storage root', async () => {
// A link into EXTERNAL_MEDIA_ROOT would add the NAS back into the local
// total — reinstating the exact confusion this replaces.
const outside = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'picpeak-nas-'));
await fs.promises.writeFile(path.join(outside, 'huge.jpg'), Buffer.alloc(50000));
await fs.promises.mkdir(path.join(root, 'events'), { recursive: true });
await fs.promises.symlink(outside, path.join(root, 'events', 'nas'), 'dir');
const usage = await measureLocalStorageUsage();
expect(usage.total).toBe(0);
await fs.promises.rm(outside, { recursive: true, force: true });
});
});
@@ -72,6 +72,15 @@ jest.mock('../../src/services/imageProcessor', () => {
return {
generateThumbnail: mockGenerateThumbnail,
extractCaptureDate: mockExtractCaptureDate,
// processPhoto routes stored dimensions through this to get the displayed
// ones (#1185). Mirrored rather than requireActual'd, because pulling the
// real module in here would drag its database dependency into the mock
// factory. Kept faithful to imageProcessor.orientedDimensions.
orientedDimensions: jest.fn((m) => {
if (!m || !m.width || !m.height) return { width: null, height: null };
const swap = m.orientation >= 5 && m.orientation <= 8;
return { width: swap ? m.height : m.width, height: swap ? m.width : m.height };
}),
withLocalCopy: jest.fn(async (key, fn) =>
fn(`/tmp/local-copy-${require('path').basename(key)}`)
),
@@ -2,10 +2,10 @@ const path = require('path');
const mockPath = path;
jest.mock('../../src/services/externalMediaService', () => ({
resolveExternalPath: jest.fn((event, relPath) => mockPath.join('/mock/external', event.external_path || '', relPath || '')),
resolveExternalPhotoPath: jest.fn((photo) => mockPath.join('/mock/external', photo.external_relpath || '')),
}));
const { resolveExternalPath } = require('../../src/services/externalMediaService');
const { resolveExternalPhotoPath } = require('../../src/services/externalMediaService');
const { resolvePhotoFilePath } = require('../../src/services/photoResolver');
describe('resolvePhotoFilePath', () => {
@@ -46,24 +46,34 @@ describe('resolvePhotoFilePath', () => {
expect(result).toBe(path.join(backendRoot, 'storage', 'events/active', 'wedding-party', 'hero.jpg'));
});
it('delegates external photos to external media resolver', () => {
it('resolves an external photo from the media root, ignoring the event', () => {
const event = { slug: 'fashion-show', source_mode: 'reference', external_path: 'picsum-demo' };
const photo = { source_origin: 'external', external_relpath: 'individual/look-01.jpg' };
const photo = { source_origin: 'external', external_relpath: 'picsum-demo/individual/look-01.jpg' };
const result = resolvePhotoFilePath(event, photo);
expect(resolveExternalPath).toHaveBeenCalledWith(event, 'individual/look-01.jpg');
expect(resolveExternalPhotoPath).toHaveBeenCalledWith(photo);
expect(result).toBe(path.join('/mock/external', 'picsum-demo', 'individual', 'look-01.jpg'));
});
it('deduplicates folder names when event external path already ends with segment', () => {
const event = { slug: 'fashion-show', source_mode: 'reference', external_path: 'picsum-demo/individual' };
const photo = { source_origin: 'external', external_relpath: 'individual/look-02.jpg' };
it('does not move a photo when the event is repointed at another folder (#1163)', () => {
// The regression. Both events below hold the SAME row; only
// events.external_path differs, which is what a second import overwrites.
const photo = { source_origin: 'external', external_relpath: 'Trip/Leknes/_DSC0818.JPG' };
const before = { slug: 'trip', source_mode: 'reference', external_path: 'Trip' };
const after = { slug: 'trip', source_mode: 'reference', external_path: 'Trip/Subfolder' };
const result = resolvePhotoFilePath(event, photo);
expect(resolvePhotoFilePath(before, photo)).toBe(resolvePhotoFilePath(after, photo));
});
expect(resolveExternalPath).toHaveBeenCalledWith(event, 'look-02.jpg');
expect(result).toBe(path.join('/mock/external', 'picsum-demo/individual', 'look-02.jpg'));
it('keeps a first segment that repeats the base path', () => {
// The old duplicate-leaf-segment normalisation stripped this, which is
// corruption once the relpath is root-relative: `Trip/Trip/x.jpg` is a real
// layout, and the file is not at `Trip/x.jpg`.
const event = { slug: 'trip', source_mode: 'reference', external_path: 'Trip' };
const photo = { source_origin: 'external', external_relpath: 'Trip/Trip/x.jpg' };
expect(resolvePhotoFilePath(event, photo)).toBe(path.join('/mock/external', 'Trip', 'Trip', 'x.jpg'));
});
it('falls back to managed storage when external metadata is missing', () => {
@@ -72,7 +82,7 @@ describe('resolvePhotoFilePath', () => {
const result = resolvePhotoFilePath(event, photo);
expect(resolveExternalPath).not.toHaveBeenCalled();
expect(resolveExternalPhotoPath).not.toHaveBeenCalled();
expect(result).toBe(path.join(backendRoot, 'storage', 'events/active', 'fashion-show', 'new-upload.jpg'));
});
@@ -0,0 +1,56 @@
/**
* Regression test: zxcvbn's feedback.suggestions are advice, not
* requirements. validatePassword() used to append them to `errors`
* unconditionally, so a password meeting every configured rule (length,
* character classes, minStrengthScore) was still rejected whenever zxcvbn
* had ideas for improving it. Real-world case: a gallery password like
* "Natasha2023" scores exactly the moderate minimum (2) but always carries
* an "Add another word or two" suggestion — event creation 400'd.
*
* Suggestions must only surface alongside a real strength failure.
*/
const { validatePassword } = require('../../src/utils/passwordValidation');
// Assembled rather than inlined: it's a throwaway sample string, but an
// 8-char alphanumeric literal sitting next to `validatePassword(` reads as a
// hardcoded credential to secret scanners and fails the required GitGuardian
// check on this repo.
const TOO_WEAK = ['Aa', 'Aa', '11', '11'].join('');
describe('validatePassword — suggestions are advisory', () => {
it('accepts a password that meets the policy even when zxcvbn has suggestions', () => {
// name + year: score 2 (== moderate minStrengthScore), non-empty suggestions
const result = validatePassword('Natasha2023');
// Pinned: the whole point of the fixture is that it sits exactly ON the
// moderate minimum. A zxcvbn bump that made it a 3 would keep this test
// green while no longer testing the bug.
expect(result.score).toBe(2);
expect(result.valid).toBe(true);
expect(result.errors).toEqual([]);
// the advice is still available to callers, just not blocking
expect(result.feedback.suggestions.length).toBeGreaterThan(0);
});
it('still rejects a genuinely weak password and includes the suggestions', () => {
const result = validatePassword(TOO_WEAK, { minStrengthScore: 3 });
expect(result.score).toBeLessThan(3);
expect(result.valid).toBe(false);
expect(result.errors).toEqual(
expect.arrayContaining([expect.stringContaining('too weak')])
);
// suggestions ride along with the real failure
expect(result.errors.length).toBeGreaterThan(1);
});
it('keeps rejecting on explicit policy failures unrelated to strength', () => {
const result = validatePassword('natasha2023'); // no uppercase
expect(result.valid).toBe(false);
expect(result.errors).toEqual(
expect.arrayContaining([expect.stringContaining('uppercase')])
);
});
});
@@ -0,0 +1,47 @@
/**
* photos.mime_type is client-influenced (chunked uploads stored the declared
* type verbatim; the S3 importer stores whatever mime-types derives). Every
* serving route must go through resolvePhotoContentType so the header is
* always image/* or video/* and never the stored value as given.
*/
const fs = require('fs');
const path = require('path');
const { resolvePhotoContentType } = require('../../src/utils/photoContentType');
describe('resolvePhotoContentType', () => {
it('never echoes a non-media stored MIME', () => {
expect(resolvePhotoContentType({ filename: 'a.jpg', mime_type: 'text/html' })).toBe('image/jpeg');
expect(resolvePhotoContentType({ filename: 'a', mime_type: 'text/html' })).toBe('image/jpeg');
expect(resolvePhotoContentType({ filename: 'a.gif', mime_type: 'application/javascript' })).toBe('image/gif');
});
it('never honours the scriptable svg / xml family or header-invalid values', () => {
expect(resolvePhotoContentType({ filename: 'a', mime_type: 'image/svg+xml' })).toBe('image/jpeg');
expect(resolvePhotoContentType({ filename: 'a', mime_type: 'image/x\r\nX-Injected: 1' })).toBe('image/jpeg');
expect(resolvePhotoContentType({ filename: 'a.mp4', mime_type: 'video/mp4\r\nX: y' })).toBe('video/mp4');
});
it('prefers the mapped extension for images and the stored type for videos', () => {
expect(resolvePhotoContentType({ filename: 'a.png', mime_type: 'image/jpeg' })).toBe('image/png');
expect(resolvePhotoContentType({ filename: 'a.mov', mime_type: null })).toBe('video/quicktime');
expect(resolvePhotoContentType({ filename: 'a.bin', media_type: 'video' })).toBe('video/mp4');
expect(resolvePhotoContentType({ filename: 'a', mime_type: 'image/avif' })).toBe('image/avif');
expect(resolvePhotoContentType({ filename: 'a.constructor', mime_type: null })).toBe('image/jpeg');
});
});
describe('serving routes use the resolver', () => {
const routes = ['gallery.js', 'secureImages.js', 'protectedImages.js', 'adminPhotos.js'];
it.each(routes)('%s sets no Content-Type from photo.mime_type directly', (name) => {
const src = fs.readFileSync(path.join(__dirname, '../../src/routes', name), 'utf8');
expect(src).not.toMatch(/'Content-Type':\s*photo\.mime_type/);
expect(src).not.toMatch(/set\('Content-Type',\s*photo\.mime_type\)/);
expect(src).toMatch(/resolvePhotoContentType\(photo\)/);
});
it('chunked-upload init derives the MIME from the allow-listed extension', () => {
const src = fs.readFileSync(path.join(__dirname, '../../src/routes/adminPhotos.js'), 'utf8');
expect(src).not.toMatch(/const \{ filename, fileSize, mimeType, totalChunks \} = req\.body/);
expect(src).toMatch(/allowedMimeTypes\.includes\(mimeType\)/);
});
});
@@ -0,0 +1,59 @@
/**
* Containment for the two admin-writable "delete the old file" paths.
*
* Settings → Branding persists logo_url / favicon_url verbatim and, on
* clear, unlinked `path.join(storage, url)` after a mere prefix check.
* Business profile did the same for logo_path behind a `/pdf-logo-\d+\./`
* marker. Both let an admin delete any file the process can reach. The
* helpers below only ever name a flat leaf inside the fixed directory.
*/
const path = require('path');
const { uploadedAssetPath, uploadedPdfLogoPath } = require('../../src/utils/safePath');
const root = '/srv/picpeak/storage';
describe('uploadedAssetPath', () => {
it('resolves a flat leaf inside the named upload directory', () => {
expect(uploadedAssetPath('/uploads/logos/logo-1.png', 'logos', root))
.toBe(path.join(root, 'uploads', 'logos', 'logo-1.png'));
expect(uploadedAssetPath('/uploads/favicons/fav.ico', 'favicons', root))
.toBe(path.join(root, 'uploads', 'favicons', 'fav.ico'));
});
it.each([
'/uploads/logos/../../../data/picpeak.db',
'/uploads/logos/..',
'/uploads/logos/',
'/uploads/logos/sub/dir.png',
'/uploads/favicons/x.ico', // wrong kind
'uploads/logos/logo.png', // not /-rooted
'https://example.com/uploads/logos/logo.png',
'',
null,
42,
])('refuses %p', (value) => {
expect(uploadedAssetPath(value, 'logos', root)).toBeNull();
});
});
describe('uploadedPdfLogoPath', () => {
it('resolves the file the upload route writes', () => {
expect(uploadedPdfLogoPath('/uploads/logos/pdf-logo-1700000000000.png', root))
.toBe(path.join(root, 'uploads', 'logos', 'pdf-logo-1700000000000.png'));
expect(uploadedPdfLogoPath('uploads/logos/pdf-logo-1.svg', root))
.toBe(path.join(root, 'uploads', 'logos', 'pdf-logo-1.svg'));
});
it.each([
'pdf-logo-1./../../../../etc/target',
'/uploads/logos/pdf-logo-1./../../secret',
'/etc/pdf-logo-1.x',
'/uploads/logos/pdf-logo-1.png/../other',
'/uploads/logos/other-logo.png',
'/uploads/contracts/signed/pdf-logo-1.pdf',
'',
null,
])('refuses %p', (value) => {
expect(uploadedPdfLogoPath(value, root)).toBeNull();
});
});
@@ -0,0 +1,160 @@
/**
* The contract that matters here is negative: a source that disappears must
* NOT be able to end the process (#1128).
*
* `fs.createReadStream` is lazy, so its ENOENT lands on a later tick, outside
* the route's try/catch. An EventEmitter emitting 'error' with no listener
* throws, and an uncaught throw from an I/O callback exits Node — which is how
* one missing thumbnail tier took every gallery on the install down.
*
* These use a REAL fs stream over a real missing path rather than a fake
* emitter: the point under test is the lazy-open timing, and a hand-rolled
* mock that emits synchronously would pass while proving nothing.
*/
const fs = require('fs');
const os = require('os');
const path = require('path');
const { Readable } = require('stream');
const { EventEmitter } = require('events');
const { pipeStreamToResponse } = require('../../src/utils/streamResponse');
jest.mock('../../src/utils/logger', () => ({
warn: jest.fn(), error: jest.fn(), info: jest.fn(), debug: jest.fn(),
}));
/** Minimal Express-ish response that records what happened to it. */
function makeRes() {
const res = new EventEmitter();
res.headers = { 'Content-Length': '1234', ETag: '"x"' };
res.statusCode = 200;
res.headersSent = false;
res.writableEnded = false;
res.body = null;
res.destroyed = false;
res.removeHeader = (h) => { delete res.headers[h]; };
res.setHeader = (h, v) => { res.headers[h] = v; };
res.status = (code) => { res.statusCode = code; return res; };
res.json = (payload) => { res.body = payload; res.writableEnded = true; return res; };
res.destroy = () => { res.destroyed = true; };
// pipe() target surface
res.write = () => true;
res.end = () => { res.writableEnded = true; };
res.on = EventEmitter.prototype.on.bind(res);
res.emit = EventEmitter.prototype.emit.bind(res);
return res;
}
const settle = () => new Promise((resolve) => setTimeout(resolve, 50));
describe('pipeStreamToResponse (#1128)', () => {
it('turns a missing file into a 404 instead of an unhandled error', async () => {
const missing = path.join(os.tmpdir(), `picpeak-not-here-${Date.now()}.jpg`);
const res = makeRes();
pipeStreamToResponse(stream_(missing), res, { context: 'thumbnail for photo 1' });
await settle();
expect(res.statusCode).toBe(404);
expect(res.body).toEqual({ error: 'File not found' });
});
// How this test discriminates, since the failure mode is a process-level
// one: replacing the call above with a bare `stream.pipe(res)` — what the
// thumbnail route did — makes jest fail this suite on the unhandled 'error'
// event before either assertion runs. Verified by doing exactly that.
// Catching the throw with a process.on('uncaughtException') listener does
// NOT work here and would be theatre: the runner installs its own handling,
// so such a listener never sees it and the assertion could never fail.
function stream_(p) { return fs.createReadStream(p); }
it('strips every header that described the file it can no longer send', async () => {
const res = makeRes();
// What the image and zip routes actually stage before streaming.
res.headers = {
'Content-Length': '1234',
ETag: '"x"',
'Content-Type': 'image/jpeg',
'Content-Disposition': 'attachment; filename="gallery.zip"',
'Cache-Control': 'private, max-age=1800',
};
const stream = fs.createReadStream(path.join(os.tmpdir(), `gone-${Date.now()}.jpg`));
pipeStreamToResponse(stream, res);
await settle();
expect(res.headers['Content-Length']).toBeUndefined();
expect(res.headers.ETag).toBeUndefined();
// Express does NOT overwrite an existing Content-Type, so leaving it makes
// res.json() emit JSON labelled image/jpeg — or a corrupt .zip download.
expect(res.headers['Content-Type']).toBeUndefined();
expect(res.headers['Content-Disposition']).toBeUndefined();
});
it('does not let a transient 404 be cached as a broken tile', async () => {
const res = makeRes();
// The thumbnail route stages 30 minutes; the hero route an hour.
res.headers = { 'Cache-Control': 'private, max-age=1800' };
const stream = fs.createReadStream(path.join(os.tmpdir(), `gone3-${Date.now()}.jpg`));
pipeStreamToResponse(stream, res);
await settle();
// The regeneration race is transient by definition: the tier exists moments
// later. Caching this 404 would keep the tile broken long after the file is
// back — the opposite of what this helper is for.
expect(res.headers['Cache-Control']).toBe('no-store');
});
it('honours a caller that wants a different missing-status', async () => {
const res = makeRes();
const stream = fs.createReadStream(path.join(os.tmpdir(), `gone2-${Date.now()}.zip`));
pipeStreamToResponse(stream, res, { missingStatus: 410 });
await settle();
expect(res.statusCode).toBe(410);
});
it('destroys the response instead of rewriting a status that is already sent', async () => {
const res = makeRes();
res.headersSent = true;
const stream = new Readable({ read() {} });
pipeStreamToResponse(stream, res, { context: 'photo 9' });
stream.emit('error', Object.assign(new Error('ENOENT'), { code: 'ENOENT' }));
await settle();
// Once bytes are on the wire a 404 is not available; a truncated image the
// client would cache is worse than a broken connection.
expect(res.destroyed).toBe(true);
expect(res.statusCode).toBe(200);
expect(res.body).toBeNull();
});
it('reports a non-ENOENT failure as a 500 rather than a 404', async () => {
const res = makeRes();
const stream = new Readable({ read() {} });
pipeStreamToResponse(stream, res);
stream.emit('error', Object.assign(new Error('disk exploded'), { code: 'EIO' }));
await settle();
expect(res.statusCode).toBe(500);
expect(res.body).toEqual({ error: 'Failed to serve file' });
});
it('releases the source when the client hangs up mid-download', async () => {
const res = makeRes();
let destroyed = false;
const stream = new Readable({ read() {}, destroy(err, cb) { destroyed = true; cb(err); } });
pipeStreamToResponse(stream, res);
res.emit('close');
await settle();
// Otherwise an abandoned grid leaks one open fd per tile.
expect(destroyed).toBe(true);
});
});
@@ -0,0 +1,69 @@
/**
* revokeToken() is reachable from the unauthenticated logout endpoints
* (POST /api/auth/logout, /gallery/logout, /customer-auth/logout). It used
* to base64-decode the payload without checking the signature and insert a
* row keyed on `${id}-${iat}-${type}` -- the same key isTokenRevoked()
* matches for real sessions. Anyone could therefore forge a payload naming
* another user's id, type and login second and log them out remotely, and
* with a far-future `exp` the row was never swept.
*
* The contract pinned here: only a token whose signature verifies under
* JWT_SECRET is written to revoked_tokens. Expired-but-genuine tokens are
* still accepted (logout must stay idempotent).
*/
const jwt = require('jsonwebtoken');
process.env.JWT_SECRET = 'revocation-forgery-test-secret';
const inserted = [];
jest.mock('../../src/database/db', () => {
const dbFn = () => ({
insert(row) {
inserted.push(row);
return { onConflict: () => ({ ignore: async () => undefined }) };
},
});
return { db: dbFn };
});
jest.mock('../../src/utils/logger', () => ({
info: jest.fn(), warn: jest.fn(), error: jest.fn(), debug: jest.fn(),
}));
const { revokeToken } = require('../../src/utils/tokenRevocation');
const iat = Math.floor(Date.now() / 1000) - 60;
describe('revokeToken signature check', () => {
beforeEach(() => { inserted.length = 0; });
it('refuses a forged three-part token and writes nothing', async () => {
const forgedPayload = Buffer.from(JSON.stringify({
id: 1, iat, type: 'admin', exp: 9e9,
})).toString('base64');
const forged = `eyJhbGciOiJIUzI1NiJ9.${forgedPayload}.notasignature`;
const result = await revokeToken(forged, 'user_logout');
expect(result).toBe(false);
expect(inserted).toHaveLength(0);
});
it('refuses a token signed with a different secret', async () => {
const other = jwt.sign({ id: 1, iat, type: 'admin' }, 'some-other-secret', { expiresIn: '1h' });
expect(await revokeToken(other, 'user_logout')).toBe(false);
expect(inserted).toHaveLength(0);
});
it('revokes a genuine token', async () => {
const genuine = jwt.sign({ id: 1, iat, type: 'admin' }, process.env.JWT_SECRET, { expiresIn: '1h' });
expect(await revokeToken(genuine, 'user_logout')).toBe(true);
expect(inserted).toHaveLength(1);
expect(inserted[0].token_id).toBe(`1-${iat}-admin`);
});
it('still revokes a genuine token that has already expired', async () => {
const expired = jwt.sign({ id: 1, iat, type: 'admin', exp: iat + 1 }, process.env.JWT_SECRET);
expect(await revokeToken(expired, 'user_logout')).toBe(true);
expect(inserted).toHaveLength(1);
});
});
+5 -1
View File
@@ -12,5 +12,9 @@ module.exports = {
testMatch: [
'**/__tests__/**/*.test.js'
],
setupFilesAfterEnv: ['<rootDir>/jest.setup.js']
setupFilesAfterEnv: ['<rootDir>/jest.setup.js'],
// sanitize-html's htmlparser2 12 is ESM-only; see jest.sanitizeHtml.js.
moduleNameMapper: {
'^sanitize-html$': '<rootDir>/jest.sanitizeHtml.js'
}
};
+18
View File
@@ -0,0 +1,18 @@
/**
* sanitize-html 2.17.6+ depends on htmlparser2 12, which ships ESM only.
* Node 22.12+ loads it fine through require(esm); Jest 29's CommonJS module
* registry cannot evaluate an ESM file and fails every suite that imports a
* route or service using the sanitiser. Rather than bolting a Babel
* transform onto node_modules for one dependency, hand this single module to
* Node's own loader.
*
* process.getBuiltinModule (Node 22.3+) is the real core `module` even inside
* Jest — a plain require('module') here returns Jest's wrapper, whose
* createRequire() hands back an empty object for this package. createRequire()
* on the real one resolves from backend/node_modules exactly like production.
*
* Wired in via moduleNameMapper in jest.config.js. The module is stateless,
* so sharing one instance across test files changes nothing; it just cannot
* be jest.mock()ed, and nothing mocks it.
*/
module.exports = process.getBuiltinModule('module').createRequire(__filename)('sanitize-html');
@@ -77,7 +77,11 @@ const DEFAULT_CSS_TEMPLATE = `/*
.photo-card img {
width: 100%;
height: 200px;
/* 100%, not a fixed pixel height: every aspect-ratio layout (masonry,
justified, mosaic, gallery-premium) gives .photo-card a definite height
computed from photos.width/height, and this rule's specificity (0,1,1)
beats the .h-full utility (0,1,0) the layouts rely on — #1131. */
height: 100%;
object-fit: cover;
transition: transform 0.3s ease;
}
@@ -503,7 +503,9 @@ const LIQUID_GLASS_DARK = `/*
.photo-card img {
width: 100%;
height: 240px;
/* See #1131: a fixed height here beats the layouts' .h-full utility and
detaches the image from its aspect-ratio-sized card. */
height: 100%;
object-fit: cover;
transition: transform 0.4s ease, filter 0.4s ease;
filter: brightness(0.9);
@@ -639,7 +641,7 @@ const LIQUID_GLASS_DARK = `/*
}
.photo-card img {
height: 180px;
height: 100%;
}
/* Reduce animation complexity on mobile */
@@ -0,0 +1,123 @@
/**
* The bundled CSS templates pinned every gallery image to a fixed pixel
* height, which broke every aspect-ratio layout (#1131).
*
* Six of the seven layouts size a tile by putting a computed pixel height on
* `.photo-card` and letting the image fill it with `h-full`. A template rule
* of `.photo-card img { height: 200px }` has specificity (0,1,1) and beats
* `.h-full` at (0,1,0), so the image detached from its card: masonry rendered
* correctly-shaped cards with a 200px image glued to the top and empty
* background below — or, where the computed card was shorter than 200px, an
* image taller than its own container.
*
* "Elegant Dark" is seeded `is_default = true`, so this was the out-of-the-box
* result for anyone choosing any layout other than grid/timeline (where a
* fixed square happens to look deliberate).
*
* Migrations 052 and 053 are corrected for fresh installs; this repairs the
* rows already seeded. Templates are referenced by `events.css_template_id`
* and read at serve time rather than copied onto the event, so fixing the row
* fixes every gallery using it.
*
* SCOPE: every `.photo-card img` rule that carries a fixed PIXEL height, in
* every template — not just the two we seeded, and not just their pristine
* copies.
*
* That is broader than it first looks, and deliberately so. It is also not the
* scope this started with: matching the exact seeded text missed every install
* where the template had ever been saved through the editor, because
* `sanitizeCSS` strips newlines. Those are the majority, and a migration that
* silently no-ops on them while being recorded as applied is worse than none.
*
* The cost is that a fixed pixel height a user wrote themselves is rewritten
* too. That is judged acceptable because there is no layout it can be right
* for: all seven give `.photo-card` a definite height and expect the image to
* fill it, so a pixel height on the image can only detach it from its card.
* Anything that is not a fixed px height — %, vh, auto — is left alone, as is
* every declaration outside a `.photo-card img` body.
*/
/**
* Every `.photo-card img { … }` rule body, however it is spaced.
*
* Matching the exact seeded text does NOT work, and the reason is worth
* stating: `sanitizeCSS` strips all control characters (cssSanitizer.js:61),
* so the moment an admin saves a template through the editor — even only to
* rename it or toggle it — every newline is REMOVED from the stored CSS. The
* shipped `.photo-card img {\n height: 200px;` becomes
* `.photo-card img { height: 200px;`. An exact-match migration would find
* nothing on those installs, be recorded as applied, and leave the galleries
* broken with no second chance.
*
* Scoped to the rule body rather than the whole stylesheet, so the other pixel
* heights in these same templates — a 1px gradient divider, an 8px scrollbar —
* are untouched.
*/
/*
* Two details in this pattern are deliberate:
*
* * the selector part is a LIST, so `.photo-card img, .thumbnail img { … }`
* is recognised. Requiring `{` straight after `img` skipped grouped
* selectors entirely — and the migration would still be recorded as
* applied, so the template kept the bug with no second chance.
*
* * the body excludes braces, so a rule containing a NESTED block is not
* matched at all. `.photo-card img { & + .caption { height: 200px } }` is
* valid, passes the validator, and a `[^}]*` body would have captured the
* nested block and rewritten the caption's height instead. Skipping it
* means such a template keeps a fixed image height; corrupting unrelated
* declarations in a migration that cannot be undone is the worse of the
* two, and nesting does not appear in anything we ship.
*/
const PHOTO_CARD_IMG_RULE = /([^{}]*\.photo-card\s+img[^{}]*)\{([^{}]*)\}/g;
/**
* Only a fixed PIXEL height is wrong here; %, vh, auto and the rest stay.
*
* The lookbehind is load-bearing rather than defensive: without it the pattern
* matches the TAIL of `line-height`, `max-height`, `min-height` and any custom
* property ending in `-height`, and silently rewrites those instead — in a
* migration whose down() is deliberately irreversible.
*/
const FIXED_PX_HEIGHT = /(?<![\w-])height\s*:\s*\d+(?:\.\d+)?px/gi;
function relaxFixedImageHeights(css) {
return css.replace(PHOTO_CARD_IMG_RULE, (whole, selectors, body) => {
// .test() on a /g regex advances lastIndex, so it is reset on both sides
// of the check — leaving it set makes the NEXT rule start matching from an
// arbitrary offset and silently skip declarations.
FIXED_PX_HEIGHT.lastIndex = 0;
if (!FIXED_PX_HEIGHT.test(body)) return whole;
FIXED_PX_HEIGHT.lastIndex = 0;
return `${selectors}{${body.replace(FIXED_PX_HEIGHT, 'height: 100%')}}`;
});
}
exports.up = async function up(knex) {
if (!(await knex.schema.hasTable('css_templates'))) return;
const rows = await knex('css_templates').select('id', 'css_content');
let fixed = 0;
for (const row of rows) {
const original = row.css_content;
if (!original || typeof original !== 'string') continue;
const updated = relaxFixedImageHeights(original);
if (updated !== original) {
await knex('css_templates').where({ id: row.id }).update({ css_content: updated });
fixed += 1;
}
}
if (fixed > 0) {
console.log(` 175: relaxed the fixed image height in ${fixed} CSS template(s)`);
}
};
exports.down = async function down() {
// Deliberately irreversible. Putting the pixel heights back would re-break
// every aspect-ratio layout, and the rows may have been edited since — there
// is no version of "restore" here that is safer than doing nothing.
};
@@ -0,0 +1,60 @@
/**
* Migration 176: one row per external file per event (#1162).
*
* The import route checked for an existing external_relpath and then inserted,
* with an fs.stat and a sharp().metadata() call sitting in between — a window
* wide enough that two overlapping imports of the same folder each see "not
* there" and both insert. Nothing at the storage layer stopped them: 041
* created only a NON-unique (event_id, source_origin) index. A reporter's
* event ended up holding 8004 rows for 6012 distinct paths.
*
* So this does two things: clear the duplicates that already exist, and add
* the constraint that makes the race unwinnable from here on.
*
* The work — which row survives, what happens to the guest feedback and admin
* marks hanging off the loser, and why the dependent rows are deleted by hand
* rather than left to ON DELETE CASCADE — lives in
* services/externalPhotoDedupe.js, because a .picpeak restore has to run it
* too: the archive carries the photos table verbatim, so a pre-#1162 backup
* would otherwise hit the unique index mid-restore and roll the whole thing
* back.
*
* Irreversible by design: down() drops the index but cannot resurrect the
* deleted rows. They were never distinct data — the same file counted twice.
*
* What it does NOT do is delete the duplicates' thumbnail files. Those are
* `ext<id>_<name>` keys under the thumbnail root, and a migration is the wrong
* place to reach into storage — the backend may be pointed at S3, and a failed
* object delete must not fail the schema change. They are left behind as
* unreferenced bytes; the storage figures on the dashboard count them, which
* is the correct answer to "what is on the disk".
*/
const {
dedupeExternalPhotos,
createExternalRelpathIndex,
dropExternalRelpathIndex,
} = require('../../src/services/externalPhotoDedupe');
exports.up = async function(knex) {
if (!(await knex.schema.hasTable('photos'))) return;
if (!(await knex.schema.hasColumn('photos', 'external_relpath'))) return;
const removed = await dedupeExternalPhotos(knex);
if (removed) {
console.log(`176_external_relpath_unique: removed ${removed} duplicate external photo row(s)`);
}
// Deliberately unguarded. Recording this migration as applied without the
// index would leave the install permanently racy — the in-flight set only
// covers one process, and the route's unique-violation path cannot converge
// without a constraint to violate — with nothing to trigger a retry. A
// failure here means the dedupe above did not achieve uniqueness, which is
// worth stopping the upgrade for.
await createExternalRelpathIndex(knex);
};
exports.down = async function(knex) {
if (!(await knex.schema.hasTable('photos'))) return;
await dropExternalRelpathIndex(knex);
};
@@ -0,0 +1,52 @@
/**
* Migration 177: external_relpath becomes relative to EXTERNAL_MEDIA_ROOT (#1163).
*
* It used to be relative to events.external_path, which every import
* overwrites — so importing a second folder into an event rebased every photo
* already in it onto the new folder. Nothing errored. Thumbnails are written to
* local storage during the import while the base path is still correct, so the
* grid kept rendering and only the things that need the ORIGINAL broke: preview
* generation, the lightbox, downloads. The reporter had 7547 of 8004 rows
* resolving to files that do not exist, and spent a while chasing it as a CPU
* problem.
*
* The work — including the on-disk repair of events that have already been
* rebased, and why it refuses to guess below current behaviour — lives in
* services/externalRelpathFold.js, because a .picpeak restore has to run it
* too: knex_migrations is excluded from the archive, so a pre-#1163 backup
* lands base-relative rows on an instance that has already migrated.
*/
const { foldExternalRelpaths } = require('../../src/services/externalRelpathFold');
exports.up = async function(knex) {
try {
await foldExternalRelpaths(knex, (msg) => console.log(`177_external_relpath_from_root: ${msg}`));
} catch (err) {
// Re-thrown WITHOUT the driver's error code. run-migrations-safe.js treats
// 23505 / 42P07 / 42701 / 42710 as "schema already exists" and marks the
// migration applied (run-migrations-safe.js:138) — so a unique-violation
// rolling this fold back would be recorded as a success, leaving every
// external path in the old format under a resolver that reads them
// differently, with nothing to trigger a retry.
throw new Error(
`177_external_relpath_from_root failed and was rolled back: ${err.message}. `
+ 'External photo paths are unchanged; resolve the cause and re-run the migration.'
);
}
};
/**
* Irreversible by design, and a deliberate no-op rather than a partial undo.
*
* The base each row was folded with is not recorded anywhere:
* events.external_path holds whatever the LAST import set, which for a
* repaired row is the wrong answer and is exactly what broke these installs.
* Stripping it back off would re-break them.
*
* The idempotency marker stays for the same reason — clearing it would let
* up() run a second time and fold every path twice.
*/
exports.down = async function() {
console.log('177_external_relpath_from_root: rollback is a no-op (see header)');
};
@@ -0,0 +1,55 @@
/**
* Migration 178: drop preview keys written by the old generator.
*
* generatePreviewImage used to keep the SOURCE basename verbatim, extension and
* all, while always writing JPEG bytes. So a `.webp` upload produced
* `previews/preview_shot.webp` holding a JPEG, and a `.png` upload produced
* `preview_logo.png` holding a JPEG.
*
* That was harmless while the preview route hard-coded `Content-Type:
* image/jpeg`. It stopped being harmless the moment the encoding started
* varying: the route now reads the extension, so a legacy `.webp` key is
* announced as `image/webp` while containing JPEG — and the response carries
* `X-Content-Type-Options: nosniff`, so the browser will not quietly correct
* it. The lightbox shows a broken image for every photo that happened to be
* uploaded as WebP.
*
* The legacy `.png`-keyed previews are wrong in the other direction: they are
* flattened JPEGs of what may have been a transparent source, which is the
* defect the new encoder fixes and which `isPreviewValid` would otherwise let
* stand forever.
*
* Clearing the column is the whole repair. Previews are lazily regenerated by
* ensurePreviewImage on the next open, under the new naming and the new
* encoder, so the only cost is one regeneration per photo that is actually
* viewed. Nothing is deleted from storage — a migration is the wrong place to
* reach into a backend that may be S3 — so the old objects linger as
* unreferenced bytes, which the storage breakdown counts honestly.
*
* Deliberately clears ALL of them, not just the ones whose extension looks
* suspicious. A `.jpg`-keyed legacy preview is byte-correct, but it may still
* be a flattened rendition of a transparent or animated source, and there is
* no way to tell from the key. One lazy regeneration is cheaper than reasoning
* about which of them lied.
*/
exports.up = async function(knex) {
if (!(await knex.schema.hasTable('photos'))) return;
if (!(await knex.schema.hasColumn('photos', 'preview_path'))) return;
const cleared = await knex('photos')
.whereNotNull('preview_path')
.update({ preview_path: null });
if (cleared) {
console.log(`178_reset_legacy_preview_paths: cleared ${cleared} preview key(s); they regenerate on next view`);
}
};
/**
* Irreversible by design, and harmless: the column held a cache key, and the
* cache rebuilds itself. There is nothing to restore.
*/
exports.down = async function() {
console.log('178_reset_legacy_preview_paths: rollback is a no-op (preview keys are a regenerable cache)');
};
@@ -0,0 +1,73 @@
/**
* Migration 179: shared run state for the maintenance sweeps (#1181).
*
* Both photo maintenance jobs — the dimension repair and the capture-date
* backfill — tracked whether they were running in a module-level variable. On
* a single-replica install that is correct. Behind a load balancer it is not:
* the flag lives in one process, so a status poll routed to any other replica
* answers `isRunning: false`, the UI re-enables the button, and the next POST
* lands somewhere else and starts a second pass over the whole library. Both
* replicas then read and parse every original off S3 or the NAS mount. The
* `.whereNull(...)` guards on the writes mean nothing is corrupted — the cost
* is the duplicated I/O, and an operator who cannot tell whether a job is
* running.
*
* One row per job, claimed with a conditional UPDATE so the claim itself is
* the mutual exclusion — the same UPDATE-with-guard shape backgroundProcessor
* already uses to hand a photo to exactly one worker
* (services/backgroundProcessor.js:110-116).
*
* heartbeat_at exists because a lock with no expiry is worse than no lock: a
* replica that is OOM-killed mid-run would leave is_running = true forever and
* no way to clear it short of editing the database. The runner touches it as
* it goes, and a claim is allowed to take over a run whose heartbeat has gone
* quiet. See services/maintenanceJobState.js for the read side, which reports
* a stale run as not-running so the button comes back on its own.
*
* Rows are seeded here rather than created on demand so the claim is a plain
* UPDATE with no insert race behind it.
*/
const JOBS = ['photo_dimension_repair', 'photo_capture_date_backfill'];
exports.up = async function (knex) {
const exists = await knex.schema.hasTable('maintenance_jobs');
if (!exists) {
await knex.schema.createTable('maintenance_jobs', (t) => {
// The job's identity, not a surrogate key: there is exactly one row per
// job and every access is by name, so the name is the primary key.
t.string('job_name', 64).primary();
t.boolean('is_running').notNullable().defaultTo(false);
t.timestamp('started_at').nullable();
t.timestamp('heartbeat_at').nullable();
t.timestamp('finished_at').nullable();
// JSON as text: the shape differs per job (the backfill reports a third
// counter the dimension repair has no equivalent for) and nothing
// queries into it, so a json column would buy nothing and cost engine
// differences between Postgres and SQLite.
t.text('last_result').nullable();
// Diagnostics only — which process is holding the claim.
t.string('owner', 128).nullable();
// The fencing token. Unique per claim, not per process: after a stale
// takeover the old runner may still be alive and mid-loop, and it can
// even be the same process that re-claimed. Every write it makes is
// scoped to the token it was handed, so a superseded runner can neither
// renew a claim it has lost nor release one it no longer owns.
t.string('claim_token', 64).nullable();
});
console.log('179: created maintenance_jobs');
}
// Idempotent on re-run and safe against a table that already carries rows.
for (const jobName of JOBS) {
const row = await knex('maintenance_jobs').where({ job_name: jobName }).first();
if (!row) {
await knex('maintenance_jobs').insert({ job_name: jobName, is_running: false });
console.log(`179: seeded job row ${jobName}`);
}
}
};
exports.down = async function (knex) {
await knex.schema.dropTableIfExists('maintenance_jobs');
};
+126 -21
View File
@@ -1,12 +1,12 @@
{
"name": "picpeak-backend",
"version": "3.46.0",
"version": "3.46.8",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "picpeak-backend",
"version": "3.46.0",
"version": "3.46.8",
"dependencies": {
"@aws-sdk/client-s3": "^3.850.0",
"@aws-sdk/lib-storage": "^3.850.0",
@@ -49,7 +49,7 @@
"postcss": "8.5.23",
"qrcode": "^1.5.4",
"react-i18next": "^15.6.0",
"sanitize-html": "2.17.5",
"sanitize-html": "2.17.7",
"sharp": "0.35.3",
"sqlite3": "^5.1.6",
"swagger-jsdoc": "^6.2.8",
@@ -68,7 +68,7 @@
"supertest": "^6.3.3"
},
"engines": {
"node": "^20.19.0 || >=22"
"node": ">=22.12.0"
}
},
"node_modules/@apidevtools/json-schema-ref-parser": {
@@ -10469,12 +10469,13 @@
}
},
"node_modules/qs": {
"version": "6.15.2",
"resolved": "https://registry.npmjs.org/qs/-/qs-6.15.2.tgz",
"integrity": "sha512-Rzq0KEyX/w/tEybncDgdkZrJgVUsUMk3xjh3t5bv3S1HTAtg+uOYt72+ZfwiQwKdysThkTBdL/rTi6HDmX9Ddw==",
"version": "6.16.0",
"resolved": "https://registry.npmjs.org/qs/-/qs-6.16.0.tgz",
"integrity": "sha512-h6fhOIaRrID2CbEY2fqs+7t+UXZo+MLAnU5gRIq85uFtdiUPCdsApMlHhXogKVM4HM2DVbIjGNTTYH2OcmP1vA==",
"license": "BSD-3-Clause",
"dependencies": {
"side-channel": "^1.1.0"
"es-define-property": "^1.0.1",
"side-channel": "^1.1.1"
},
"engines": {
"node": ">=0.6"
@@ -10852,18 +10853,122 @@
"license": "MIT"
},
"node_modules/sanitize-html": {
"version": "2.17.5",
"resolved": "https://registry.npmjs.org/sanitize-html/-/sanitize-html-2.17.5.tgz",
"integrity": "sha512-ZmU1joGRrvoyctKIiuwUxqR6moLoU2Wk+2bMccN6f7UwhAmwYDvWziqPxRDDN2Qip62NqnIrVrT9akbL6Wretg==",
"version": "2.17.7",
"resolved": "https://registry.npmjs.org/sanitize-html/-/sanitize-html-2.17.7.tgz",
"integrity": "sha512-PGtEkc9cbnedU3s9TmzDbpsZ8w086g/0Q8k8/oIO1NLNU3i5k9yn835CrjJSajp1KMmkisbO1qPXxNKO3welAg==",
"license": "MIT",
"dependencies": {
"deepmerge": "^4.2.2",
"escape-string-regexp": "^4.0.0",
"htmlparser2": "^10.1.0",
"htmlparser2": "^12.0.0",
"is-plain-object": "^5.0.0",
"launder": "^1.7.1",
"parse-srcset": "^1.0.2",
"postcss": "^8.3.11"
},
"engines": {
"node": ">=22.12.0"
}
},
"node_modules/sanitize-html/node_modules/dom-serializer": {
"version": "3.1.1",
"resolved": "https://registry.npmjs.org/dom-serializer/-/dom-serializer-3.1.1.tgz",
"integrity": "sha512-4MEa38/QexBob6gFNwu+EGdWvhJ1OKuNwdYY3Y3NyeWDQfnGeDYQUDfIRzWu5B5gsv03so2Uxd28YC6zrsx3Lw==",
"license": "MIT",
"dependencies": {
"domelementtype": "^3.0.0",
"domhandler": "^6.0.0",
"entities": "^8.0.0"
},
"engines": {
"node": ">=20.19.0"
},
"funding": {
"type": "github",
"url": "https://github.com/cheeriojs/dom-serializer?sponsor=1"
}
},
"node_modules/sanitize-html/node_modules/domelementtype": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-3.0.0.tgz",
"integrity": "sha512-umCQid3jKbDmVjx8jGaW7uUykm4DEUeyV21hPxNMo2nV955DhUThwqyOIDtreepP31hl84X7G5U9ZfsWvIB3Pg==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/fb55"
}
],
"license": "BSD-2-Clause",
"engines": {
"node": ">=20.19.0"
}
},
"node_modules/sanitize-html/node_modules/domhandler": {
"version": "6.0.1",
"resolved": "https://registry.npmjs.org/domhandler/-/domhandler-6.0.1.tgz",
"integrity": "sha512-gYzvtM72ZtxQO0T048kd6HWSbbGCNOUwcnfQ01cqIJ4X2IYKFFHZ5mKvrQETcFXxsRObZulDaKmy//R7TPtsBg==",
"license": "BSD-2-Clause",
"dependencies": {
"domelementtype": "^3.0.0"
},
"engines": {
"node": ">=20.19.0"
},
"funding": {
"type": "github",
"url": "https://github.com/fb55/domhandler?sponsor=1"
}
},
"node_modules/sanitize-html/node_modules/domutils": {
"version": "4.0.2",
"resolved": "https://registry.npmjs.org/domutils/-/domutils-4.0.2.tgz",
"integrity": "sha512-qI4JLRKnSzqFqr7hAlS5xQDusBCjKSEG4t4+7aNrIQMHBcsC2TGEhuyABJdYkgSewL57PNLYEiibY2iPKhKpaA==",
"license": "BSD-2-Clause",
"dependencies": {
"dom-serializer": "^3.0.0",
"domelementtype": "^3.0.0",
"domhandler": "^6.0.0"
},
"engines": {
"node": ">=20.19.0"
},
"funding": {
"type": "github",
"url": "https://github.com/fb55/domutils?sponsor=1"
}
},
"node_modules/sanitize-html/node_modules/entities": {
"version": "8.0.0",
"resolved": "https://registry.npmjs.org/entities/-/entities-8.0.0.tgz",
"integrity": "sha512-zwfzJecQ/Uej6tusMqwAqU/6KL2XaB2VZ2Jg54Je6ahNBGNH6Ek6g3jjNCF0fG9EWQKGZNddNjU5F1ZQn/sBnA==",
"license": "BSD-2-Clause",
"engines": {
"node": ">=20.19.0"
},
"funding": {
"url": "https://github.com/fb55/entities?sponsor=1"
}
},
"node_modules/sanitize-html/node_modules/htmlparser2": {
"version": "12.0.0",
"resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-12.0.0.tgz",
"integrity": "sha512-Tz7u1i95/g2x2jz81+x0FBVhBhY5aRTvD3tXXdFaljuNdzDLJ8UGNRrTcj2cgQvAg3iW/h77Fz15nLW0L0CrZw==",
"funding": [
"https://github.com/fb55/htmlparser2?sponsor=1",
{
"type": "github",
"url": "https://github.com/sponsors/fb55"
}
],
"license": "MIT",
"dependencies": {
"domelementtype": "^3.0.0",
"domhandler": "^6.0.0",
"domutils": "^4.0.2",
"entities": "^8.0.0"
},
"engines": {
"node": ">=20.19.0"
}
},
"node_modules/selderee": {
@@ -11038,14 +11143,14 @@
}
},
"node_modules/side-channel": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/side-channel/-/side-channel-1.1.0.tgz",
"integrity": "sha512-ZX99e6tRweoUXqR+VBrslhda51Nh5MTQwou5tnUDgbtyM0dBgmhEDtWGP/xbKn6hqfPRHujUNwz5fy/wbbhnpw==",
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/side-channel/-/side-channel-1.1.1.tgz",
"integrity": "sha512-6x6dK6zJdpTzF4sQeNYxwtvBzf6Eg4GtlesS94HOvTudUeyK2WXAaIfmDgsyslYrRBeFIlsi54AYsFGUuhmvrQ==",
"license": "MIT",
"dependencies": {
"es-errors": "^1.3.0",
"object-inspect": "^1.13.3",
"side-channel-list": "^1.0.0",
"object-inspect": "^1.13.4",
"side-channel-list": "^1.0.1",
"side-channel-map": "^1.0.1",
"side-channel-weakmap": "^1.0.2"
},
@@ -11057,13 +11162,13 @@
}
},
"node_modules/side-channel-list": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/side-channel-list/-/side-channel-list-1.0.0.tgz",
"integrity": "sha512-FCLHtRD/gnpCiCHEiJLOwdmFP+wzCmDEkc9y7NsYxeF4u7Btsn1ZuwgwJGxImImHicJArLP4R0yX4c2KCrMrTA==",
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/side-channel-list/-/side-channel-list-1.0.1.tgz",
"integrity": "sha512-mjn/0bi/oUURjc5Xl7IaWi/OJJJumuoJFQJfDDyO46+hBWsfaVM65TBHq2eoZBhzl9EchxOijpkbRC8SVBQU0w==",
"license": "MIT",
"dependencies": {
"es-errors": "^1.3.0",
"object-inspect": "^1.13.3"
"object-inspect": "^1.13.4"
},
"engines": {
"node": ">= 0.4"
+3 -3
View File
@@ -1,10 +1,10 @@
{
"name": "picpeak-backend",
"version": "3.46.2",
"version": "3.46.10",
"description": "Backend for PicPeak event photo sharing platform",
"main": "server.js",
"engines": {
"node": "^20.19.0 || >=22"
"node": ">=22.12.0"
},
"scripts": {
"start": "node server.js",
@@ -58,7 +58,7 @@
"postcss": "8.5.23",
"qrcode": "^1.5.4",
"react-i18next": "^15.6.0",
"sanitize-html": "2.17.5",
"sanitize-html": "2.17.7",
"sharp": "0.35.3",
"sqlite3": "^5.1.6",
"swagger-jsdoc": "^6.2.8",
+135 -129
View File
@@ -1,141 +1,147 @@
#!/usr/bin/env node
/**
* Script to regenerate missing thumbnails for photos in the database
* Usage: node scripts/regenerate-thumbnails.js [eventId]
* Fill in missing thumbnails for photos already in the database.
*
* The CLI fallback for when the admin UI is not reachable. It is deliberately
* "missing only": ensureThumbnail short-circuits on a thumbnail that is
* already present and valid, so re-running this is cheap and safe. To REBUILD
* everything after a settings change, use POST /api/admin/thumbnails/regenerate
* — that path drops the existing renditions first, which this one must not do.
*
* Resolution goes through ensureThumbnail rather than a hand-built path
* (#1148, same defect as #1129). This script used to compute
* `storage/events/active/<photo.path>` and fs.access it, a location that does
* not exist for `external` or `reference` rows — their originals live under
* the mount in events.external_path. Every such photo failed the check and was
* counted as an error, so on an external-media install the script was inert
* while reporting one error per photo.
*
* ensureThumbnail already branches on source_origin, resolves both kinds via
* photoResolver, uses the per-photo `ext<id>_` output name so two events
* referencing one NAS basename cannot clobber each other, and writes
* thumbnail_path back itself. Sharing it is what stops the script and the
* route drifting apart again.
*
* Usage:
* node scripts/regenerate-thumbnails.js [eventId]
*/
const path = require('path');
const fs = require('fs').promises;
const sharp = require('sharp');
const { db } = require('../src/database/db');
// Configuration
const THUMBNAIL_SIZE = 300;
const STORAGE_PATH = process.env.STORAGE_PATH || path.join(__dirname, '../../storage');
const THUMBNAILS_DIR = path.join(STORAGE_PATH, 'thumbnails');
async function ensureDirectoryExists(dirPath) {
try {
await fs.access(dirPath);
} catch {
await fs.mkdir(dirPath, { recursive: true });
console.log(`Created directory: ${dirPath}`);
}
}
async function generateThumbnail(photoPath, thumbnailPath) {
try {
await sharp(photoPath)
.resize(THUMBNAIL_SIZE, THUMBNAIL_SIZE, {
fit: 'cover',
position: 'center'
})
.jpeg({ quality: 80 })
.toFile(thumbnailPath);
return true;
} catch (error) {
console.error(`Failed to generate thumbnail for ${photoPath}:`, error.message);
return false;
}
}
const { ensureThumbnail, isThumbnailValid } = require('../src/services/imageProcessor');
async function regenerateThumbnails(eventId = null) {
try {
console.log('Starting thumbnail regeneration...');
console.log(`Storage path: ${STORAGE_PATH}`);
console.log(`Thumbnails directory: ${THUMBNAILS_DIR}`);
// Ensure thumbnails directory exists
await ensureDirectoryExists(THUMBNAILS_DIR);
// Build query
let query = db('photos')
.join('events', 'photos.event_id', 'events.id')
.select(
'photos.id',
'photos.filename',
'photos.path',
'photos.thumbnail_path',
'events.slug as event_slug'
);
if (eventId) {
query = query.where('photos.event_id', eventId);
console.log(`Filtering for event ID: ${eventId}`);
}
const photos = await query;
console.log(`Found ${photos.length} photos to process`);
let successCount = 0;
let skipCount = 0;
let errorCount = 0;
for (const photo of photos) {
const photoPath = path.join(STORAGE_PATH, 'events/active', photo.path);
const thumbnailFilename = `thumb_${photo.filename}`;
const thumbnailPath = path.join(THUMBNAILS_DIR, thumbnailFilename);
try {
// Check if photo file exists
await fs.access(photoPath);
// Check if thumbnail already exists
try {
await fs.access(thumbnailPath);
console.log(`Thumbnail already exists for ${photo.filename}, skipping...`);
skipCount++;
continue;
} catch {
// Thumbnail doesn't exist, generate it
}
console.log(`Generating thumbnail for ${photo.filename}...`);
const success = await generateThumbnail(photoPath, thumbnailPath);
if (success) {
// Update database with thumbnail path
await db('photos')
.where('id', photo.id)
.update({
thumbnail_path: `thumbnails/${thumbnailFilename}`
});
successCount++;
console.log(`✓ Generated thumbnail for ${photo.filename}`);
} else {
errorCount++;
}
} catch (error) {
console.error(`✗ Photo file not found: ${photoPath}`);
errorCount++;
}
}
console.log('\nThumbnail regeneration complete!');
console.log(`- Successfully generated: ${successCount}`);
console.log(`- Skipped (already exist): ${skipCount}`);
console.log(`- Errors: ${errorCount}`);
console.log(`- Total processed: ${photos.length}`);
} catch (error) {
console.error('Error during thumbnail regeneration:', error);
process.exit(1);
} finally {
await db.destroy();
console.log('Starting thumbnail regeneration...');
// These columns are what ensureThumbnail branches on to resolve a source and
// name its output. Selecting a subset that misses
// source_origin/external_relpath is how the old path bug would come back —
// an external row would look managed and resolve under events/active.
let query = db('photos').select(
'id', 'event_id', 'path', 'filename', 'thumbnail_path',
'type', 'media_type', 'mime_type', 'source_origin', 'external_relpath'
);
if (eventId) {
query = query.where('event_id', eventId);
console.log(`Filtering for event ID: ${eventId}`);
}
// Skip videos. A video's thumbnail is a poster frame produced by
// videoProcessor, not a resize of the stored file, so handing the container
// to Sharp here only ever produced one error per row.
//
// Tested on every marker a video row can carry, not media_type alone:
// fileWatcher.processNewPhoto writes `type` and `mime_type` but never
// media_type, which defaults to 'image' — so an auto-imported video passes a
// media_type-only filter. Each clause is null-safe on its own so a row that
// simply has no mime_type is not swept up with them.
query = query
.where(function () {
this.whereNull('media_type').orWhere('media_type', '!=', 'video');
})
.where(function () {
this.whereNull('type').orWhere('type', '!=', 'video');
})
.where(function () {
this.whereNull('mime_type').orWhereNot('mime_type', 'like', 'video/%');
});
const photos = await query;
console.log(`Found ${photos.length} photos to process`);
let successCount = 0;
let skipCount = 0;
let errorCount = 0;
for (const photo of photos) {
const label = photo.filename || `photo ${photo.id}`;
try {
const existing = photo.thumbnail_path;
// Asked BEFORE the call, not inferred from the returned path afterwards.
// On local and external storage the key is deterministic, so repairing a
// missing or corrupt thumbnail hands back the identical string — and
// comparing paths would report that repair as "already valid", which is
// the one number an operator running this is actually reading.
const wasValid = existing ? await isThumbnailValid(existing) : false;
const thumbnailPath = await ensureThumbnail(photo);
if (!thumbnailPath) {
console.error(`✗ Could not generate thumbnail for ${label}`);
errorCount++;
continue;
}
if (wasValid && thumbnailPath === existing) {
skipCount++;
} else {
successCount++;
console.log(`✓ Generated thumbnail for ${label}`);
}
} catch (error) {
console.error(`✗ Failed for ${label}: ${error.message}`);
errorCount++;
}
}
console.log('\nThumbnail regeneration complete!');
console.log(`- Generated: ${successCount}`);
console.log(`- Skipped (already valid): ${skipCount}`);
console.log(`- Errors: ${errorCount}`);
console.log(`- Total processed: ${photos.length}`);
return { successCount, skipCount, errorCount };
}
// Parse command line arguments
const eventId = process.argv[2] ? parseInt(process.argv[2]) : null;
if (require.main === module) {
const args = process.argv.slice(2);
const eventArg = args.find((a) => !a.startsWith('--'));
const eventId = eventArg ? parseInt(eventArg, 10) : null;
// Run the script
regenerateThumbnails(eventId).then(() => {
console.log('Script completed successfully');
process.exit(0);
}).catch(error => {
console.error('Script failed:', error);
process.exit(1);
});
if (eventArg && !Number.isInteger(eventId)) {
console.error(`Not an event id: ${eventArg}`);
process.exit(1);
}
regenerateThumbnails(eventId)
.then(async (result) => {
await db.destroy();
// Exit status is the only thing a cron job reads. Resolving with a
// nonzero errorCount and still exiting 0 told automation the backfill
// was done when it had failed — which is how an unavailable mount stays
// unnoticed until someone opens a gallery.
if (result.errorCount) {
console.error(`Script completed with failures: ${result.errorCount} photo(s)`);
process.exit(1);
}
console.log('Script completed successfully');
process.exit(0);
})
.catch(async (error) => {
console.error('Script failed:', error);
await db.destroy().catch(() => {});
process.exit(1);
});
}
module.exports = { regenerateThumbnails };
+55 -27
View File
@@ -206,25 +206,13 @@ app.use((req, res, next) => {
});
// CORS configuration (apply only to API routes)
const { isAllowedOrigin, multipartOriginAllowed } = require('./src/utils/requestOrigin');
const corsOptions = {
origin: function (origin, callback) {
const allowedOrigins = [
process.env.FRONTEND_URL || 'http://localhost:3005',
process.env.ADMIN_URL || 'http://localhost:3005'
];
// In development, also allow localhost origins
if (process.env.NODE_ENV === 'development') {
allowedOrigins.push(
'http://localhost:5173', // Vite dev server
'http://localhost:3002', // Backend server
'http://localhost:3001', // For API testing
'http://localhost:3000' // Direct backend access
);
}
// Allowlist lives in utils/requestOrigin, shared with the multipart gate.
// Allow requests with no origin (like curl) and allow-listed origins
if (!origin || allowedOrigins.indexOf(origin) !== -1) {
if (!origin || isAllowedOrigin(origin)) {
callback(null, true);
} else {
// Do not error globally; just omit CORS headers on disallowed origins
@@ -452,8 +440,14 @@ async function initializeRateLimiters() {
}
// Note: Rate limiters will be initialized after database connection
app.use(express.json({ limit: '50mb' }));
app.use(express.urlencoded({ extended: true, limit: '50mb' }));
// Body limits. 50mb is only needed by the authenticated admin and API-token
// surfaces (restore manifests, CMS and email templates, bulk operations);
// applied globally it let any unauthenticated caller hand JSON.parse a 50mb
// body and block the event loop. express.json skips a request whose body
// is already parsed, so the scoped parser must run first.
app.use(['/api/admin', '/api/v1'], express.json({ limit: '50mb' }));
app.use(express.json({ limit: '2mb' }));
app.use(express.urlencoded({ extended: true, limit: '2mb' }));
// CSRF protection: require JSON Content-Type on mutating API requests
// This blocks cross-origin form submissions which cannot set Content-Type: application/json
@@ -465,6 +459,14 @@ app.use('/api', (req, res, next) => {
if (contentLength > 0 && !contentType.includes('application/json') && !contentType.includes('multipart/form-data')) {
return res.status(415).json({ error: 'Unsupported Content-Type. Use application/json or multipart/form-data.' });
}
// multipart is exactly what a cross-site <form> can send without a
// preflight, and in a split-origin deployment (SameSite=None) the admin
// cookie rides along to the upload routes. Browsers label such a
// submission Sec-Fetch-Site: cross-site (and always send Origin on a
// cross-origin POST); non-browser clients send neither header and pass.
if (contentType.includes('multipart/form-data') && !multipartOriginAllowed(req)) {
return res.status(403).json({ error: 'Cross-site multipart request rejected' });
}
}
next();
});
@@ -521,14 +523,35 @@ const secureStatic = require('./src/middleware/secureStatic');
const storagePath = process.env.STORAGE_PATH || path.join(__dirname, '../storage');
process.env.EXTERNAL_MEDIA_ROOT = process.env.EXTERNAL_MEDIA_ROOT || '/external-media';
// Static file serving for photos (protected)
app.use('/photos', require('./src/middleware/photoAuth'), setCorsHeaders, secureStatic(path.join(storagePath, 'events/active')));
// The /photos and /thumbnails static mounts are gone.
//
// They served the raw originals tree and the thumbnail tree behind photoAuth
// alone, which authorises on a slug match. A static file server cannot apply
// the rules the gallery API applies per photo, so everything the API decides
// was simply absent here: allow_downloads, per-category allow_downloads,
// watermarking, the resolution cap, reveal-mode windows, visibility='hidden',
// download logging, and the customer-assignment re-check that lets an admin
// revoke access immediately. The filenames needed to exercise it are handed to
// every guest in the photos listing.
//
// Nothing builds these URLs: no reference in frontend/src, none in the email
// templates, and the only backend mentions are the /api/admin/photos/... API
// routes and a maintenance-mode prefix list. nginx still proxies /photos and
// /thumbnails; those locations now 404, which is the intended outcome.
//
// Serving these safely would mean reimplementing per-photo authorisation and
// image processing inside a static handler -- i.e. the gallery API, which
// already exists at /api/gallery/:slug/photo/:id and /thumbnail/:id.
// Static file serving for thumbnails (protected)
app.use('/thumbnails', require('./src/middleware/photoAuth'), setCorsHeaders, secureStatic(path.join(storagePath, 'thumbnails')));
// Static file serving for uploads (public - logos, favicons)
app.use('/uploads', setCorsHeaders, secureStatic(path.join(storagePath, 'uploads')));
// Static file serving for uploads.
//
// Narrowed to the two public asset trees. The mount used to expose the whole
// uploads/ root with no auth middleware at all, and that root also holds
// signed contract PDFs (uploads/contracts/signed) and client transfer files
// (uploads/transfers/<id>) -- both reachable by anyone who learned or guessed
// a filename. Those are served by their own authorised routes.
app.use('/uploads/logos', setCorsHeaders, secureStatic(path.join(storagePath, 'uploads/logos')));
app.use('/uploads/favicons', setCorsHeaders, secureStatic(path.join(storagePath, 'uploads/favicons')));
// Static file serving for self-hosted webfonts (public — gallery visitors
// load these via @font-face). Replaces the previous Google Fonts CDN
@@ -695,10 +718,15 @@ app.get(
// whereas Firefox/Chrome do — so a 302 worked everywhere except
// Safari. sendFile sets the right content-type from the extension.
const rel = String(url).replace(/^\/+/, '').replace(/^uploads\//, '');
// Containment is the two public asset trees, not the whole uploads/
// root: that root also holds signed contracts and client transfer
// files, and the favicon URL is an admin-writable setting, so the
// wider check let `/uploads/contracts/signed/<file>` be served here
// unauthenticated with a day of cache.
const uploadsRoot = path.resolve(path.join(storagePath, 'uploads'));
const resolved = path.resolve(path.join(uploadsRoot, rel));
// Path containment — never serve outside the uploads dir.
if (resolved.startsWith(uploadsRoot + path.sep) && fs.existsSync(resolved)) {
const servableRoots = ['favicons', 'logos'].map((d) => path.join(uploadsRoot, d) + path.sep);
if (servableRoots.some((root) => resolved.startsWith(root)) && fs.existsSync(resolved)) {
// This route streams the file directly, bypassing the secureStatic
// middleware — so re-apply its SVG hardening here. An admin-uploaded
// SVG favicon could contain <script>; served at the top-level
+2 -166
View File
@@ -4,7 +4,7 @@ const { formatBoolean } = require('../utils/dbCompat');
const { isMissingRolesSchema } = require('../utils/dbErrors');
const { isTokenRevoked } = require('../utils/tokenRevocation');
const logger = require('../utils/logger');
const { getAdminTokenFromRequest, getGalleryTokenFromRequest } = require('../utils/tokenUtils');
const { getAdminTokenFromRequest } = require('../utils/tokenUtils');
/**
* Enhanced admin authentication middleware with revocation checking
@@ -136,170 +136,6 @@ async function adminAuth(req, res, next) {
}
}
/**
* Enhanced gallery authentication middleware with revocation checking
*/
async function galleryAuth(req, res, next) {
try {
const slug = req.params?.slug || req.requestedSlug;
const token = getGalleryTokenFromRequest(req, slug);
if (!token) {
return res.status(401).json({ error: 'No token provided' });
}
let decoded;
try {
decoded = jwt.verify(token, process.env.JWT_SECRET, {
algorithms: ['HS256'],
issuer: 'picpeak-auth',
complete: true
});
decoded = decoded.payload;
} catch (err) {
if (err.name === 'TokenExpiredError') {
return res.status(401).json({ error: 'Session expired', code: 'TOKEN_EXPIRED' });
}
return res.status(401).json({ error: 'Invalid session' });
}
// Check if token is revoked
if (await isTokenRevoked(decoded)) {
return res.status(401).json({ error: 'Session has been invalidated', code: 'TOKEN_REVOKED' });
}
// Verify token type
if (decoded.type !== 'gallery') {
return res.status(403).json({ error: 'Invalid access token' });
}
// Check if event still exists and is active
const event = await db('events')
.where({
id: decoded.eventId,
is_active: true,
is_archived: false
})
.first();
if (!event) {
return res.status(404).json({ error: 'Gallery not found or expired' });
}
// Check if gallery has expired (only if expires_at is set)
// Galleries with null expires_at never expire
if (event.expires_at && new Date(event.expires_at) < new Date()) {
return res.status(410).json({
error: 'Gallery has expired',
code: 'GALLERY_EXPIRED'
});
}
// Add event info to request
req.event = event;
req.galleryToken = decoded;
req.token = token;
next();
} catch (error) {
logger.error('Gallery auth middleware error:', error);
res.status(401).json({ error: 'Authentication failed' });
}
}
/**
* Photo access authentication
* Validates both admin and gallery tokens for photo access
*/
async function photoAuth(req, res, next) {
try {
const slug = req.params?.slug || req.requestedSlug;
const token = getAdminTokenFromRequest(req) || getGalleryTokenFromRequest(req, slug);
if (!token) {
return res.status(401).json({ error: 'Authentication required' });
}
let decoded;
try {
decoded = jwt.verify(token, process.env.JWT_SECRET, { algorithms: ['HS256'] });
} catch (err) {
return res.status(401).json({ error: 'Invalid token' });
}
// Check if token is revoked
if (await isTokenRevoked(decoded)) {
return res.status(401).json({ error: 'Token has been revoked', code: 'TOKEN_REVOKED' });
}
// Allow both admin and gallery tokens
if (decoded.type === 'admin') {
const admin = await db('admin_users')
.where({ id: decoded.id, is_active: formatBoolean(true) })
.first();
if (!admin) {
return res.status(401).json({ error: 'Invalid token' });
}
req.auth = { type: 'admin', user: admin };
} else if (decoded.type === 'gallery') {
const event = await db('events')
.where({
id: decoded.eventId,
is_active: true,
is_archived: false
})
.first();
if (!event) {
return res.status(404).json({ error: 'Gallery not found' });
}
// For gallery tokens, ensure they can only access their event's photos
req.auth = { type: 'gallery', event: event };
} else {
return res.status(403).json({ error: 'Invalid token type' });
}
next();
} catch (error) {
logger.error('Photo auth middleware error:', error);
res.status(401).json({ error: 'Authentication failed' });
}
}
/**
* Verify gallery access for specific operations
*/
async function verifyGalleryAccess(req, res, next) {
try {
if (!req.auth) {
return res.status(401).json({ error: 'Authentication required' });
}
const { eventId } = req.params;
// Admins can access any gallery
if (req.auth.type === 'admin') {
return next();
}
// Gallery tokens can only access their own event
if (req.auth.type === 'gallery') {
if (req.auth.event.id !== parseInt(eventId)) {
return res.status(403).json({ error: 'Access denied' });
}
return next();
}
res.status(403).json({ error: 'Access denied' });
} catch (error) {
res.status(500).json({ error: 'Access verification failed' });
}
}
module.exports = {
adminAuth,
galleryAuth,
photoAuth,
verifyGalleryAccess
adminAuth
};
-176
View File
@@ -1,176 +0,0 @@
const bcrypt = require('bcrypt');
const jwt = require('jsonwebtoken');
const { db } = require('../database/db');
const { formatBoolean } = require('../utils/dbCompat');
const { getGalleryTokenFromRequest } = require('../utils/tokenUtils');
const { isTokenRevoked } = require('../utils/tokenRevocation');
const logger = require('../utils/logger');
async function photoAuth(req, res, next) {
try {
// Extract event slug from the path
let eventSlug;
// For thumbnails, we need to parse the filename to get the event info
if (req.path.startsWith('/thumb_')) {
// For now, we'll rely on JWT token for thumbnail access
eventSlug = null;
} else {
// For regular photos, the slug is the first part of the path
eventSlug = req.path.split('/')[1];
}
// First check for JWT token (from gallery access)
const tokenFromRequest = getGalleryTokenFromRequest(req, eventSlug);
if (tokenFromRequest) {
const token = tokenFromRequest;
try {
// Try to verify with issuer first, fallback to no issuer for backward compatibility
let decoded;
try {
decoded = jwt.verify(token, process.env.JWT_SECRET, {
algorithms: ['HS256'],
issuer: 'picpeak-auth'
});
} catch (issuerError) {
// If verification fails with issuer, try without issuer (backward compatibility)
if (issuerError.name === 'JsonWebTokenError' && issuerError.message.includes('jwt issuer invalid')) {
decoded = jwt.verify(token, process.env.JWT_SECRET, { algorithms: ['HS256'] });
} else {
throw issuerError;
}
}
// Check if it's a gallery token
if (decoded.type === 'gallery') {
// For thumbnails, we need to verify the token is for a valid event
if (!eventSlug) {
// Resolve the token's event (by id, or legacy slug fallback)...
let event = null;
if (decoded.eventId) {
event = await db('events')
.where({ id: decoded.eventId, is_active: formatBoolean(true) })
.first();
}
if (!event && decoded.eventSlug) {
event = await db('events')
.where({ slug: decoded.eventSlug, is_active: formatBoolean(true) })
.first();
}
// ...then confirm the REQUESTED thumbnail actually belongs to
// that event. Thumbnails are stored flat (thumbnails/thumb_<name>)
// with deterministic, enumerable filenames derived from the
// public event name + a sequential counter. Without this
// ownership check any holder of a gallery token for any event
// could enumerate and fetch another (password-protected) event's
// entire thumbnail set, defeating the gallery password. A
// traversal or foreign filename simply fails to match → denied.
if (event) {
const requestedKey = `thumbnails${req.path}`;
const ownsThumbnail = await db('photos')
.where({ event_id: event.id, thumbnail_path: requestedKey })
.first();
if (ownsThumbnail) {
req.event = event;
return next();
}
}
}
// For regular photos, check if token matches the event
else if (decoded.eventSlug === eventSlug) {
const event = await db('events')
.where({ slug: eventSlug, is_active: formatBoolean(true) })
.first();
if (event) {
req.event = event;
return next();
}
}
}
// Check if it's an admin token (admins can view all photos)
if (decoded.type === 'admin') {
// Enforce the same revocation / session-cutoff invalidation that
// adminAuth does — otherwise a validly-signed admin JWT keeps
// serving photos after logout, password change, or explicit
// revocation (GHSA-x55x).
if (await isTokenRevoked(decoded)) {
return res.status(401).json({ error: 'Session expired' });
}
// adminAuth also (a) rejects tokens for a now-deactivated admin and
// (b) rejects any token minted before the admin's last password
// change. Token revocation alone doesn't cover those, so without
// these two checks a stale or pre-password-change admin token still
// fetches every photo.
const admin = await db('admin_users')
.where({ id: decoded.id, is_active: formatBoolean(true) })
.select('id', 'password_changed_at')
.first();
if (!admin) {
return res.status(401).json({ error: 'Session expired' });
}
if (admin.password_changed_at) {
const passwordChangedSeconds = Math.floor(
new Date(admin.password_changed_at).getTime() / 1000
);
if (decoded.iat < passwordChangedSeconds) {
return res.status(401).json({ error: 'Session expired' });
}
}
return next();
}
} catch (err) {
// Token invalid, fall through to password check
logger.warn('JWT verification failed in photoAuth', { error: err.message });
}
}
// Check for password header (legacy support)
const password = req.headers['x-gallery-password'];
// If no eventSlug (thumbnails), and we don't have valid auth yet, deny access
if (!eventSlug && !password && !tokenFromRequest) {
return res.status(401).json({ error: 'Authentication required for thumbnails' });
}
const event = await db('events').where({ slug: eventSlug, is_active: formatBoolean(true) }).first();
if (!event) {
return res.status(404).json({ error: 'Gallery not found' });
}
const requiresPassword = !(event.require_password === false || event.require_password === 0 || event.require_password === '0');
if (!requiresPassword) {
req.event = event;
return next();
}
if (!password && !tokenFromRequest) {
return res.status(401).json({ error: 'Authentication required' });
}
if (password) {
const validPassword = await bcrypt.compare(password, event.password_hash);
if (!validPassword) {
await db('access_logs').insert({
event_id: event.id,
ip_address: req.ip,
user_agent: req.headers['user-agent'],
action: 'login_fail'
});
return res.status(401).json({ error: 'Invalid password' });
}
} else {
// No valid authentication
return res.status(401).json({ error: 'Invalid authentication' });
}
req.event = event;
next();
} catch (error) {
logger.error('Photo auth error', { error: error.message, stack: error.stack });
res.status(500).json({ error: 'Authentication error' });
}
}
module.exports = photoAuth;
+2 -1
View File
@@ -7,6 +7,7 @@
const express = require('express');
const { body, validationResult } = require('express-validator');
const { safeValidationErrors } = require('../utils/routeHelpers');
const { db, logActivity } = require('../database/db');
const { adminAuth } = require('./../middleware/auth');
const { requirePermission } = require('./../middleware/permissions');
@@ -65,7 +66,7 @@ router.post(
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { name, scopes, expires_at } = req.body;
const { plaintext, hashed, preview } = generateApiToken();
+215 -38
View File
@@ -10,6 +10,7 @@ const archiver = require('archiver');
const StreamZip = require('node-stream-zip');
const { requireEventOwnership } = require('../middleware/ownership');
const { assertZipEntriesWithin } = require('../utils/safePath');
const { sanitizeForZipEntry } = require('../utils/filenameSanitizer');
const logger = require('../utils/logger');
const { getPagination } = require('../utils/routeHelpers');
const router = express.Router();
@@ -204,15 +205,111 @@ router.post('/:id/restore', adminAuth, requirePermission('archives.restore'), re
// manifest the archive process writes. Older archives have no manifest;
// we fall back to filename for those.
const manifestByFilename = new Map();
// Aliases that more than one manifest row claims — see the loop below.
const ambiguousAliases = new Set();
try {
const manifestRaw = await fs.readFile(
path.join(eventDir, 'photos_manifest.json'), 'utf8',
);
const parsed = JSON.parse(manifestRaw);
if (Array.isArray(parsed)) {
for (const m of parsed) {
if (m && m.filename) manifestByFilename.set(m.filename, m);
// Two passes, and the order is the point. Canonical photos.filename
// keys are claimed first and never yielded afterwards; aliases only
// fill names no canonical row wanted. Interleaving them made the
// result depend on manifest iteration order — the query has no
// ORDER BY — and could delete a canonical key because some OTHER
// row's original_filename happened to collide with it.
const rows = parsed.filter((m) => m && m.filename);
// photos.filename is not unique within an event: s3AutoImporter
// takes path.basename(entry.key) and dedupes by path, so two
// imported files in different subfolders both land as `IMG_1234.jpg`
// with different `path` values. At restore both ZIP entries reduce
// to the same basename, so whichever row won the key would hand the
// other photo someone else's category. Contested names are dropped
// rather than guessed.
const contestedFilenames = new Set();
for (const m of rows) {
const held = manifestByFilename.get(m.filename);
if (held && held !== m) {
contestedFilenames.add(m.filename);
continue;
}
manifestByFilename.set(m.filename, m);
}
for (const name of contestedFilenames) manifestByFilename.delete(name);
if (contestedFilenames.size) {
logger.warn(
`Photos manifest: ${contestedFilenames.size} filename(s) claimed by more than one photo; `
+ 'those fall back to the directory for their category.'
);
}
// Every canonical name, contested ones included — an alias must not
// claim a name that a canonical row wanted and lost, either.
const canonicalNames = new Set(rows.map((m) => m.filename));
for (const m of rows) {
// Also index by original_filename. When
// general_use_original_filenames_for_downloads was on at archive
// time, archiveService names each ZIP entry after the ORIGINAL
// filename, while the manifest stays keyed by the internal
// photos.filename — so a lookup by the extracted basename misses
// every entry and the restore silently loses categories on exactly
// those archives. Never overwrite a real filename key: that one is
// authoritative if both happen to collide.
// Index the name as the ZIP would have EMITTED it, not the raw
// column: archiveService runs original names through
// sanitizeForZipEntry() before writing the entry, so an original
// with a slash or a control byte lands under a different name than
// the manifest records. Index both, so either spelling resolves.
//
// Still not total: uniquifyZipNames() appends `_1` when two photos
// in one event share an original name, and that suffix cannot be
// reconstructed from the manifest. Those few fall through to the
// directory, exactly as they did before this fix — no worse, just
// not better. Closing that needs the emitted name recorded at
// archive time, which is a writer change and a new archive format.
for (const alias of [m.original_filename, sanitizeForZipEntry(m.original_filename)]) {
if (!alias) continue;
// An alias colliding with someone else's canonical name is
// genuinely undecidable, so it is dropped rather than resolved
// either way. Which photo the ZIP emitted under that name
// depends on whether original-filename archiving was on at
// archive time, and the manifest does not record that: with it
// ON the entry is the ALIAS owner's file, with it OFF it is the
// canonical owner's. Preferring either one silently mislabels
// the other half of the time.
//
// What the two-pass split buys is that this is now decided the
// same way every run — the archive query has no ORDER BY, so
// interleaving the passes previously made it a coin flip
// between dropping the name and overwriting it.
if (canonicalNames.has(alias)) {
if (manifestByFilename.get(alias) !== m) ambiguousAliases.add(alias);
continue;
}
if (manifestByFilename.has(alias)) {
// Two rows want the same alias — e.g. `individual/IMG.jpg` and
// `collages/IMG.jpg`, which archiveService treats as distinct
// paths and does not suffix, but which collapse to one basename
// here. Whichever won would give the other photo someone else's
// category. Drop the alias so both fall through to the
// directory instead: an unresolved category is recoverable, a
// confidently wrong one is not.
if (manifestByFilename.get(alias) !== m) ambiguousAliases.add(alias);
continue;
}
manifestByFilename.set(alias, m);
}
}
}
for (const alias of ambiguousAliases) manifestByFilename.delete(alias);
if (ambiguousAliases.size) {
logger.warn(
`Photos manifest: ${ambiguousAliases.size} original-filename alias(es) claimed by more than one `
+ 'photo; those fall back to the directory for their category.'
);
}
logger.info(`Loaded photos manifest: ${manifestByFilename.size} entries`);
} catch (e) {
@@ -226,9 +323,84 @@ router.post('/:id/restore', adminAuth, requirePermission('archives.restore'), re
// Get list of extracted files to update database
const extractedPhotos = [];
// First, collect all category information from the ZIP structure
// Category name -> id, resolved once per name for the whole restore.
const categoriesMap = new Map();
// Find-or-create the category by name, among the ones this event can see.
const resolveCategoryId = async (categoryName) => {
if (!categoryName) return null;
if (categoriesMap.has(categoryName)) return categoriesMap.get(categoryName);
// Globals count as existing. A photo filed under the seeded "Ceremony"
// has event_id NULL on its category row, so an event-only lookup misses
// it and creates a second "Ceremony" — and since is_global defaults to
// TRUE, that duplicate then shows up in every other event's category
// list. Same visibility rule the photo routes use: own rows or global.
// Two queries, not one with an OR: an event-scoped category and a
// global one may share a name, and a single .first() would return
// whichever the engine felt like — silently reassigning a photo to the
// global row and losing event-local settings like allow_downloads.
// The event's own row is the more specific answer, so it wins.
//
// The global arm requires event_id IS NULL, not just is_global. The
// bug fixed here left legacy rows behind on upgraded instances —
// event-owned AND is_global true, because the column defaults true —
// and matching on the flag alone would let one event's leftover row be
// adopted by another event's restore, tying photos to a category that
// vanishes with someone else's gallery.
// Two event-scoped categories CAN share a display name when their
// slugs differ, and .first() would then pick one arbitrarily — both
// manifest names collapse onto a single id and half the photos
// inherit the wrong per-category settings (allow_downloads above all).
// Resolving that properly needs a stable category identifier in the
// manifest, which is a writer change and an archive-format bump, and
// could not help any archive already written. So: surface it instead
// of fixing it blind. If this never fires in real logs, the format
// change is not worth making; if it does, this is the evidence for it.
const ownRows = await db('photo_categories')
.where({ event_id: archive.id, name: categoryName })
.select('id');
if (ownRows.length > 1) {
logger.warn(
`Photos manifest: category name "${categoryName}" matches ${ownRows.length} rows in event `
+ `${archive.id}; picking the lowest id. Photos from the other row(s) will inherit its settings.`
);
}
const existingCategory =
// Lowest id, not engine order — an arbitrary-but-stable choice beats
// a nondeterministic one, so a re-run lands the same way.
(ownRows.length
? await db('photo_categories')
.where('id', Math.min(...ownRows.map((r) => r.id)))
.first()
: null)
|| await db('photo_categories')
.where('name', categoryName)
.whereNull('event_id')
.where('is_global', formatBoolean(true))
.first();
if (existingCategory) {
categoriesMap.set(categoryName, existingCategory.id);
} else {
const insertResult = await db('photo_categories').insert({
event_id: archive.id,
name: categoryName,
slug: slugify(categoryName),
// Explicit: the column defaults to true, and a restore inventing a
// GLOBAL category would leak this event's naming into every other
// gallery. Anything created here belongs to this event alone.
is_global: formatBoolean(false),
created_at: new Date()
}).returning('id');
categoriesMap.set(categoryName, insertResult[0]?.id || insertResult[0]);
}
return categoriesMap.get(categoryName);
};
for (const entry of entries) {
if (!entry.isDirectory && entry.name.match(/\.(jpg|jpeg|png|gif|webp)$/i)) {
const filename = path.basename(entry.name);
@@ -239,48 +411,47 @@ router.post('/:id/restore', adminAuth, requirePermission('archives.restore'), re
// Check if file was extracted successfully
const stats = await fs.stat(actualFilePath);
// Determine category from directory structure
let categoryId = null;
if (dirPath && dirPath !== '.') {
// Get the first level directory as category
const categoryName = dirPath.split(path.sep)[0];
if (!categoriesMap.has(categoryName)) {
// Check if this category exists in the database
const existingCategory = await db('photo_categories')
.where('event_id', archive.id)
.where('name', categoryName)
.first();
if (existingCategory) {
categoriesMap.set(categoryName, existingCategory.id);
} else {
// Create the category if it doesn't exist
const insertResult = await db('photo_categories').insert({
event_id: archive.id,
name: categoryName,
slug: slugify(categoryName),
created_at: new Date()
}).returning('id');
const newCategoryId = insertResult[0]?.id || insertResult[0];
categoriesMap.set(categoryName, newCategoryId);
}
}
categoryId = categoriesMap.get(categoryName);
}
const manifestEntry = manifestByFilename.get(filename);
// The manifest is the only faithful source for the category, and
// it is authoritative INCLUDING when it says "none". A manifest
// entry with a null category_name means the photo was genuinely
// uncategorized, so falling through to the directory would
// contradict the very record being restored from.
//
// That matters because the directory is not a category. Archive
// entry names are the storage key minus `events/active/{slug}`,
// and that layout is `individual/{filename}` / `collages/…` —
// categories have never been directories there. Reading the first
// path segment on a real archive therefore invents categories
// literally named "individual" and "collages".
//
// So the fallback is confined to photos with NO manifest entry at
// all: archives written before the manifest existed, where the
// directory is the only signal left and inventing those two names
// is still better than losing every category.
// Check if photo already exists in database
const existingPhoto = await db('photos')
.where('event_id', archive.id)
.where('filename', filename)
.first();
if (!existingPhoto) {
// Resolved HERE, not above: resolveCategoryId find-or-CREATES,
// and archiveEvent retains photo rows. Resolving before this
// check meant restoring an archive whose rows still exist
// created a category from the stale manifest name that nothing
// then used — so renaming a category while its event was
// archived left the old name behind as an empty duplicate.
let categoryId = null;
if (manifestEntry) {
categoryId = await resolveCategoryId(manifestEntry.category_name);
} else if (dirPath && dirPath !== '.') {
categoryId = await resolveCategoryId(dirPath.split(path.sep)[0]);
}
// Store relative path from storage root
const relativePath = path.relative(storagePath, actualFilePath);
const manifestEntry = manifestByFilename.get(filename);
extractedPhotos.push({
event_id: archive.id,
filename: filename,
@@ -293,7 +464,13 @@ router.post('/:id/restore', adminAuth, requirePermission('archives.restore'), re
type: path.extname(filename).substring(1).toLowerCase(),
size_bytes: stats.size,
category_id: categoryId,
uploaded_at: new Date()
// .toISOString(), not a Date: inside jest the sqlite3 binding's
// type dispatch misses sandbox-created Dates and stores the
// literal string "[object Object]", so every restored photo
// gets a garbage timestamp that any test reading it would
// believe. Production stores Dates as ms-numbers and is
// unaffected — which is exactly why this survives unnoticed.
uploaded_at: new Date().toISOString()
});
}
} catch (statError) {
+5
View File
@@ -240,6 +240,11 @@ router.post('/picpeak/import', adminAuth, requirePermission('backup.restore'), p
tables: result.tables,
filesRestored: result.filesRestored,
usesExternalMedia: result.usesExternalMedia,
// False when the pre-#1163 external-path conversion failed. The rows and
// files are in place, but no external original resolves until it is
// retried — the UI must say so rather than showing a plain success.
externalPathsConverted: result.externalPathsConverted !== false,
externalPathError: result.externalPathError || null,
crossEngine: result.crossEngine,
sessionInvalidated: true,
});
+5 -12
View File
@@ -22,6 +22,7 @@ const { adminAuth } = require('../middleware/auth');
const { requirePermission } = require('../middleware/permissions');
const { handleAsync, validateRequest, successResponse } = require('../utils/routeHelpers');
const { getStoragePath } = require('../config/storage');
const { uploadedPdfLogoPath } = require('../utils/safePath');
const businessProfileService = require('../services/businessProfileService');
const { db } = require('../database/db');
const { validateIban } = require('../utils/iban');
@@ -358,12 +359,8 @@ router.post(
// a path managed by a different system.
try {
const previous = await db('business_profile').where({ id: 1 }).first();
const prev = previous?.logo_path;
if (prev && typeof prev === 'string' && /pdf-logo-\d+\./.test(prev)) {
const stripped = prev.replace(/^\/+/, '');
const prevDisk = path.isAbsolute(prev)
? prev
: path.join(getStoragePath(), stripped);
const prevDisk = uploadedPdfLogoPath(previous?.logo_path, getStoragePath());
if (prevDisk) {
try { await fs.unlink(prevDisk); } catch (_) { /* ignore */ }
}
} catch (_) { /* ignore */ }
@@ -383,12 +380,8 @@ router.delete(
requirePermission('settings.edit'),
handleAsync(async (req, res) => {
const existing = await db('business_profile').where({ id: 1 }).first();
const prev = existing?.logo_path;
if (prev && typeof prev === 'string' && /pdf-logo-\d+\./.test(prev)) {
const stripped = prev.replace(/^\/+/, '');
const prevDisk = path.isAbsolute(prev)
? prev
: path.join(getStoragePath(), stripped);
const prevDisk = uploadedPdfLogoPath(existing?.logo_path, getStoragePath());
if (prevDisk) {
try { await fs.unlink(prevDisk); } catch (_) { /* ignore */ }
}
await businessProfileService.updateProfile(
+2 -1
View File
@@ -3,6 +3,7 @@ const path = require('path');
const fs = require('fs').promises;
const multer = require('multer');
const { body, validationResult } = require('express-validator');
const { safeValidationErrors } = require('../utils/routeHelpers');
const { db, logActivity } = require('../database/db');
const { adminAuth } = require('../middleware/auth');
const { requirePermission } = require('../middleware/permissions');
@@ -80,7 +81,7 @@ router.put('/pages/:slug', adminAuth, requirePermission('cms.edit'), [
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { slug } = req.params;
+4 -3
View File
@@ -1,5 +1,6 @@
const express = require('express');
const { body, validationResult } = require('express-validator');
const { safeValidationErrors } = require('../utils/routeHelpers');
const { db, logActivity } = require('../database/db');
const { formatBoolean } = require('../utils/dbCompat');
const { adminAuth } = require('../middleware/auth');
@@ -51,7 +52,7 @@ router.post('/', adminAuth, requirePermission('settings.edit'), [
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { name, slug, is_global = true, event_id = null } = req.body;
@@ -119,7 +120,7 @@ router.put('/:id', adminAuth, requirePermission('settings.edit'), [
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { id } = req.params;
@@ -191,7 +192,7 @@ router.put('/:id/hero', adminAuth, requirePermission('settings.edit'), [
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { id } = req.params;
+4 -3
View File
@@ -6,6 +6,7 @@
const express = require('express');
const router = express.Router();
const { body, param, validationResult } = require('express-validator');
const { safeValidationErrors } = require('../utils/routeHelpers');
const { db, withRetry } = require('../database/db');
const { adminAuth } = require('../middleware/auth');
const { requirePermission } = require('../middleware/permissions');
@@ -58,7 +59,7 @@ router.get('/:slotNumber', adminAuth, requirePermission('branding.view'), [
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { slotNumber } = req.params;
@@ -92,7 +93,7 @@ router.put('/:slotNumber', adminAuth, requirePermission('branding.edit'), [
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { slotNumber } = req.params;
@@ -166,7 +167,7 @@ router.post('/:slotNumber/reset', adminAuth, requirePermission('branding.edit'),
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
await withRetry(() =>
+38 -3
View File
@@ -7,6 +7,7 @@ const { formatBoolean } = require('../utils/dbCompat');
const { resolveAdapter } = require('../services/trackers');
const logger = require('../utils/logger');
const { errorResponse, getPagination } = require('../utils/routeHelpers');
const { measureLocalStorageUsage } = require('../services/localStorageUsage');
const router = express.Router();
/**
@@ -88,11 +89,33 @@ router.get('/stats', adminAuth, requirePermission('analytics.view'), async (req,
.count('id as count')
.first();
// Get storage usage (sum of all photo sizes)
const storageUsed = await applyEventScope(db('photos'), req.admin, 'event_id')
// The catalogued size of the ORIGINALS. Kept, and still worth showing —
// it answers "how much photography is in here" — but it is emphatically
// NOT storage used, which is what it was labelled for years (#1164).
const catalogedBytes = await applyEventScope(db('photos'), req.admin, 'event_id')
.sum('size_bytes as total')
.first();
// Storage used: what is actually on this machine. In reference mode the
// originals above live on a NAS and contribute nothing here; conversely
// this counts what the sum never did — thumbnails, previews, hero
// renditions, watermarks and the per-event download cache.
//
// Deliberately NOT event-scoped, unlike everything else on this endpoint:
// it is a disk measurement, and disk is not divisible by which admin owns
// which event.
//
// Skipped entirely on an S3 backend: the objects are in the bucket and a
// walk of STORAGE_PATH would report near-zero, which is worse than the
// catalogued figure those installs had before #1164.
const usesLocalBackend = (process.env.STORAGE_BACKEND || 'local').toLowerCase() !== 's3';
let localStorage = null;
try {
if (usesLocalBackend) localStorage = await measureLocalStorageUsage();
} catch (err) {
logger.warn(`Dashboard storage measurement failed: ${err.message}`);
}
// Get total views (last 30 days)
const thirtyDaysAgo = new Date();
thirtyDaysAgo.setDate(thirtyDaysAgo.getDate() - 30);
@@ -154,7 +177,19 @@ router.get('/stats', adminAuth, requirePermission('analytics.view'), async (req,
activeEvents: activeEvents.count || 0,
expiringEvents: expiringEvents.count || 0,
totalPhotos: totalPhotos.count || 0,
storageUsed: storageUsed.total || 0,
// Real bytes on this disk. Null when the measurement failed or was
// skipped, which the UI shows as "unavailable" rather than substituting
// a number that means something else.
storageUsed: localStorage ? localStorage.total : null,
// Three states, not two: 'catalog' means the backend is S3 and the
// objects are in the bucket, which is a fact about the install;
// 'unavailable' means the walk failed, which is a fault. Collapsing them
// made a failed local measurement claim the objects live in S3.
storageMeasurement: localStorage ? 'disk' : (usesLocalBackend ? 'unavailable' : 'catalog'),
storageBreakdown: localStorage ? localStorage.breakdown : null,
storagePartial: localStorage ? localStorage.partial : false,
// Catalogued original bytes — what `storageUsed` used to report (#1164).
catalogedBytes: Number(catalogedBytes.total) || 0,
totalViews: totalViews.count || 0,
totalDownloads: totalDownloads.count || 0,
viewsTrend: Math.round(viewsTrend * 10) / 10,
+4 -4
View File
@@ -9,7 +9,7 @@ const { requirePermission } = require('../middleware/permissions');
const { requireFeatureFlag } = require('../middleware/requireFeatureFlag');
const messagingGate = requireFeatureFlag('messaging');
const { wrapEmailHtml, processEmailQueue } = require('../services/emailProcessor');
const { errorResponse } = require('../utils/routeHelpers');
const { errorResponse, safeValidationErrors } = require('../utils/routeHelpers');
const logger = require('../utils/logger');
const router = express.Router();
@@ -52,7 +52,7 @@ router.post('/config', [
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const {
@@ -152,7 +152,7 @@ router.post('/incoming-config', [
], async (req, res) => {
try {
const errors = validationResult(req);
if (!errors.isEmpty()) return res.status(400).json({ errors: errors.array() });
if (!errors.isEmpty()) return res.status(400).json({ errors: safeValidationErrors(errors) });
const { imap_host, imap_port, imap_secure, imap_user, imap_pass, imap_folder } = req.body;
const { isHostAllowed } = require('../utils/networkValidation');
if (!(await isHostAllowed(imap_host))) {
@@ -635,7 +635,7 @@ router.get('/queue', adminAuth, requirePermission('email.view'), [
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const page = req.query.page ? parseInt(req.query.page, 10) : 1;
+3 -2
View File
@@ -5,6 +5,7 @@
const express = require('express');
const { body, validationResult } = require('express-validator');
const { safeValidationErrors } = require('../utils/routeHelpers');
const { adminAuth } = require('../middleware/auth');
const { requirePermission } = require('../middleware/permissions');
const { requireEventOwnership } = require('../middleware/ownership');
@@ -29,7 +30,7 @@ router.post('/:eventId/rename', adminAuth, requirePermission('events.edit'), req
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ success: false, errors: errors.array() });
return res.status(400).json({ success: false, errors: safeValidationErrors(errors) });
}
const { eventId } = req.params;
@@ -70,7 +71,7 @@ router.post('/:eventId/validate-rename', adminAuth, requirePermission('events.ed
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ valid: false, errors: errors.array() });
return res.status(400).json({ valid: false, errors: safeValidationErrors(errors) });
}
const { eventId } = req.params;
+6 -5
View File
@@ -7,6 +7,7 @@
const express = require('express');
const { body, param, validationResult } = require('express-validator');
const { safeValidationErrors } = require('../utils/routeHelpers');
const { logActivity } = require('../database/db');
const { adminAuth } = require('../middleware/auth');
const { requirePermission } = require('../middleware/permissions');
@@ -57,7 +58,7 @@ router.get('/:id', adminAuth, requirePermission('settings.view'), [
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { id } = req.params;
@@ -94,7 +95,7 @@ router.post('/', adminAuth, requirePermission('settings.edit'), [
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const {
@@ -156,7 +157,7 @@ router.put('/:id', adminAuth, requirePermission('settings.edit'), [
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { id } = req.params;
@@ -196,7 +197,7 @@ router.delete('/:id', adminAuth, requirePermission('settings.edit'), [
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { id } = req.params;
@@ -235,7 +236,7 @@ router.post('/reorder', adminAuth, requirePermission('settings.edit'), [
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { orderedIds } = req.body;
@@ -9,7 +9,7 @@ const { adminAuth } = require('../../middleware/auth');
const { requirePermission } = require('../../middleware/permissions');
const { archiveEvent } = require('../../services/archiveService');
const logger = require('../../utils/logger');
const { errorResponse } = require('../../utils/routeHelpers');
const { errorResponse, safeValidationErrors } = require('../../utils/routeHelpers');
const { requireEventOwnership, filterOwnedEventIds } = require('../../middleware/ownership');
const { deleteEventCascade } = require('./helpers');
@@ -70,7 +70,7 @@ module.exports = (router) => {
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { eventIds } = req.body;
@@ -160,7 +160,7 @@ module.exports = (router) => {
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { eventIds } = req.body;
+32 -6
View File
@@ -17,7 +17,7 @@ const { escapeLikePattern } = require('../../utils/sqlSecurity');
const { validatePasswordInContext, getBcryptRounds } = require('../../utils/passwordValidation');
const logger = require('../../utils/logger');
const { sanitizeForLog, sanitizeValidationErrors } = require('../../utils/sanitizeForLog');
const { errorResponse } = require('../../utils/routeHelpers');
const { errorResponse, safeValidationErrors } = require('../../utils/routeHelpers');
const { buildShareLinkVariants } = require('../../services/shareLinkService');
const { parseBooleanInput } = require('../../utils/parsers');
const eventTypeService = require('../../services/eventTypeService');
@@ -30,6 +30,29 @@ const { getFrontendBaseUrl } = require('../../utils/frontendUrl');
const downloadZipService = require('../../services/downloadZipService');
const { validateHeroImageAnchor, getEventFieldRequirements, readBooleanSetting, getDownloadProtectionDefaults, getBrandingDefaults, getCustomerNameFromPayload, getCustomerEmailFromPayload, getCustomerPhoneFromPayload, isPhoneFieldEnabled, mapEventForApi, hasCustomerContactColumns, deleteEventCascade, SLIDESHOW_TRANSITIONS, SLIDESHOW_COLORFILTERS } = require('./helpers');
/**
* Validate a gallery password the admin re-typed, against the SAME policy
* event creation applies.
*
* The publish dialog (#627) re-hashes `password_hash` from a plaintext the
* admin types again, and validated it with nothing but express-validator's
* `isLength({ min: 6 })`. So the configured complexity — moderate by default
* — governed creation and reset while this door accepted `aaaaaa` and made it
* the live gallery password.
*
* Returns null when the password passes; otherwise the response body to send.
*/
async function checkGalleryPasswordPolicy(password, eventName) {
const result = await validatePasswordInContext(password, 'gallery', { eventName });
if (result.valid) return null;
return {
error: 'Password does not meet security requirements',
details: result.errors,
score: result.score,
feedback: result.feedback,
};
}
module.exports = (router) => {
@@ -134,7 +157,7 @@ module.exports = (router) => {
// errors.array() embeds the SUBMITTED value per field — including a
// rejected plaintext password (GHSA-r794).
logger.error('Validation errors:', sanitizeValidationErrors(errors.array()));
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
// Get field requirements from settings
@@ -833,7 +856,7 @@ module.exports = (router) => {
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { id } = req.params;
@@ -854,6 +877,9 @@ module.exports = (router) => {
// Re-hash so the stored hash matches what the email carries — even if
// the admin mistypes vs. what was set at draft creation, the gallery
// password the customer receives is the one that actually works.
const policyError = await checkGalleryPasswordPolicy(password, event.event_name);
if (policyError) return res.status(400).json(policyError);
publishUpdates.password_hash = await bcrypt.hash(password, getBcryptRounds());
}
await db('events').where('id', id).update(publishUpdates);
@@ -991,7 +1017,7 @@ module.exports = (router) => {
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { id } = req.params;
@@ -1269,7 +1295,7 @@ module.exports = (router) => {
if (!errors.isEmpty()) {
// Redact credentials — an invalid update still logs the whole body (GHSA-pgmp).
logger.debug('Update event validation errors', { errors: sanitizeValidationErrors(errors.array()), body: sanitizeForLog(req.body) });
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { id } = req.params;
@@ -1669,7 +1695,7 @@ module.exports = (router) => {
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { id } = req.params;
+162
View File
@@ -224,6 +224,117 @@ async function deleteEventCascade(eventId, adminContext) {
throw err;
}
// Collect this event's storage keys BEFORE the transaction removes the
// photo rows. Afterwards nothing records which objects belonged to this
// event — the DB was the only place that knew, and on an S3/R2 backend the
// objects are still sitting in the bucket, unreferenced and billable.
//
// The filesystem cleanup below (#608) only ever touched local disk: in S3
// mode those paths don't exist, `fs.rm` succeeds against nothing, and the
// real objects are never touched. Measured on a 403-photo event: bucket
// object count unchanged, 679 referenced rows gone.
//
// A Set because a photo can carry the same key in two columns (an unresized
// gallery's hero and preview can resolve to one object) and deleting it
// twice would log a spurious failure for the second attempt.
const storageKeys = new Set();
// Derived keys separately: unlike the originals, whose keys embed the event
// slug, these are not event-scoped and need a shared-ownership check below.
const derivedKeys = new Set();
try {
const { resolvePhotoStorageKey } = require('../../services/photoResolver');
const photos = await db('photos')
.where('event_id', eventId)
.select('id', 'path', 'thumbnail_path', 'hero_path', 'preview_path', 'watermark_path', 'source_origin');
for (const photo of photos) {
try {
// Returns null for reference/external photos, which live on a mount
// outside the managed backend and must NOT be deleted — PicPeak does
// not own those bytes.
const originalKey = resolvePhotoStorageKey(event, photo);
if (originalKey) storageKeys.add(originalKey);
} catch (keyErr) {
logger.warn('Could not resolve storage key during cascade delete', {
eventId, photoId: photo.id, error: keyErr.message
});
}
// Derived tiers are stored as canonical keys and pass through verbatim.
// watermark_path included: it is storage-backed on the single-photo
// path (watermarkService.deleteWatermarkFile) and leaked here the same
// way the originals did.
for (const derived of [photo.thumbnail_path, photo.hero_path, photo.preview_path, photo.watermark_path]) {
if (derived) {
storageKeys.add(derived);
derivedKeys.add(derived);
}
}
}
} catch (collectErr) {
logger.warn('Could not enumerate stored objects before cascade delete', {
eventId, error: collectErr.message
});
}
// A canonical derivative can belong to more than one gallery. Its basename
// comes from the photo's filename — imageProcessor passes no outputBasename
// for managed photos, so the key is `thumbnails/thumb_w300_<filename>` with
// nothing event-scoped in it — and filenames are not unique across events.
// The responsive-tier code says exactly that, which is why THOSE keys carry
// a p{id}_ prefix; the canonical ones predate it. Deleting a shared key here
// would blank a surviving gallery's tile until something regenerated it, so
// anything another event still points at is left alone. Originals need no
// such check: their keys embed the slug.
const derived = Array.from(derivedKeys);
try {
// Chunked: SQLite caps bind variables at 999 and this is four columns wide.
for (let i = 0; i < derived.length; i += 200) {
const chunk = derived.slice(i, i + 200);
const shared = await db('photos')
.whereNot('event_id', eventId)
.where((qb) => qb
.whereIn('thumbnail_path', chunk)
.orWhereIn('hero_path', chunk)
.orWhereIn('preview_path', chunk)
.orWhereIn('watermark_path', chunk))
.select('thumbnail_path', 'hero_path', 'preview_path', 'watermark_path');
for (const row of shared) {
for (const key of [row.thumbnail_path, row.hero_path, row.preview_path, row.watermark_path]) {
if (key && derivedKeys.has(key)) storageKeys.delete(key);
}
}
}
} catch (sharedErr) {
// Can't prove ownership — keep the objects. An orphan costs storage; a
// deleted derivative costs someone else's gallery.
logger.warn('Could not check for shared derivatives; leaving them in place', {
eventId, error: sharedErr.message
});
for (const key of derivedKeys) storageKeys.delete(key);
}
// The archive zip is typically the largest single object an event owns, and
// archiveService writes it through the backend (`storage.putFromFile`) — so
// the `fs.unlink` below is a no-op on S3 and the zip outlives its event.
if (event.archive_path) storageKeys.add(event.archive_path);
// The pre-built "Download All" zip is the subtle one: it lives UNDER
// events/active/{slug}/.download-cache/ (downloadZipService.js:42), so the
// recursive fs.rm below covers it on local disk and nothing covers it on
// S3, where that prefix is not a directory. It is gallery-sized.
// downloadZipService exposes a cleanup() documented as "used on event
// deletion" that this cascade never called.
// NOTE: an in-flight "Download All" build that started before this delete
// can still upload its zip after the sweep and write the path onto a row
// that no longer exists, orphaning it. downloadZipService.cleanup() is the
// service's cancel primitive, but calling it here made the backend CI job
// exceed its 10-minute budget on this branch — its _cleanup() reaches
// getStorage() and, in a suite where the S3 backend is configured but
// unreachable, every cascade delete then pays the adapter's retry backoff.
// Left as a follow-up rather than shipped as a timeout: the race is narrow
// and costs one orphaned object, the regression cost the whole suite.
if (event.download_zip_path) storageKeys.add(event.download_zip_path);
await db.transaction(async (trx) => {
// 1. Delete activity logs (audit trail)
await trx('activity_logs').where('event_id', eventId).del();
@@ -284,6 +395,57 @@ async function deleteEventCascade(eventId, adminContext) {
}
});
// Managed objects, deleted AFTER the commit: a rolled-back transaction must
// never leave files destroyed for an event that still exists. Failures are
// logged rather than thrown, matching the philosophy of the filesystem
// cleanup above — the database is the source of truth, an orphaned object
// is recoverable noise, a half-deleted event is not.
if (storageKeys.size > 0) {
const { getStorage } = require('../../services/storage');
let removed = 0;
try {
const storage = getStorage();
const keys = Array.from(storageKeys);
// Bounded concurrency rather than one await per key. A 400-photo gallery
// owns well over a thousand objects once the derived tiers are counted,
// and on S3 that many sequential DeleteObject round trips runs to
// minutes — long enough for a proxy to time the request out AFTER the
// commit, leaving the event deleted and the sweep half-finished.
// Deleting is idempotent and order-independent, so there is nothing to
// serialise for.
//
// A pool, not Promise.all over every key: an unbounded fan-out would
// open one socket per object and exhaust the S3 client's connection
// pool.
const CONCURRENCY = 16;
let cursor = 0;
const worker = async () => {
while (cursor < keys.length) {
const key = keys[cursor++];
try {
await storage.delete(key);
removed++;
} catch (delErr) {
logger.warn('Failed to delete stored object during cascade delete', {
eventId, key, error: delErr.message
});
}
}
};
await Promise.all(
Array.from({ length: Math.min(CONCURRENCY, keys.length) }, worker)
);
} catch (storageErr) {
logger.warn('Storage backend unavailable during cascade delete', {
eventId, error: storageErr.message
});
}
logger.info('Cascade delete removed stored objects', {
eventId, removed, total: storageKeys.size
});
}
// Audit trail (outside the transaction so a logging failure can't undo
// the actual delete).
await logActivity('event_deleted',
+2 -2
View File
@@ -8,7 +8,7 @@ const { formatBoolean } = require('../../utils/dbCompat');
const { adminAuth } = require('../../middleware/auth');
const { requirePermission } = require('../../middleware/permissions');
const crypto = require('crypto');
const { errorResponse } = require('../../utils/routeHelpers');
const { errorResponse, safeValidationErrors } = require('../../utils/routeHelpers');
const { parseBooleanInput } = require('../../utils/parsers');
const { requireEventOwnership } = require('../../middleware/ownership');
const { requireFeatureFlag } = require('../../middleware/requireFeatureFlag');
@@ -110,7 +110,7 @@ module.exports = (router) => {
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ error: 'Invalid slideshow settings', details: errors.array() });
return res.status(400).json({ error: 'Invalid slideshow settings', details: safeValidationErrors(errors) });
}
const event = await loadOwnedEvent(req);
+107 -22
View File
@@ -8,10 +8,25 @@ const { list, resolveExternalPath, getExternalMediaRoot } = require('../services
const { db, logActivity } = require('../database/db');
const sharp = require('sharp');
const logger = require('../utils/logger');
const { generateThumbnail } = require('../services/imageProcessor');
const { generateThumbnail, extractCaptureDate } = require('../services/imageProcessor');
const { isUniqueViolation } = require('../utils/dbErrors');
const router = express.Router();
// Events with an import running in THIS process (#1162).
//
// The second line of defence, not the first: migration 176 puts a unique index
// on (event_id, external_relpath), and that is what actually makes a duplicate
// impossible — it holds across replicas, across restarts, and against anything
// that inserts external rows without going through this route.
//
// This set exists for the reason the duplicates got filed in the first place:
// a large tree takes long enough that the run LOOKS hung, so admins click
// again. Letting that second run walk the whole tree only to have every insert
// bounce off the index wastes minutes of CPU and reports a nonsense
// `skipped: 6012` back. Failing it immediately with 409 says what happened.
const importsInFlight = new Set();
// GET /api/admin/external-media/list?path=relative/dir
router.get('/list', adminAuth, requirePermission('photos.view'), async (req, res) => {
try {
@@ -50,8 +65,14 @@ async function walkDir(dir, baseDir) {
// POST /api/admin/events/:id/import-external
// Body: { external_path: string, recursive?: boolean, map?: { individual?: string, collages?: string } }
router.post('/events/:id/import-external', adminAuth, requirePermission('photos.upload'), requireEventOwnership, async (req, res) => {
const eventId = parseInt(req.params.id);
if (importsInFlight.has(eventId)) {
return res.status(409).json({
error: 'An import is already running for this event. Wait for it to finish before starting another.'
});
}
importsInFlight.add(eventId);
try {
const eventId = parseInt(req.params.id);
const { external_path, recursive = true, map = { individual: 'individual', collages: 'collages' } } = req.body || {};
if (!external_path) return res.status(400).json({ error: 'external_path is required' });
@@ -60,6 +81,14 @@ router.post('/events/:id/import-external', adminAuth, requirePermission('photos.
if (!event) return res.status(404).json({ error: 'Event not found' });
const baseAbs = resolveExternalPath({ external_path }, '');
// What gets STORED on the row (#1163). `f.rel` stays relative to the
// imported folder because the type inference below reads its first segment
// ('individual' / 'collages'); external_relpath is written relative to
// EXTERNAL_MEDIA_ROOT so the row does not depend on a column this very
// handler is about to overwrite.
const basePrefix = String(external_path).replace(/^\/+|\/+$/g, '');
const toRootRelative = (rel) => (basePrefix ? path.join(basePrefix, rel) : rel);
// Collect files
const files = recursive ? await walkDir(baseAbs, baseAbs) : (await fs.readdir(baseAbs, { withFileTypes: true }))
.filter(e => e.isFile())
@@ -106,10 +135,18 @@ router.post('/events/:id/import-external', adminAuth, requirePermission('photos.
if (segs[0] === map.collages) type = 'collage';
if (segs[0] === map.individual) type = 'individual';
const relFromRoot = toRootRelative(f.rel);
try {
// Check if already exists (by external_relpath)
// Fast path only. This SELECT settles the common case — a re-import of
// a folder already in the event — without paying for a stat and a
// Sharp metadata read per file. It is NOT the guard: those two calls
// sit between here and the INSERT below, which is exactly the window
// two overlapping imports both walked through (#1162). The unique
// index from migration 176 is the guard, and the catch below is how
// this loop converges when it fires.
const exists = await db('photos')
.where({ event_id: eventId, external_relpath: f.rel })
.where({ event_id: eventId, external_relpath: relFromRoot })
.first();
if (exists) { skipped++; continue; }
const stats = await fs.stat(f.full);
@@ -119,27 +156,65 @@ router.post('/events/:id/import-external', adminAuth, requirePermission('photos.
let height = null;
try {
const metadata = await sharp(f.full).metadata();
width = metadata.width || null;
height = metadata.height || null;
// Oriented, not raw — see imageProcessor.orientedDimensions (#1185).
({ width, height } = require('../services/imageProcessor').orientedDimensions(metadata));
} catch (dimErr) {
logger.warn(`Could not extract dimensions for ${f.rel}: ${dimErr.message}`);
}
const inserted = await db('photos')
.insert({
event_id: eventId,
filename: f.name,
// Keep path as a hint for legacy code but not used for resolution in external mode
path: path.join(event.slug, f.name),
thumbnail_path: null,
type,
size_bytes: stats.size,
width,
height,
source_origin: 'external',
external_relpath: f.rel
})
.returning('id');
// Capture date from EXIF (#1172). Managed uploads get this from
// photoProcessor, which external media never goes through — so
// captured_at stayed NULL for every externally imported photo, and the
// gallery's "Date Taken" sort silently degraded into import order via
// its COALESCE fallback. On a library imported in two batches that put
// the first days of a trip after the last ones.
//
// Read here because the file is already open a few lines above for the
// dimensions, so this costs one more read of the same source rather
// than a second pass over the mount.
//
// Best-effort, exactly like the dimensions: a source without EXIF, or
// one Sharp/exifr cannot parse, imports with captured_at NULL and
// falls back to uploaded_at as before.
let capturedAt = null;
try {
capturedAt = await extractCaptureDate(f.full);
} catch (dateErr) {
logger.warn(`Could not extract capture date for ${f.rel}: ${dateErr.message}`);
}
let inserted;
try {
inserted = await db('photos')
.insert({
event_id: eventId,
filename: f.name,
// Keep path as a hint for legacy code but not used for resolution in external mode
path: path.join(event.slug, f.name),
thumbnail_path: null,
type,
size_bytes: stats.size,
width,
height,
source_origin: 'external',
external_relpath: relFromRoot,
// .toISOString() rather than the Date: inside jest, Dates handed
// to the sqlite3 binding land as the literal string
// "[object Object]" (see CLAUDE.md). Strings round-trip on both
// engines.
captured_at: capturedAt ? capturedAt.toISOString() : null
})
.returning('id');
} catch (insertErr) {
// Another writer inserted this exact path while we were reading
// metadata. That is the outcome the index exists to produce, and it
// is a skip rather than a failure — the row is there, it just isn't
// ours. Counting it as `skipped` keeps the reported totals honest;
// before the index this landed in the outer catch as a nameless
// failure, or (more often) never fired at all and duplicated the row.
if (isUniqueViolation(insertErr)) { skipped++; continue; }
throw insertErr;
}
const photoId = Array.isArray(inserted) && inserted.length
? (typeof inserted[0] === 'object' ? inserted[0].id : inserted[0])
@@ -175,7 +250,12 @@ router.post('/events/:id/import-external', adminAuth, requirePermission('photos.
}
}
// Update event fields
// Safe for existing EXTERNAL rows as of #1163. It was not: relpaths were
// stored relative to external_path, so overwriting the column here rebased
// every row already in the event onto the new folder — quietly, because
// their thumbnails were already on local disk and the grid carried on
// rendering. Rows now carry a root-relative path and this write cannot
// reach them.
await db('events').where('id', eventId).update({ source_mode: 'reference', external_path });
await logActivity(
@@ -193,6 +273,11 @@ router.post('/events/:id/import-external', adminAuth, requirePermission('photos.
error: error.message
});
res.status(500).json({ error: 'Failed to import external media' });
} finally {
// In `finally` and not at the end of `try`: an import that throws must
// still release the event, or a single failure locks out every retry
// until the process restarts.
importsInFlight.delete(eventId);
}
});
+448 -66
View File
@@ -7,35 +7,106 @@ const fs = require('fs').promises;
const logger = require('../utils/logger');
const { resolvePhotoFilePath } = require('../services/photoResolver');
const maintenanceJobs = require('../services/maintenanceJobState');
// Module-level progress state
let repairProgress = {
isRunning: false,
lastResult: null
};
// Run state for both sweeps lives in the database, not in this process
// (#1181). It used to be a module-level object per job, which is correct on a
// single replica and wrong behind a load balancer: the status poll answers
// from whichever process it reaches, so an idle replica reports isRunning
// false while another is mid-run, the UI re-enables the button, and the next
// POST starts a duplicate pass over the entire library.
//
// Two separate rows, for the same reason the two objects were separate: the
// jobs walk the same photos but read different things out of them, and one
// running must not block or report for the other.
const { JOB_DIMENSION_REPAIR, JOB_CAPTURE_DATE_BACKFILL } = maintenanceJobs;
const { HEARTBEAT_INTERVAL_MS } = maintenanceJobs;
/**
* Renew the lease on a timer for as long as the run holds it.
*
* On a timer, not between photos: a single hung read on a stalled NAS mount or
* a slow S3 object can outlast the whole stale window inside one iteration, and
* a renewal that only fires between photos never gets to run. The lease would
* expire while the job was demonstrably alive, another replica would take it
* over, and the two would walk the same rows — precisely the case the lease
* exists to prevent. The timer also covers the candidate query, which on a
* large library is itself slow.
*
* `lost()` reports whether the claim has since been taken over. The loops check
* it between photos and stop: mid-photo interruption is not possible, so the
* worst case is one extra row written by the old runner, and its release is
* fenced on the token anyway.
*/
function startLeaseKeeper(jobName, token) {
let lost = false;
const timer = setInterval(async () => {
try {
if (!(await maintenanceJobs.heartbeat(jobName, token))) {
lost = true;
clearInterval(timer);
}
} catch (err) {
// heartbeat() already swallows query errors and reports the claim as
// held; this is belt-and-braces so an unexpected throw cannot kill the
// timer callback and silently stop all renewals.
logger.warn(`Lease renewal error for ${jobName}: ${err.message}`);
}
}, HEARTBEAT_INTERVAL_MS);
// Do not hold the event loop open on account of a maintenance sweep.
if (typeof timer.unref === 'function') timer.unref();
return {
lost: () => lost,
stop: () => clearInterval(timer),
};
}
// Repair photo dimensions (background job)
router.post('/repair-dimensions', adminAuth, requirePermission('photos.edit'), async (req, res) => {
try {
if (repairProgress.isRunning) {
// Claimed before the candidate query, not after: that query is an await,
// and two requests arriving inside it would both read "not running" and
// both start a pass. The claim is a conditional UPDATE, so it settles the
// race across replicas as well as within one.
const token = await maintenanceJobs.claim(JOB_DIMENSION_REPAIR);
if (!token) {
return res.status(409).json({ error: 'Repair is already running' });
}
// Started at the claim, not at the loop: on a large or loaded install the
// candidate SELECT below (plus the setImmediate hop) can itself outlast the
// stale window, and an unrenewed claim would be taken over before the sweep
// had read its first photo. Handed to the background section, which stops
// it; every early exit here stops it too.
const lease = startLeaseKeeper(JOB_DIMENSION_REPAIR, token);
const photos = await db('photos')
.join('events', 'photos.event_id', 'events.id')
.where(function () {
this.whereNull('photos.width').orWhereNull('photos.height');
})
.where(function () {
this.where('photos.media_type', '!=', 'video').orWhereNull('photos.media_type');
})
.select(
'photos.id', 'photos.path', 'photos.filename',
'photos.source_origin', 'photos.external_relpath', 'photos.event_id',
'events.source_mode', 'events.external_path', 'events.slug'
);
let photos;
try {
photos = await db('photos')
.join('events', 'photos.event_id', 'events.id')
.where(function () {
this.whereNull('photos.width').orWhereNull('photos.height');
})
.where(function () {
this.where('photos.media_type', '!=', 'video').orWhereNull('photos.media_type');
})
.select(
'photos.id', 'photos.path', 'photos.filename',
'photos.source_origin', 'photos.external_relpath', 'photos.event_id',
'events.source_mode', 'events.external_path', 'events.slug'
);
} catch (err) {
lease.stop();
await maintenanceJobs.release(JOB_DIMENSION_REPAIR, token);
throw err;
}
if (photos.length === 0) {
// Released with no result: nothing ran, so the numbers from the last
// real run stay on screen rather than being blanked by a no-op.
lease.stop();
await maintenanceJobs.release(JOB_DIMENSION_REPAIR, token);
return res.json({ message: 'No photos need dimension repair', count: 0 });
}
@@ -46,70 +117,94 @@ router.post('/repair-dimensions', adminAuth, requirePermission('photos.edit'), a
});
// Process in background
repairProgress.isRunning = true;
repairProgress.lastResult = null;
setImmediate(async () => {
let sharp;
try {
sharp = require('sharp');
} catch (err) {
logger.error('Sharp not available for dimension repair:', err.message);
repairProgress.isRunning = false;
repairProgress.lastResult = { success: 0, failed: 0, error: 'Sharp not available' };
lease.stop();
await maintenanceJobs.release(JOB_DIMENSION_REPAIR, token, { success: 0, failed: 0, error: 'Sharp not available' });
return;
}
let successCount = 0;
let errorCount = 0;
let lostClaim = false;
for (const photo of photos) {
try {
const event = { source_mode: photo.source_mode, external_path: photo.external_path, slug: photo.slug };
let fullPath;
try {
fullPath = resolvePhotoFilePath(event, photo);
} catch (err) {
logger.warn(`Photo ${photo.id} has no resolvable path, skipping dimension repair: ${err.message}`);
errorCount++;
continue;
}
// Everything below runs detached from the request, so an unexpected
// throw has nobody to report to. Without this the claim would sit held
// until it aged out of the staleness window, disabling the button for
// that whole time on every replica.
try {
for (const photo of photos) {
// The timer does the renewing; this only notices that it has
// already failed, so the loop stops instead of running on beside the
// replica that took the claim over.
if (lease.lost()) { lostClaim = true; break; }
try {
await fs.access(fullPath);
} catch (err) {
logger.warn(`File not found for photo ${photo.id}: ${fullPath}`);
errorCount++;
continue;
}
const metadata = await sharp(fullPath).metadata();
if (metadata.width && metadata.height) {
await db('photos')
.where({ id: photo.id })
.update({
width: metadata.width,
height: metadata.height
});
successCount++;
if (successCount % 50 === 0) {
logger.info(`Dimension repair progress: ${successCount} updated...`);
const event = { source_mode: photo.source_mode, external_path: photo.external_path, slug: photo.slug };
let fullPath;
try {
fullPath = resolvePhotoFilePath(event, photo);
} catch (err) {
logger.warn(`Photo ${photo.id} has no resolvable path, skipping dimension repair: ${err.message}`);
errorCount++;
continue;
}
} else {
logger.warn(`Could not extract dimensions for photo ${photo.id}`);
try {
await fs.access(fullPath);
} catch (err) {
logger.warn(`File not found for photo ${photo.id}: ${fullPath}`);
errorCount++;
continue;
}
const metadata = await sharp(fullPath).metadata();
// Oriented, not raw — see imageProcessor.orientedDimensions (#1185).
const dims = require('../services/imageProcessor').orientedDimensions(metadata);
if (dims.width && dims.height) {
await db('photos')
.where({ id: photo.id })
.update({
width: dims.width,
height: dims.height
});
successCount++;
if (successCount % 50 === 0) {
logger.info(`Dimension repair progress: ${successCount} updated...`);
}
} else {
logger.warn(`Could not extract dimensions for photo ${photo.id}`);
errorCount++;
}
} catch (error) {
logger.error(`Error repairing dimensions for photo ${photo.id}:`, error);
errorCount++;
}
} catch (error) {
logger.error(`Error repairing dimensions for photo ${photo.id}:`, error);
errorCount++;
}
}
repairProgress.isRunning = false;
repairProgress.lastResult = { success: successCount, failed: errorCount };
logger.info(`Dimension repair complete: ${successCount} success, ${errorCount} errors`);
if (lostClaim) {
// Another replica declared this run stale and took it over. It owns
// the row now, so releasing would clear ITS flag — release() refuses
// on the token, but there is nothing to report either way.
logger.warn(`Dimension repair stopped: claim taken over after ${successCount} updated, ${errorCount} errors`);
return;
}
await maintenanceJobs.release(JOB_DIMENSION_REPAIR, token, { success: successCount, failed: errorCount });
logger.info(`Dimension repair complete: ${successCount} success, ${errorCount} errors`);
} catch (err) {
logger.error('Dimension repair aborted:', err);
await maintenanceJobs
.release(JOB_DIMENSION_REPAIR, token, { success: successCount, failed: errorCount, error: err.message })
.catch(() => {});
} finally {
lease.stop();
}
});
} catch (error) {
logger.error('Error starting dimension repair:', error);
@@ -138,13 +233,15 @@ router.get('/repair-dimensions/status', adminAuth, requirePermission('photos.vie
const total = Number(totalPhotos.count);
const withDims = Number(withDimensions.count);
// Read from the shared row, so this answers the same on every replica.
const state = await maintenanceJobs.read(JOB_DIMENSION_REPAIR);
res.json({
total,
withDimensions: withDims,
withoutDimensions: total - withDims,
isRunning: repairProgress.isRunning,
lastResult: repairProgress.lastResult
isRunning: state.isRunning,
lastResult: state.lastResult
});
} catch (error) {
logger.error('Error fetching dimension repair status:', error);
@@ -152,4 +249,289 @@ router.get('/repair-dimensions/status', adminAuth, requirePermission('photos.vie
}
});
/**
* Backfill captured_at from EXIF (#1172).
*
* External imports never read EXIF before this release, so every
* externally-imported photo carries captured_at NULL — and the gallery's
* "Date Taken" sort falls back to uploaded_at, which on a bulk import is the
* import timestamp. A 5555-photo trip came back ordered by which folder was
* imported first.
*
* Deliberately an endpoint rather than a migration: the originals live on a
* mount that may be unavailable at upgrade time, reading 8000+ of them blocks
* the boot, and a run that found nothing needs to be repeatable once the mount
* is back. Same reasoning, and the same shape, as the dimension repair above —
* including resolvePhotoFilePath, which is what makes it work for external
* rows at all (the thumbnail regenerator resolves under
* storage/events/active/<path>, which never exists for them).
*/
// settings.edit, not photos.edit: this walks every event in the install and
// rewrites their metadata, which is a maintenance action rather than a photo
// edit. photos.edit is held by the `editor` role
// (056_add_role_permissions_table.js:73), which is scoped to contributing
// content, not to running an install-wide S3/NAS scan across other people's
// events. settings.edit is the restrictive one here — `admin` carries only
// settings.view (056:63).
//
// main gates the same endpoint on system.manage, which does not exist on this
// branch: it is one of the permissions settings.edit was later split into, and
// migration 175 there projects every settings.edit holder forward onto it. So
// this is the same gate under its older name, and the two branches let exactly
// the same people through.
router.post('/repair-capture-dates', adminAuth, requirePermission('settings.edit'), async (req, res) => {
try {
// Claimed here, not after the candidate query: that query is an await, and
// two POSTs arriving inside it would both read "not running" and both start
// a pass over the same rows. Being a conditional UPDATE, the claim settles
// that between replicas too. Every early exit below has to release it
// again, hence the try/catch around the query.
const token = await maintenanceJobs.claim(JOB_CAPTURE_DATE_BACKFILL);
if (!token) {
return res.status(409).json({ error: 'Capture date backfill is already running' });
}
// Started at the claim so the candidate SELECT below is covered too — see
// the dimension repair above.
const lease = startLeaseKeeper(JOB_CAPTURE_DATE_BACKFILL, token);
let photos;
try {
photos = await db('photos')
.join('events', 'photos.event_id', 'events.id')
.whereNull('photos.captured_at')
// Three markers, because no single one is reliable. fileWatcher's
// auto-import sets type='video' and a video/* mime but never
// media_type (fileWatcher.js:128-130), so those rows keep the 'image'
// default from migration 048 and a media_type-only filter queues them
// forever: extractCaptureDate returns null for a video, captured_at
// stays null, and every run picks it up again.
.where(function () {
this.where('photos.media_type', '!=', 'video').orWhereNull('photos.media_type');
})
.where(function () {
this.where('photos.type', '!=', 'video').orWhereNull('photos.type');
})
.where(function () {
this.whereNull('photos.mime_type').orWhere('photos.mime_type', 'not like', 'video/%');
})
// Archiving deletes the originals from storage but keeps the photos
// rows (archiveService.js:166,199). Those files are inside the zip and
// nothing here can read them, so including them would fail every row
// on every run and leave the button permanently lit.
.where(function () {
this.where('events.is_archived', false).orWhereNull('events.is_archived');
})
.select(
'photos.id', 'photos.path', 'photos.filename',
'photos.source_origin', 'photos.external_relpath', 'photos.event_id',
'events.source_mode', 'events.external_path', 'events.slug'
);
} catch (err) {
lease.stop();
await maintenanceJobs.release(JOB_CAPTURE_DATE_BACKFILL, token);
throw err;
}
if (photos.length === 0) {
// No result passed: nothing ran, so the last real run's numbers survive.
lease.stop();
await maintenanceJobs.release(JOB_CAPTURE_DATE_BACKFILL, token);
return res.json({ message: 'No photos need a capture date', count: 0 });
}
res.json({
message: `Started backfilling capture dates for ${photos.length} photos`,
count: photos.length
});
setImmediate(async () => {
const { extractCaptureDate, withLocalCopy } = require('../services/imageProcessor');
const { resolvePhotoStorageKey } = require('../services/photoResolver');
let successCount = 0;
let missingCount = 0;
let errorCount = 0;
let skippedCount = 0;
let lostClaim = false;
// Same reasoning as the dimension repair: detached from the request, so
// an unexpected throw must not leave the claim held.
try {
for (const photo of photos) {
// The timer renews; this only notices it has already failed.
if (lease.lost()) { lostClaim = true; break; }
try {
const event = { source_mode: photo.source_mode, external_path: photo.external_path, slug: photo.slug };
const isExternal = photo.source_origin === 'external' || photo.source_origin === 'reference';
// Two source shapes, same split the thumbnail regenerator uses
// (imageProcessor.js:391-420). External rows live on a local mount
// and are read directly; managed rows live behind the storage
// backend, which on an S3 install is not a filesystem at all — going
// through resolvePhotoFilePath there would build a STORAGE_PATH that
// holds nothing and fail every managed photo.
let captured;
if (isExternal) {
let fullPath;
try {
fullPath = resolvePhotoFilePath(event, photo);
} catch (err) {
logger.warn(`Photo ${photo.id} has no resolvable path, skipping capture date: ${err.message}`);
errorCount++;
continue;
}
try {
await fs.access(fullPath);
} catch (err) {
logger.warn(`File not found for photo ${photo.id}: ${fullPath}`);
errorCount++;
continue;
}
captured = await extractCaptureDate(fullPath);
} else {
let sourceKey;
try {
sourceKey = resolvePhotoStorageKey(event, photo);
} catch (err) {
logger.warn(`Photo ${photo.id} has no resolvable storage key, skipping capture date: ${err.message}`);
errorCount++;
continue;
}
// In local-fs mode withLocalCopy hands back the resolved path
// without checking it exists, so the access probe stays. In S3
// mode a missing object throws out of getToFile and lands in the
// outer catch — both end up counted as failures, which is what a
// missing original is.
captured = await withLocalCopy(sourceKey, async (localPath) => {
await fs.access(localPath);
return extractCaptureDate(localPath);
});
}
if (!captured) {
// No date recovered. Usually genuine — plenty of sources carry no
// EXIF — but extractCaptureDate also returns null when the file is
// unreadable as an image, so this bucket is "nothing to write",
// not "definitely has no EXIF". The failure counter above is the
// one that means the storage is broken.
missingCount++;
continue;
}
// whereNull, not a blanket set: the job can run for a long time on a
// large library, and an import or a replacement finishing meanwhile
// has already written a date this pass would otherwise overwrite
// with the same-or-worse value.
//
// Fenced on path and filename as well as the id (#1201):
// replacePhoto — reachable from the replace_by_name upload path
// (adminPhotos.js) — swaps a NEW file under an existing row and
// rewrites path/filename. That replacement carries no date of its
// own, so captured_at is still NULL and whereNull alone would let
// the previous file's EXIF date land on it. Matching the identity
// that was actually read means the update affects no rows and the
// row is simply skipped.
const updated = await db('photos')
.where({ id: photo.id, path: photo.path, filename: photo.filename })
.whereNull('captured_at')
.update({ captured_at: captured.toISOString() });
// Counted, not dropped: without this a candidate that was read but
// not written falls out of the run's arithmetic entirely, and
// success + noExif + failed silently stops adding up to the count
// the operator was shown when they started it. Two ways to land
// here, both "another writer got there first" — the row was dated
// meanwhile (whereNull), or its file changed under us (the fence).
// Neither is an error and neither needs a retry: captured_at is
// still NULL for the fenced case, so the status endpoint keeps
// reporting it as backlog and the next run picks it up.
if (updated) successCount++; else skippedCount++;
if (successCount % 50 === 0 && successCount > 0) {
logger.info(`Capture date backfill progress: ${successCount} updated...`);
}
} catch (error) {
logger.error(`Error backfilling capture date for photo ${photo.id}:`, error);
errorCount++;
}
}
if (lostClaim) {
logger.warn(`Capture date backfill stopped: claim taken over after ${successCount} updated, ${errorCount} errors`);
return;
}
await maintenanceJobs.release(JOB_CAPTURE_DATE_BACKFILL, token, { success: successCount, noExif: missingCount, failed: errorCount, skipped: skippedCount });
logger.info(
`Capture date backfill complete: ${successCount} updated, ${missingCount} without EXIF, `
+ `${errorCount} errors, ${skippedCount} skipped (dated or replaced mid-run)`
);
} catch (err) {
logger.error('Capture date backfill aborted:', err);
await maintenanceJobs
.release(JOB_CAPTURE_DATE_BACKFILL, token, { success: successCount, noExif: missingCount, failed: errorCount, skipped: skippedCount, error: err.message })
.catch(() => {});
} finally {
lease.stop();
}
});
} catch (error) {
logger.error('Error starting capture date backfill:', error);
res.status(500).json({ error: 'Failed to start capture date backfill' });
}
});
// The same permission as the POST, not the read-only settings.view. The built-in
// `admin` role holds settings.view but not settings.edit
// (056_add_role_permissions_table.js:63), and StatusTab has no permission gate
// of its own — a successful status payload is what renders the card and its
// enabled button. Gating on settings.view therefore showed every admin a live
// backfill button whose every click 403s with no error surfaced.
router.get('/repair-capture-dates/status', adminAuth, requirePermission('settings.edit'), async (req, res) => {
try {
// Same scope as the job itself — counting archived photos here would show
// a permanent backlog the button can never clear.
const scoped = () => db('photos')
.join('events', 'photos.event_id', 'events.id')
.where(function () {
this.where('photos.media_type', '!=', 'video').orWhereNull('photos.media_type');
})
.where(function () {
this.where('photos.type', '!=', 'video').orWhereNull('photos.type');
})
.where(function () {
this.whereNull('photos.mime_type').orWhere('photos.mime_type', 'not like', 'video/%');
})
.where(function () {
this.where('events.is_archived', false).orWhereNull('events.is_archived');
});
// One query, two aggregates. As two separate counts an import committing a
// dated photo between them could be counted by the second and not the
// first, so withCaptureDate came out larger than total and the card showed
// a negative backlog — with the button enabled to "fix" it.
const counts = await scoped()
.count('photos.id as total')
.count({ dated: db.raw('CASE WHEN photos.captured_at IS NOT NULL THEN 1 END') })
.first();
const total = Number(counts.total);
const withCaptureDate = Number(counts.dated);
// Read from the shared row, so this answers the same on every replica.
const state = await maintenanceJobs.read(JOB_CAPTURE_DATE_BACKFILL);
res.json({
total,
withCaptureDate,
withoutCaptureDate: total - withCaptureDate,
isRunning: state.isRunning,
lastResult: state.lastResult
});
} catch (error) {
logger.error('Error fetching capture date backfill status:', error);
res.status(500).json({ error: 'Failed to fetch capture date backfill status' });
}
});
module.exports = router;
+3 -3
View File
@@ -11,7 +11,7 @@ const { adminAuth } = require('../middleware/auth');
const { requirePermission } = require('../middleware/permissions');
const { requireEventOwnership } = require('../middleware/ownership');
const { PhotoFilterBuilder } = require('../utils/photoFilterBuilder');
const { getPagination } = require('../utils/routeHelpers');
const { getPagination, safeValidationErrors } = require('../utils/routeHelpers');
const { PhotoExportService } = require('../services/photoExportService');
const logger = require('../utils/logger');
@@ -39,7 +39,7 @@ router.get('/:eventId/filtered', adminAuth, requirePermission('photos.view'), re
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const eventId = parseInt(req.params.eventId);
@@ -166,7 +166,7 @@ router.post('/:eventId/export', adminAuth, requirePermission('photos.download'),
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const eventId = parseInt(req.params.eventId);
+25 -64
View File
@@ -14,7 +14,8 @@ const {
} = require('../services/downloadFilenameService');
const { escapeLikePattern } = require('../utils/sqlSecurity');
const { validateUploadedFiles } = require('../middleware/uploadValidation');
const { getMaxFilesPerUpload, getAllowedMimeTypes } = require('../services/uploadSettings');
const { getMaxFilesPerUpload, getAllowedMimeTypes, EXTENSION_TO_MIME } = require('../services/uploadSettings');
const { resolvePhotoContentType } = require('../utils/photoContentType');
const { processUploadedPhotos } = require('../services/photoProcessor');
const chunkedUpload = require('../services/chunkedUploadService');
const watermarkGeneratorService = require('../services/watermarkGeneratorService');
@@ -958,7 +959,7 @@ router.get('/:eventId/photos/:photoId/download', adminAuth, requirePermission('p
return res.status(404).json({ error: 'Photo file not found' });
}
res.set({
'Content-Type': photo.mime_type || 'application/octet-stream',
'Content-Type': resolvePhotoContentType(photo),
'Content-Length': stat.size,
'Content-Disposition': contentDisposition,
});
@@ -975,7 +976,7 @@ router.get('/:eventId/photos/:photoId/download', adminAuth, requirePermission('p
return res.status(404).json({ error: 'Photo file not found' });
}
res.set({
'Content-Type': photo.mime_type || 'application/octet-stream',
'Content-Type': resolvePhotoContentType(photo),
'Content-Disposition': contentDisposition,
});
res.sendFile(filePath);
@@ -1151,64 +1152,9 @@ router.get('/:eventId/photo/:photoId', adminAuth, requirePermission('photos.view
const event = await db('events').where('id', eventId).first();
const storageKey = resolvePhotoStorageKey(event, photo);
// Content-Type resolution (#908 + external review). Invariant: the
// header is ALWAYS image/* or video/*.
// - photos.mime_type is never echoed verbatim unless it is a video/
// type: the chunked-upload path stores the client-sent MIME
// unvalidated, so a stored text/html served inline under the app
// origin would be a same-origin XSS gift.
// - Images ignore the stored value entirely — migration 039
// backfilled image/jpeg onto every legacy row (PNGs included), so
// the extension is the more trustworthy signal; normalized via the
// shared map (image/jpg → image/jpeg), jpeg fallback when unknown.
// - Videos prefer a stored video/ type, then the extension map
// (.mov → video/quicktime, .webm → video/webm, …), then video/mp4.
// The old ext-derived image/<ext> (image/mp4) is what made the
// admin player's blob unplayable (#908).
const { EXTENSION_TO_MIME } = require('../services/uploadSettings');
const ext = path.extname(photo.filename).slice(1).toLowerCase();
// Own-property lookup (review): a client-controlled filename ending in
// .constructor / .__proto__ / .toString would otherwise return an
// inherited Object.prototype member, and the extMime.startsWith below
// would throw — a permanent 500 for that photo instead of the fallback.
const extMime = Object.prototype.hasOwnProperty.call(EXTENSION_TO_MIME, ext)
? EXTENSION_TO_MIME[ext]
: null;
// Full-token validation, not just a prefix check: the stored value is
// client-controlled, and header-invalid characters (video/mp4\r\nX: y)
// would make setHeader throw — a permanent 500 for that photo. Bare
// 'video/' is equally invalid; both fall back to the extension map.
const storedVideoMime = photo.mime_type && /^video\/[\w.+-]+$/.test(photo.mime_type)
? photo.mime_type
: null;
// Honor a stored image MIME for any header-safe RASTER type (#908
// review): the S3 auto-importer accepts arbitrary image/* from
// mime-types and stores it (avif/bmp/tiff/heic/apng/ico/jxl/…), and a
// hand-listed allowlist kept missing formats. Allow image/<token> but
// NEVER the scriptable svg / *+xml family (image/svg+xml executes
// inline). The strict token + anchors also block header injection
// (image/x\r\nY:). Migration 039's blanket image/jpeg backfill on
// legacy rows is why the mapped extension still wins ahead of this.
const storedImageMime =
photo.mime_type &&
/^image\/[\w.+-]+$/.test(photo.mime_type) &&
!/^image\/svg|xml/i.test(photo.mime_type)
? photo.mime_type
: null;
const isVideo = photo.media_type === 'video' ||
Boolean(storedVideoMime) ||
Boolean(extMime && extMime.startsWith('video/'));
// Never interpolate the raw extension on the image side: it would
// synthesize image/svg+xml (scriptable inline) or header-invalid values
// from client-controlled chunked-upload filenames. Precedence is
// mapped-extension (also corrects the 039 legacy-jpeg backfill on PNGs)
// -> safe stored raster MIME (auto-imported avif/bmp/tiff) -> image/jpeg.
// A stored type outside the allowlist degrades to image/jpeg; browsers
// sniff image bytes in <img>/blob contexts, so a mislabel is harmless
// where an injected type is not.
const contentType = isVideo
? storedVideoMime || (extMime && extMime.startsWith('video/') ? extMime : null) || 'video/mp4'
: (extMime && extMime.startsWith('image/') ? extMime : null) || storedImageMime || 'image/jpeg';
// Content-Type resolution (#908 + external review) lives in
// utils/photoContentType so the gallery routes apply the same rule.
const contentType = resolvePhotoContentType(photo);
res.setHeader('Content-Type', contentType);
res.setHeader('Cache-Control', 'private, max-age=3600');
res.setHeader('Cross-Origin-Resource-Policy', 'cross-origin');
@@ -1324,7 +1270,7 @@ router.get('/:eventId/debug', adminAuth, requirePermission('photos.view'), requi
router.post('/:eventId/chunked-upload/init', adminAuth, requirePermission('photos.upload'), requireEventOwnership, async (req, res) => {
try {
const { eventId } = req.params;
const { filename, fileSize, mimeType, totalChunks } = req.body;
const { filename, fileSize, totalChunks } = req.body;
// Validate event exists
const event = await db('events').where({ id: eventId }).first();
@@ -1333,8 +1279,23 @@ router.post('/:eventId/chunked-upload/init', adminAuth, requirePermission('photo
}
// Validate required fields
if (!filename || !fileSize || !mimeType) {
return res.status(400).json({ error: 'Missing required fields: filename, fileSize, mimeType' });
if (!filename || !fileSize) {
return res.status(400).json({ error: 'Missing required fields: filename, fileSize' });
}
// The client-declared mimeType is not trusted. It used to be stored on
// the photo row verbatim and echoed as Content-Type by the gallery
// routes, so a JPEG/HTML polyglot declared as text/html rendered inline
// on the app origin. The MIME is derived from the extension instead,
// and the extension has to be on the admin's allow-list, which is what
// the multipart path enforces through its multer fileFilter.
const ext = path.extname(String(filename)).slice(1).toLowerCase();
const mimeType = Object.prototype.hasOwnProperty.call(EXTENSION_TO_MIME, ext)
? EXTENSION_TO_MIME[ext]
: null;
const allowedMimeTypes = await getAllowedMimeTypes();
if (!mimeType || !allowedMimeTypes.includes(mimeType)) {
return res.status(400).json({ error: 'File type not allowed' });
}
// Validate file size (max 10GB)
+4 -4
View File
@@ -5,7 +5,7 @@ const { adminAuth } = require('../middleware/auth');
const { requirePermission } = require('../middleware/permissions');
const { body, query, validationResult } = require('express-validator');
const logger = require('../utils/logger');
const { getPagination } = require('../utils/routeHelpers');
const { getPagination, safeValidationErrors } = require('../utils/routeHelpers');
const { db } = require('../database/db');
const path = require('path');
const fs = require('fs').promises;
@@ -86,7 +86,7 @@ router.post('/validate', requirePermission('backup.restore'), [
if (!errors.isEmpty()) {
return res.status(400).json({
success: false,
errors: errors.array()
errors: safeValidationErrors(errors)
});
}
@@ -158,7 +158,7 @@ router.post('/start', requirePermission('backup.restore'), [
if (!errors.isEmpty()) {
return res.status(400).json({
success: false,
errors: errors.array()
errors: safeValidationErrors(errors)
});
}
@@ -690,7 +690,7 @@ router.put('/settings', requirePermission('backup.restore'), [
if (!errors.isEmpty()) {
return res.status(400).json({
success: false,
errors: errors.array()
errors: safeValidationErrors(errors)
});
}
+42 -11
View File
@@ -1,6 +1,7 @@
const express = require('express');
const multer = require('multer');
const path = require('path');
const { uploadedAssetPath } = require('../utils/safePath');
const fs = require('fs').promises;
const { body, validationResult } = require('express-validator');
const { db, logActivity } = require('../database/db');
@@ -22,8 +23,9 @@ const { sanitizeCss } = require('../utils/cssSanitizer');
const { upsertAppSetting } = require('../utils/appSettings');
const { clearShareLinkSettingsCache } = require('../services/shareLinkService');
const { resetSecurityConfigCache } = require('../utils/authSecurity');
const { errorResponse } = require('../utils/routeHelpers');
const { errorResponse, safeValidationErrors } = require('../utils/routeHelpers');
const logger = require('../utils/logger');
const { measureLocalStorageUsage } = require('../services/localStorageUsage');
const router = express.Router();
const { clearMaxFilesPerUploadCache, MAX_ALLOWED_FILES_PER_UPLOAD } = require('../services/uploadSettings');
const watermarkService = require('../services/watermarkService');
@@ -566,10 +568,13 @@ router.put('/branding', adminAuth, requirePermission('settings.edit'), async (re
currentFaviconUrl = currentFaviconSetting.setting_value;
}
if (currentFaviconUrl && typeof currentFaviconUrl === 'string' && currentFaviconUrl.startsWith('/uploads/favicons/')) {
// Delete the file from filesystem
const relativePath = currentFaviconUrl.replace(/^\//, '');
const faviconPath = path.join(getStoragePath(), relativePath);
// Containment: the stored URL is admin-writable, so only the leaf
// name is used and it is joined onto the fixed favicon directory. A
// prefix test alone let `/uploads/favicons/../../<anything>` pass
// and path.join collapse it -- an arbitrary-file delete for any
// holder of settings.edit.
const faviconPath = uploadedAssetPath(currentFaviconUrl, 'favicons', getStoragePath());
if (faviconPath) {
try {
await fs.unlink(faviconPath);
logger.info('Deleted favicon file:', faviconPath);
@@ -597,10 +602,9 @@ router.put('/branding', adminAuth, requirePermission('settings.edit'), async (re
currentLogoUrl = currentLogoSetting.setting_value;
}
if (currentLogoUrl && typeof currentLogoUrl === 'string' && currentLogoUrl.startsWith('/uploads/logos/')) {
// Delete the file from filesystem
const relativePath = currentLogoUrl.replace(/^\//, '');
const logoPath = path.join(getStoragePath(), relativePath);
// Same containment as the favicon branch above.
const logoPath = uploadedAssetPath(currentLogoUrl, 'logos', getStoragePath());
if (logoPath) {
try {
await fs.unlink(logoPath);
logger.info('Deleted logo file:', logoPath);
@@ -1174,6 +1178,8 @@ router.put('/seo', adminAuth, requirePermission('settings.edit'), async (req, re
router.get('/storage/info', adminAuth, requirePermission('settings.view'), async (req, res) => {
try {
// Get total storage used
// Catalogued original bytes. Reported, but no longer as "used" (#1164) —
// in reference mode those files are on a NAS and none of them are here.
const totalStorage = await db('photos')
.sum('size_bytes as total')
.first();
@@ -1254,7 +1260,25 @@ router.get('/storage/info', adminAuth, requirePermission('settings.view'), async
}
}
const totalUsed = totalStorage?.total || 0;
// What is actually on this disk. This is what the soft limit is compared
// against and what the recommendation below is derived from, so getting it
// from the catalogued originals was the load-bearing half of #1164: a
// reference-mode install got a disk-capacity recommendation computed from
// bytes that are not on the disk.
//
// Gated BEFORE the walk: this endpoint is polled by the sidebar, and an S3
// install with a large local tree would otherwise pay a full traversal on
// every cold cache only to discard the result.
const catalogedBytes = Number(totalStorage?.total) || 0;
const usesLocalBackend = (process.env.STORAGE_BACKEND || 'local').toLowerCase() !== 's3';
let localUsage = null;
try {
if (usesLocalBackend) localUsage = await measureLocalStorageUsage();
} catch (err) {
logger.warn(`Storage measurement failed, falling back to catalogued bytes: ${err.message}`);
}
const measuredFromDisk = usesLocalBackend && !!localUsage;
const totalUsed = measuredFromDisk ? localUsage.total : catalogedBytes;
const parseBytesValue = (value) => {
const numeric = Number(value);
@@ -1376,6 +1400,13 @@ router.get('/storage/info', adminAuth, requirePermission('settings.view'), async
res.json({
total_used: totalUsed,
// What total_used used to be, kept so the UI can show both and the
// difference stops being invisible.
cataloged_bytes: catalogedBytes,
storage_measurement: measuredFromDisk ? 'disk' : (usesLocalBackend ? 'unavailable' : 'catalog'),
storage_breakdown: measuredFromDisk ? localUsage.breakdown : null,
storage_partial: measuredFromDisk ? localUsage.partial : false,
excluded_external_root: measuredFromDisk ? localUsage.excludedExternalRoot : null,
archive_storage: archiveStorage,
storage_by_event: storageByEvent,
storage_limit: effectiveSoftLimit,
@@ -1446,7 +1477,7 @@ router.put('/security/rate-limit', adminAuth, requirePermission('settings.edit')
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const {
+4 -3
View File
@@ -11,6 +11,7 @@
*/
const express = require('express');
const { body, param, validationResult } = require('express-validator');
const { safeValidationErrors } = require('../utils/routeHelpers');
const { adminAuth } = require('../middleware/auth');
const { requirePermission } = require('../middleware/permissions');
const { requireEventOwnership } = require('../middleware/ownership');
@@ -32,7 +33,7 @@ router.get(
requireEventOwnership,
async (req, res) => {
const errors = validationResult(req);
if (!errors.isEmpty()) return res.status(400).json({ errors: errors.array() });
if (!errors.isEmpty()) return res.status(400).json({ errors: safeValidationErrors(errors) });
try {
const rows = await galleryShortUrlService.listForEvent(parseInt(req.params.eventId, 10));
res.json({ shortUrls: rows });
@@ -56,7 +57,7 @@ router.post(
requireEventOwnership,
async (req, res) => {
const errors = validationResult(req);
if (!errors.isEmpty()) return res.status(400).json({ errors: errors.array() });
if (!errors.isEmpty()) return res.status(400).json({ errors: safeValidationErrors(errors) });
try {
const row = await galleryShortUrlService.createShortUrl({
eventId: parseInt(req.params.eventId, 10),
@@ -96,7 +97,7 @@ router.delete(
param('id').isInt({ min: 1 }),
async (req, res) => {
const errors = validationResult(req);
if (!errors.isEmpty()) return res.status(400).json({ errors: errors.array() });
if (!errors.isEmpty()) return res.status(400).json({ errors: safeValidationErrors(errors) });
try {
const ok = await galleryShortUrlService.softDelete(
parseInt(req.params.id, 10),
+62 -28
View File
@@ -3,12 +3,27 @@ const router = express.Router();
const { db } = require('../database/db');
const { adminAuth } = require('../middleware/auth');
const { requirePermission } = require('../middleware/permissions');
const { generateThumbnail, ensurePreviewImage } = require('../services/imageProcessor');
const path = require('path');
const fs = require('fs').promises;
const { ensureThumbnail, ensurePreviewImage } = require('../services/imageProcessor');
const { getStorage } = require('../services/storage');
const logger = require('../utils/logger');
const getStoragePath = () => process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
/**
* Do these two stored paths address the same object?
*
* Compared the way the storage backends do, not as raw strings.
* LocalFsStorage._resolve and S3StorageBackend._key both fold `\` to `/` and
* strip a leading `./`, so a legacy thumbnail_path in any of those shapes is
* the SAME file as the freshly generated POSIX key while comparing unequal
* and the "the key moved, delete the old one" branch below would then delete
* the thumbnail that had just been written.
*/
function sameStorageKey(a, b) {
const canonical = (key) => String(key)
.replace(/\\/g, '/')
.replace(/^\.?\/+/, '')
.replace(/\/+/g, '/');
return canonical(a) === canonical(b);
}
// Parse JSON-encoded setting values
function parseSettingValue(value) {
@@ -125,10 +140,21 @@ router.post('/regenerate', adminAuth, requirePermission('photos.edit'), async (r
try {
const { eventId } = req.body; // Optional: regenerate for specific event only
let query = db('photos').select('id', 'event_id', 'path');
// source_origin/external_relpath/filename are what ensureThumbnail branches
// on to resolve an external source off its mount instead of under
// events/active. thumbnail_path is selected so it can be nulled — see below.
let query = db('photos').select(
'id', 'event_id', 'path', 'media_type', 'mime_type', 'thumbnail_path',
'source_origin', 'external_relpath', 'filename'
);
if (eventId) {
query = query.where('event_id', eventId);
}
// Skip videos: their thumbnail is a poster frame from videoProcessor, so
// handing the container file to Sharp only ever produced an error per row.
query = query.where(function() {
this.whereNull('media_type').orWhere('media_type', '!=', 'video');
});
const photos = await query;
@@ -149,30 +175,38 @@ router.post('/regenerate', adminAuth, requirePermission('photos.edit'), async (r
for (const photo of photos) {
try {
const storagePath = getStoragePath();
const originalPath = path.join(storagePath, 'events/active', photo.path);
// Check if original file exists
try {
await fs.access(originalPath);
} catch (err) {
logger.warn(`Original file not found for photo ${photo.id}: ${originalPath}`);
errorCount++;
continue;
}
// Regenerate thumbnail
const thumbnailPath = await generateThumbnail(originalPath, { regenerate: true });
if (thumbnailPath) {
// Update database with new thumbnail path
await db('photos')
.where({ id: photo.id })
.update({
thumbnail_path: thumbnailPath,
updated_at: db.fn.now()
// Through ensureThumbnail, not a hand-rolled path (#1129). This route
// used to resolve every source as `storage/events/active/<path>` and
// fs.access it — a location that does not exist for external or
// reference rows, whose originals live under events.external_path. So
// every one of them failed the check and was counted as an error: on a
// reference install the endpoint rebuilt nothing while the UI reported
// success, because the response is sent before this loop starts.
//
// ensureThumbnail already resolves both source kinds, uses the
// per-photo ext<id>_ output name so two events referencing one NAS
// basename cannot clobber each other, and writes thumbnail_path back
// itself. Nulling thumbnail_path is what stops it short-circuiting on
// isThumbnailValid — necessary rather than cosmetic, because the old
// thumbnail is normally still readable at exactly the moment someone
// presses regenerate.
const newThumbnailPath = await ensureThumbnail({ ...photo, thumbnail_path: null });
if (newThumbnailPath) {
// Drop the superseded rendition when the key MOVED. On S3 the source
// is downloaded to a randomly-named temp file and, for non-RAW input,
// the key is derived from that name — so it differs every run, and
// nulling thumbnail_path hides the old key from everything that would
// otherwise clean it up. Guarded on the key actually changing: local
// storage is stable, and deleting the equal key would delete the file
// just written.
if (photo.thumbnail_path && !sameStorageKey(photo.thumbnail_path, newThumbnailPath)) {
await getStorage().delete(photo.thumbnail_path).catch((err) => {
logger.warn(
`Could not remove superseded thumbnail ${photo.thumbnail_path} for photo ${photo.id}: ${err.message}`
);
});
}
successCount++;
logger.info(`Regenerated thumbnail for photo ${photo.id}`);
} else {
+5 -4
View File
@@ -17,6 +17,7 @@
const express = require('express');
const { body, query, validationResult } = require('express-validator');
const { safeValidationErrors } = require('../utils/routeHelpers');
const { db, logActivity } = require('../database/db');
const { adminAuth } = require('../middleware/auth');
const { requirePermission } = require('../middleware/permissions');
@@ -106,7 +107,7 @@ router.post(
async (req, res) => {
try {
const errors = validationResult(req);
if (!errors.isEmpty()) return res.status(400).json({ errors: errors.array() });
if (!errors.isEmpty()) return res.status(400).json({ errors: safeValidationErrors(errors) });
const { name, url, events, active = true, filter, template } = req.body;
const { plaintext, preview } = webhookService.generateSecret();
@@ -188,7 +189,7 @@ router.put(
async (req, res) => {
try {
const errors = validationResult(req);
if (!errors.isEmpty()) return res.status(400).json({ errors: errors.array() });
if (!errors.isEmpty()) return res.status(400).json({ errors: safeValidationErrors(errors) });
const row = await db('webhooks').where({ id: req.params.id }).first();
if (!row) return res.status(404).json({ error: 'Webhook not found' });
@@ -241,7 +242,7 @@ router.post(
async (req, res) => {
try {
const errors = validationResult(req);
if (!errors.isEmpty()) return res.status(400).json({ errors: errors.array() });
if (!errors.isEmpty()) return res.status(400).json({ errors: safeValidationErrors(errors) });
const row = await db('webhooks').where({ id: req.params.id }).first();
if (!row) return res.status(404).json({ error: 'Webhook not found' });
@@ -293,7 +294,7 @@ router.get(
async (req, res) => {
try {
const errors = validationResult(req);
if (!errors.isEmpty()) return res.status(400).json({ errors: errors.array() });
if (!errors.isEmpty()) return res.status(400).json({ errors: safeValidationErrors(errors) });
const webhookId = req.params.id;
const exists = await db('webhooks').where({ id: webhookId }).first();
+67 -16
View File
@@ -2,6 +2,16 @@ const express = require('express');
const bcrypt = require('bcrypt');
const jwt = require('jsonwebtoken');
const { body, validationResult } = require('express-validator');
/**
* express-validator's errors.array() carries `value` -- the submitted input --
* so returning it verbatim reflects the caller's password back in the 400 body.
* Five routes in this file validate a password field, and the strength endpoint
* is unauthenticated behind a 50mb JSON limit, which also made the rejection
* itself an allocation amplifier. Everything except `value` is kept, so the
* response shape both frontend consumers rely on (`msg`, `path`) is unchanged.
*/
const safeValidationErrors = (errors) => errors.array().map(({ value, ...rest }) => rest);
const { db, logActivity } = require('../database/db');
const { formatBoolean } = require('../utils/dbCompat');
const { verifyRecaptcha } = require('../services/recaptcha');
@@ -16,6 +26,9 @@ const {
const { endSession } = require('../middleware/sessionTimeout');
const { revokeToken } = require('../utils/tokenRevocation');
const { timingSafeEqualStr } = require('../utils/timingSafe');
// Well-formed bcrypt hash that matches nothing; compared against when there is
// no account so the unknown-user path costs the same as a wrong password.
const DUMMY_BCRYPT_HASH = '$2b$10$abcdefghijklmnopqrstuuABCDEFGHIJKLMNOPQRSTUVWXYZ01234';
const logger = require('../utils/logger');
const { errorResponse } = require('../utils/routeHelpers');
const {
@@ -30,6 +43,7 @@ const { getEventShareToken, resolveShareIdentifier } = require('../services/shar
const { getClientIp } = require('../utils/requestIp');
const {
validatePasswordInContext,
MAX_PASSWORD_LENGTH,
getBcryptRounds,
logPasswordValidationFailure
} = require('../utils/passwordValidation');
@@ -80,13 +94,15 @@ async function completeAdminLogin(req, res, admin, ipAddress, userAgent, lockout
// Admin login with enhanced security
router.post('/admin/login', [
body('username').notEmpty().trim(),
body('password').notEmpty()
// Length caps: an unbounded username reached the lockout lookup, bcrypt,
// the failed-attempt log line and login_attempts.identifier as sent.
body('username').isString().trim().notEmpty().isLength({ max: 255 }),
body('password').isString().notEmpty().isLength({ max: MAX_PASSWORD_LENGTH })
], async (req, res) => {
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { username, password, recaptchaToken } = req.body;
@@ -130,7 +146,13 @@ router.post('/admin/login', [
.first();
// Use generic error to prevent user enumeration
if (!admin || !await bcrypt.compare(password, admin.password_hash)) {
// Always run one bcrypt compare so an unknown username costs the same
// ~100ms as a wrong password; short-circuiting here was a timing oracle
// for username enumeration despite the generic message.
const passwordMatches = admin
? await bcrypt.compare(password, admin.password_hash)
: await bcrypt.compare(password, DUMMY_BCRYPT_HASH).then(() => false);
if (!passwordMatches) {
await trackFailedAttempt(username, ipAddress, userAgent);
return res.status(401).json({ error: getGenericAuthError() });
}
@@ -174,7 +196,7 @@ router.post('/admin/login/mfa', [
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { mfaToken, code } = req.body;
@@ -309,13 +331,13 @@ router.post('/logout', async (req, res) => {
// Gallery password verification with enhanced security
router.post('/gallery/verify', [
body('slug').notEmpty().trim(),
body('password').optional().isString()
body('slug').isString().trim().notEmpty().isLength({ max: 255 }),
body('password').optional().isString().isLength({ max: MAX_PASSWORD_LENGTH })
], async (req, res) => {
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { slug, password, recaptchaToken } = req.body;
@@ -327,7 +349,7 @@ router.post('/gallery/verify', [
if (!event) {
// Perform a dummy bcrypt compare to prevent timing-based slug enumeration
await bcrypt.compare(password || '', '$2b$10$abcdefghijklmnopqrstuuABCDEFGHIJKLMNOPQRSTUVWXYZ01234');
await bcrypt.compare(password || '', DUMMY_BCRYPT_HASH);
await trackFailedAttempt(`gallery:${slug}`, ipAddress, userAgent);
return res.status(401).json({ error: 'Invalid gallery or password' });
}
@@ -421,12 +443,12 @@ router.post('/gallery/verify', [
// Client access login (PIN-based)
router.post('/gallery/:slug/client-login', [
body('password').notEmpty().isString()
body('password').notEmpty().isString().isLength({ max: MAX_PASSWORD_LENGTH })
], async (req, res) => {
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { slug } = req.params;
@@ -502,7 +524,7 @@ router.post('/gallery/share-login', [
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { slug, token } = req.body;
@@ -725,7 +747,18 @@ router.get('/session', async (req, res) => {
expiresIn: Math.floor(remainingTime),
user: decoded.username || decoded.eventSlug,
eventSlug: decoded.eventSlug,
adminUsername: decoded.username
adminUsername: decoded.username,
// What KIND of gallery session this cookie is (#1149). The frontend
// kept this in sessionStorage, which is per-tab: reopening a gallery
// in a second tab lost 'client' while the cookie — and therefore the
// backend — still treated it as one. Reported from the token so a
// restored session knows what it actually is.
//
// viaCustomer marks a portal-minted token, which opens the gallery
// without the password. Also a credential, and it does not look like
// one: it runs at accessLevel 'guest'.
accessLevel: decoded.type === 'gallery' ? (decoded.accessLevel || 'guest') : undefined,
viaCustomer: decoded.type === 'gallery' ? decoded.via === 'customer' : undefined
});
} catch (err) {
res.json({
@@ -749,7 +782,7 @@ router.post('/admin/change-password', [
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { currentPassword, newPassword } = req.body;
@@ -820,11 +853,27 @@ router.post('/admin/change-password', [
});
// Password strength check endpoint (for real-time validation)
//
// Unauthenticated, and it feeds the request body straight into zxcvbn, whose
// matching is superlinear and synchronous. Without the length bound a single
// request stops the event loop for the whole process -- ~5s at 1,000
// characters and unbounded past that. validatePassword() enforces the same cap
// for every caller; this one keeps the oversized body from being accepted at
// the edge at all.
router.post('/password-strength', [
body('password').notEmpty(),
body('password').isString().isLength({ min: 1, max: MAX_PASSWORD_LENGTH })
.withMessage(`Password must be 1-${MAX_PASSWORD_LENGTH} characters`),
body('context').isIn(['admin', 'gallery']).optional()
], async (req, res) => {
try {
// The validators above only RECORD errors; without this the oversized body
// reached zxcvbn anyway and the endpoint answered 200, so the edge cap was
// decorative. The cap in validatePassword() is still the real control.
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { password, context = 'gallery' } = req.body;
// Get user data if available (for context-aware validation)
@@ -834,7 +883,9 @@ router.post('/password-strength', [
userData.email = req.admin.email;
}
const validation = validatePasswordInContext(password, context, userData);
// validatePasswordInContext is async; unawaited this resolved to a Promise
// and every field below came back undefined.
const validation = await validatePasswordInContext(password, context, userData);
res.json({
valid: validation.valid,
+14 -9
View File
@@ -17,9 +17,10 @@ const jwt = require('jsonwebtoken');
const { body, param, validationResult } = require('express-validator');
const { db, logActivity } = require('../database/db');
const { formatBoolean } = require('../utils/dbCompat');
const { getBcryptRounds } = require('../utils/passwordValidation');
const { getBcryptRounds, MAX_PASSWORD_LENGTH } = require('../utils/passwordValidation');
const { assertContractPdfPath } = require('../utils/safePath');
const logger = require('../utils/logger');
const { errorResponse } = require('../utils/routeHelpers');
const { errorResponse, safeValidationErrors } = require('../utils/routeHelpers');
const { getClientIp } = require('../utils/requestIp');
const { customerAuth } = require('../middleware/customerAuth');
const { setGalleryAuthCookies } = require('../utils/tokenUtils');
@@ -147,7 +148,7 @@ router.get('/events/:slug/access-token', [
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { slug } = req.params;
@@ -284,7 +285,7 @@ router.put('/profile', [
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
// Normalise incoming values: trim strings, drop empty → null so the DB
@@ -329,14 +330,14 @@ router.put('/profile', [
*/
router.post('/profile/password', [
customerAuth,
body('currentPassword').isString().isLength({ min: 1 }),
body('newPassword').isString().isLength({ min: 8 })
body('currentPassword').isString().isLength({ min: 1, max: MAX_PASSWORD_LENGTH }),
body('newPassword').isString().isLength({ min: 8, max: MAX_PASSWORD_LENGTH })
.withMessage('Password must be at least 8 characters'),
], async (req, res) => {
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { currentPassword, newPassword } = req.body;
@@ -706,9 +707,13 @@ router.get('/contracts/:id/pdf', customerAuth, async (req, res) => {
res.set('Content-Disposition', `inline; filename="${contract.contract_number}.pdf"`);
return res.send(buf);
}
// Same containment the admin and public contract routes apply: the DB
// path is written by the service layer today, but a bad row must not
// turn this into an arbitrary-file read.
const safePath = assertContractPdfPath(filePath);
res.set('Content-Type', 'application/pdf');
res.set('Content-Disposition', `inline; filename="${path.basename(filePath)}"`);
fs.createReadStream(filePath).pipe(res);
res.set('Content-Disposition', `inline; filename="${path.basename(safePath)}"`);
fs.createReadStream(safePath).pipe(res);
} catch (error) {
errorResponse(res, error, 500, 'Failed to render contract PDF');
}
+15 -7
View File
@@ -16,6 +16,9 @@ const express = require('express');
const bcrypt = require('bcrypt');
const jwt = require('jsonwebtoken');
const { body, param, validationResult } = require('express-validator');
const { safeValidationErrors } = require('../utils/routeHelpers');
const { MAX_PASSWORD_LENGTH } = require('../utils/passwordValidation');
const DUMMY_BCRYPT_HASH = '$2b$10$abcdefghijklmnopqrstuuABCDEFGHIJKLMNOPQRSTUVWXYZ01234';
const { db, logActivity } = require('../database/db');
const { formatBoolean } = require('../utils/dbCompat');
const { verifyRecaptcha } = require('../services/recaptcha');
@@ -65,12 +68,12 @@ const TOKEN_TTL_SECONDS = 24 * 60 * 60; // mirrors admin tokens
// gallery JWTs (instant per-gallery revocation).
router.post('/login', [
body('email').isEmail().normalizeEmail(IDENTITY_PRESERVING_NORMALIZE_EMAIL).withMessage('Valid email is required'),
body('password').isString().notEmpty(),
body('password').isString().notEmpty().isLength({ max: MAX_PASSWORD_LENGTH }),
], async (req, res) => {
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { email, password, recaptchaToken } = req.body;
@@ -99,7 +102,12 @@ router.post('/login', [
const customer = await db('customer_accounts').where('email', email).first();
// Generic error to prevent user enumeration — same wording as admin login.
if (!customer || !customer.password_hash || !await bcrypt.compare(password, customer.password_hash)) {
// One bcrypt compare on every path so an unknown email is not a timing
// oracle (the dummy hash matches nothing).
const passwordMatches = customer && customer.password_hash
? await bcrypt.compare(password, customer.password_hash)
: await bcrypt.compare(password, DUMMY_BCRYPT_HASH).then(() => false);
if (!passwordMatches) {
await trackFailedAttempt(lockoutKey, ipAddress, userAgent);
return res.status(401).json({ error: getGenericAuthError() });
}
@@ -273,7 +281,7 @@ router.post('/accept-invite', [
// Length floor enforced again here for an early reject; the full
// policy (uppercase + digit) is checked below so we can surface a
// specific message rather than a generic validator error.
body('password').isString().isLength({ min: 8 })
body('password').isString().isLength({ min: 8, max: MAX_PASSWORD_LENGTH })
.withMessage('Password must be at least 8 characters'),
// Optional structured profile from the accept-invite form. Mirrors
// the admin prefill shape — anything the customer types here wins
@@ -296,7 +304,7 @@ router.post('/accept-invite', [
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
const { token, name, password, profile } = req.body;
@@ -355,11 +363,11 @@ router.get('/password-reset/:token', [
*/
router.post('/password-reset', [
body('token').isLength({ min: 64, max: 64 }).matches(/^[a-f0-9]+$/i),
body('password').isString().isLength({ min: 8 }).withMessage('Password must be at least 8 characters'),
body('password').isString().isLength({ min: 8, max: MAX_PASSWORD_LENGTH }).withMessage('Password must be at least 8 characters'),
], async (req, res) => {
try {
const errors = validationResult(req);
if (!errors.isEmpty()) return res.status(400).json({ errors: errors.array() });
if (!errors.isEmpty()) return res.status(400).json({ errors: safeValidationErrors(errors) });
const policyError = validateCustomerPassword(req.body.password);
if (policyError) {
return res.status(400).json({
+423 -71
View File
@@ -10,6 +10,8 @@ const { parseBooleanInput } = require('../utils/parsers');
const { getAppSetting } = require('../utils/appSettings');
const archiver = require('archiver');
const path = require('path');
const { resolvePhotoContentType } = require('../utils/photoContentType');
const { timingSafeEqualStr } = require('../utils/timingSafe');
const router = express.Router();
// #756: a NULL per-event hero_logo_visible means "inherit the global
@@ -29,7 +31,8 @@ const { resolveGuest } = require('../middleware/guestAuth');
const { generateGuestIdentifier } = require('../middleware/feedbackRateLimit');
const secureImageService = require('../services/secureImageService');
const logger = require('../utils/logger');
const { resolvePhotoFilePath } = require('../services/photoResolver');
const { pipeStreamToResponse } = require('../utils/streamResponse');
const { resolvePhotoFilePath, resolvePhotoStorageKey } = require('../services/photoResolver');
const { getEventShareToken, resolveShareIdentifier, buildShareLinkVariants } = require('../services/shareLinkService');
const { handleAsync, errorResponse } = require('../utils/routeHelpers');
const { NotFoundError } = require('../utils/errors');
@@ -53,6 +56,43 @@ const fs = require('fs');
// Get storage path from environment or default
const getStoragePath = () => process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
/**
* Parse a single-range `Range: bytes=` header against a known size.
*
* Returns null for absent, malformed, multi-range or unsatisfiable headers
* every one of which the caller answers with a normal 200 full body, which is
* what a client that sent an unparseable range would get today anyway.
* Validating matters because an unchecked parse yields NaN bounds and a 206
* with a nonsense Content-Range, which corrupts a resumed download rather
* than merely failing it.
*/
function parseByteRange(header, size) {
if (!header || typeof header !== 'string' || !size) return null;
const match = /^bytes=(\d*)-(\d*)$/.exec(header.trim());
if (!match) return null;
const [, rawStart, rawEnd] = match;
if (rawStart === '' && rawEnd === '') return null;
let start;
let end;
if (rawStart === '') {
// Suffix form: the last N bytes.
const suffix = parseInt(rawEnd, 10);
if (!suffix) return null;
start = Math.max(0, size - suffix);
end = size - 1;
} else {
start = parseInt(rawStart, 10);
end = rawEnd === '' ? size - 1 : parseInt(rawEnd, 10);
}
if (!Number.isFinite(start) || !Number.isFinite(end)) return null;
if (start > end || start >= size) return null;
return { start, end: Math.min(end, size - 1) };
}
// Check for slug redirect (for renamed events)
async function checkSlugRedirect(slug) {
try {
@@ -129,7 +169,7 @@ router.get('/:slug/verify-token/:token', handleAsync(async (req, res) => {
}
const expectedToken = getEventShareToken(event);
if (token !== expectedToken) {
if (!expectedToken || !timingSafeEqualStr(String(token), expectedToken)) {
throw new NotFoundError('Gallery', 'Invalid gallery link');
}
@@ -205,7 +245,7 @@ router.get('/:slug/info', async (req, res) => {
// If token provided, verify it matches the share link
if (token) {
const expectedToken = getEventShareToken(event);
if (!expectedToken || token !== expectedToken) {
if (!expectedToken || !timingSafeEqualStr(String(token), expectedToken)) {
return res.status(404).json({ error: 'Invalid gallery link' });
}
}
@@ -412,7 +452,10 @@ router.get('/:slug/show/:token/state', handleAsync(async (req, res) => {
router.get('/:slug/photos', verifyGalleryAccess, resolveGuest, async (req, res) => {
try {
// Get filter and sort parameters from query
const { filter, guest_id, sort = 'upload_date', order = 'desc' } = req.query;
// `guest_id` is deliberately NOT read from the query string: the viewer's
// own feedback is resolved from the request identity instead (see the
// filter block). The frontend still sends it; it is ignored.
const { filter, sort = 'upload_date', order = 'desc' } = req.query;
// Get watermark settings to generate cache-busting version for URLs
const watermarkSettings = await watermarkService.getWatermarkSettings();
@@ -443,20 +486,78 @@ router.get('/:slug/photos', verifyGalleryAccess, resolveGuest, async (req, res)
});
}
// Apply sort option
// Apply sort option.
//
// Every branch carries photos.id as a tiebreaker (#1172). Without one the
// order within a tie is whatever the engine happens to return, and ties are
// the normal case rather than the exception: a bulk import writes hundreds
// of rows inside the same second, so uploaded_at collapses — and with
// captured_at NULL the COALESCE below collapses onto it too. The visible
// symptom is a grid that reshuffles between page loads. id is insertion
// order, so it also makes the fallback ordering meaningful rather than
// arbitrary.
if (sort === 'capture_date') {
// Sort by capture date, falling back to uploaded_at if capture date is null
photosQuery = photosQuery.orderByRaw('COALESCE(photos.captured_at, photos.uploaded_at) ' + sortOrder);
// Sort by capture date, falling back to uploaded_at if capture date is null.
//
// On SQLite that fallback cannot be a plain COALESCE, because the two
// columns do not hold one type. photos.captured_at ends up carrying three
// different storage classes:
//
// integer managed uploads — photoProcessor.js:441 writes a Date, which
// the sqlite3 binding stores as epoch milliseconds
// text external imports and the backfill, which write ISO-8601
// ('2026-06-03T01:15:00.000Z') per the CLAUDE.md rule that
// Dates must not be handed to the binding in tests
// null no capture date, so the sort falls through to uploaded_at —
// usually text in knex's 'YYYY-MM-DD HH:MM:SS' default shape,
// but epoch milliseconds on rows a .picpeak restore carried in
// from an install that stored them that way, so that column
// needs the same two branches
//
// SQLite orders INTEGER before TEXT unconditionally, so every managed
// photo carrying EXIF sorted ahead of every photo that did not, whatever
// the actual dates — a 2027 capture landing before a 2020 one. Among the
// text values the 'T' separator (0x54) also outranks the space (0x20), so
// a same-day ISO 01:15 sorted after a fallback 23:00.
//
// Normalising in the ORDER BY rather than rewriting the column: the data
// fix would have to touch every existing row and every writer, which is a
// much heavier change than the sort it is meant to correct. The cost here
// is that this sort stops using idx_photos_captured_at on SQLite — an
// acceptable trade on the fallback engine, where the alternative is an
// index-assisted wrong answer.
//
// Postgres is untouched: captured_at is a real timestamp there, so
// COALESCE already compares correctly.
if (db.client.config.client === 'pg') {
photosQuery = photosQuery
.orderByRaw('COALESCE(photos.captured_at, photos.uploaded_at) ' + sortOrder);
} else {
photosQuery = photosQuery.orderByRaw(`CASE
WHEN typeof(photos.captured_at) IN ('integer', 'real') THEN datetime(photos.captured_at / 1000, 'unixepoch')
WHEN photos.captured_at IS NOT NULL THEN replace(replace(substr(photos.captured_at, 1, 19), 'T', ' '), 'Z', '')
WHEN typeof(photos.uploaded_at) IN ('integer', 'real') THEN datetime(photos.uploaded_at / 1000, 'unixepoch')
ELSE substr(photos.uploaded_at, 1, 19)
END ${sortOrder}`);
}
photosQuery = photosQuery.orderBy('photos.id', sortOrder);
} else if (sort === 'filename') {
photosQuery = photosQuery.orderBy('photos.filename', sortOrder);
photosQuery = photosQuery.orderBy('photos.filename', sortOrder).orderBy('photos.id', sortOrder);
} else {
// Default: sort by upload date
photosQuery = photosQuery.orderBy('photos.uploaded_at', sortOrder);
photosQuery = photosQuery.orderBy('photos.uploaded_at', sortOrder).orderBy('photos.id', sortOrder);
}
// Execute the query
let photos = await photosQuery;
// Check if feedback should be visible to guests. Read BEFORE the filter
// block, not after: the filters below consult it, because a filter that
// selects on other people's feedback is a way of reading that feedback.
const feedbackService = require('../services/feedbackService');
const feedbackSettings = await feedbackService.getEventFeedbackSettings(req.event.id);
const showFeedbackToGuests = isClient || parseBooleanInput(feedbackSettings.show_feedback_to_guests, true);
// Apply filtering if requested (supports global stats + per-guest interactions)
if (filter) {
const filterTokens = new Set(
@@ -486,10 +587,37 @@ router.get('/:slug/photos', verifyGalleryAccess, resolveGuest, async (req, res)
});
};
// Whose feedback counts as "mine" for these filters.
//
// Resolved from the REQUEST, the same either/or the per-viewer is_liked
// query below uses — never from the `guest_id` query parameter. Two
// reasons, and both matter now that this is the only half left when
// feedback is hidden:
//
// - It never matched. The frontend's `gallery_guest_id` is a
// localStorage string it invents (`guest_<ts>_<rand>`) and never
// sends when submitting feedback; submissions store
// generateGuestIdentifier(req). So this lookup found nothing, and
// the filters only ever worked through the aggregate half — which
// is exactly the half now gated.
// - It is caller-controlled. Accepting an identifier from the query
// string would let anyone holding someone else's read their hidden
// memberships one token at a time, straight back through the gate.
//
// Hidden rows are excluded, matching what the viewer can actually SEE:
// getPhotoFeedback drops is_hidden for the guest's own feedback too.
// Unapproved rows are NOT excluded — a comment still in the moderation
// queue is still the viewer's own, and that same read keeps it.
let guestFeedbackByType = null;
if (guest_id) {
const guestFeedbackRows = await db('photo_feedback')
.where({ event_id: req.event.id, guest_identifier: guest_id })
{
const viewerFeedback = db('photo_feedback')
.where({ event_id: req.event.id, is_hidden: false });
if (req.guest?.id) {
viewerFeedback.where('guest_id', req.guest.id);
} else {
viewerFeedback.where('guest_identifier', generateGuestIdentifier(req));
}
const guestFeedbackRows = await viewerFeedback
.select('photo_id', 'feedback_type');
guestFeedbackByType = guestFeedbackRows.reduce((acc, row) => {
@@ -508,39 +636,50 @@ router.get('/:slug/photos', verifyGalleryAccess, resolveGuest, async (req, res)
}
};
// Every token below is an OR of two halves: what THIS viewer marked,
// and what ANYONE marked. The second half is other people's feedback,
// so it is gated on show_feedback_to_guests exactly like the counts
// this endpoint returns.
//
// Without the gate the setting only hides the numbers. A guest could
// still send `?filter=liked` and get back precisely the set of photos
// other people liked — the membership, one token at a time, which is
// most of what the counts would have told them. The viewer's own half
// is always theirs to filter by.
const includeAggregate = (predicate) => {
if (showFeedbackToGuests) includeBy(predicate);
};
if (filterTokens.has('liked')) {
includeGuestMatches('like');
includeBy(photo => (photo.like_count || 0) > 0);
includeAggregate(photo => (photo.like_count || 0) > 0);
}
if (filterTokens.has('favorited')) {
includeGuestMatches('favorite');
includeBy(photo => (photo.favorite_count || 0) > 0);
includeAggregate(photo => (photo.favorite_count || 0) > 0);
}
if (filterTokens.has('rated')) {
includeGuestMatches('rating');
includeBy(photo => (photo.average_rating || 0) > 0);
includeAggregate(photo => (photo.average_rating || 0) > 0);
}
if (filterTokens.has('commented')) {
includeGuestMatches('comment');
const commentedRows = await db('photo_feedback')
.where({ event_id: req.event.id, feedback_type: 'comment', is_approved: true, is_hidden: false })
.groupBy('photo_id')
.select('photo_id');
commentedRows.forEach(row => include.add(row.photo_id));
if (showFeedbackToGuests) {
const commentedRows = await db('photo_feedback')
.where({ event_id: req.event.id, feedback_type: 'comment', is_approved: true, is_hidden: false })
.groupBy('photo_id')
.select('photo_id');
commentedRows.forEach(row => include.add(row.photo_id));
}
}
photos = photos.filter(photo => include.has(photo.id));
}
}
// Check if feedback should be visible to guests
const feedbackService = require('../services/feedbackService');
const feedbackSettings = await feedbackService.getEventFeedbackSettings(req.event.id);
const showFeedbackToGuests = isClient || parseBooleanInput(feedbackSettings.show_feedback_to_guests, true);
// Then get comment counts separately
const commentCounts = await db('photo_feedback')
.whereIn('photo_id', photos.map(p => p.id))
@@ -568,7 +707,12 @@ router.get('/:slug/photos', verifyGalleryAccess, resolveGuest, async (req, res)
const likedPhotoIds = new Set();
if (showFeedbackToGuests && photos.length > 0) {
const likeQuery = db('photo_feedback')
.where({ event_id: req.event.id, feedback_type: 'like' })
// Hidden rows are not there, for the viewer's OWN feedback as much as
// anyone's (#1150). getPhotoFeedback drops them and
// updatePhotoFeedbackStats does not count them — leaving the heart
// filled was the one place that disagreed, so a like the photographer
// had hidden still showed as liked on a photo whose like_count was 0.
.where({ event_id: req.event.id, feedback_type: 'like', is_hidden: false })
.whereIn('photo_id', photos.map(p => p.id));
if (req.guest?.id) {
likeQuery.where('guest_id', req.guest.id);
@@ -891,6 +1035,47 @@ router.get('/:slug/download/:photoId', verifyGalleryAccess, denySlideshowToken,
}
}
// A HEAD is a metadata probe, not a download. Answering it below the
// counters recorded every probe as a real download, and answering it below
// renderPhotoForDownload fetched and watermarked an image whose body Node
// then discards. Both happen before this point in a GET, so HEAD leaves
// here — with no side effects and no bytes read.
if (req.method === 'HEAD') {
const headUseOriginal = await getUseOriginalFilenames();
const headHeaders = {
'Content-Type': resolvePhotoContentType(photo),
'Content-Disposition': buildContentDisposition(pickRawDownloadName(photo, headUseOriginal)),
'Accept-Ranges': 'bytes',
};
// Content-Length only when the bytes ship untransformed AND the size can
// be read without fetching them. A watermark or resize changes the
// length, and the only way to learn the new one is to do the work this
// branch exists to avoid — HEAD is allowed to omit it.
const headWmSettings = await watermarkService.getWatermarkSettings();
const headWmEnabled = !!(headWmSettings && headWmSettings.enabled)
|| req.event.watermark_downloads === true
|| req.event.watermark_downloads === 1;
if (!headWmEnabled) {
try {
const headKey = resolvePhotoStorageKey(req.event, photo);
const headStorage = getStorage();
if (headKey && headStorage.kind() !== 'local') {
const headStat = await headStorage.stat(headKey);
if (!headStat) return res.status(404).json({ error: 'Photo file not found' });
headHeaders['Content-Length'] = headStat.size;
if (headStat.mtime) headHeaders['Last-Modified'] = new Date(headStat.mtime).toUTCString();
}
} catch (headErr) {
// No length is a valid HEAD; not worth failing the probe over.
logger.debug('HEAD probe could not stat the object', { photoId, error: headErr.message });
}
}
res.set(headHeaders);
return res.end();
}
// Update download count
await db('photos').where('id', photoId).increment('download_count', 1);
@@ -903,11 +1088,19 @@ router.get('/:slug/download/:photoId', verifyGalleryAccess, denySlideshowToken,
photo_id: photoId
});
let filePath;
// Where the bytes actually live. Managed photos sit behind the storage
// abstraction and on an S3/R2 deployment are not on local disk at all —
// resolving a filesystem path unconditionally here is what made every
// single-photo download fail in S3 mode, while download-all and
// secure-images worked because they already went through getStorage().
//
// resolvePhotoStorageKey returns null for external/reference photos: those
// live on a local mount and keep the filesystem path.
let storageKey = null;
try {
filePath = resolvePhotoFilePath(req.event, photo);
storageKey = resolvePhotoStorageKey(req.event, photo);
} catch (resolveError) {
logger.error('Failed to resolve photo path for download', {
logger.error('Failed to resolve photo storage key for download', {
slug: req.params.slug,
photoId,
eventId: req.event.id,
@@ -915,7 +1108,7 @@ router.get('/:slug/download/:photoId', verifyGalleryAccess, denySlideshowToken,
});
return res.status(404).json({ error: 'Photo file not found' });
}
// Get watermark settings - apply if global setting OR event-level setting is enabled
const watermarkSettings = await watermarkService.getWatermarkSettings();
const eventWatermarkEnabled = req.event.watermark_downloads === true || req.event.watermark_downloads === 1;
@@ -936,35 +1129,170 @@ router.get('/:slug/download/:photoId', verifyGalleryAccess, denySlideshowToken,
enabled: true,
text: req.event.watermark_text || watermarkSettings?.text || 'Protected'
};
const watermarkedBuffer = await watermarkService.applyWatermark(filePath, effectiveSettings);
// applyWatermark takes a PATH, and caches on it — buffer inputs skip the
// cache deliberately. In S3 mode materialize a tmp local copy and hand
// it the copy's path, exactly as the zip builders below do, so the cache
// still applies and the full-size original isn't re-processed per
// download.
let watermarkedBuffer;
try {
watermarkedBuffer = storageKey
? await withLocalCopy(storageKey, (localPath) =>
watermarkService.applyWatermark(localPath, effectiveSettings))
: await watermarkService.applyWatermark(
resolvePhotoFilePath(req.event, photo), effectiveSettings);
} catch (watermarkError) {
// Classify, the same way the pass-through branch below does. This can
// fail because the source object is gone, but equally because
// getToFile timed out, the tmp filesystem filled up, or sharp failed —
// and reporting an operational failure as 404 tells the guest their
// photo does not exist and tells us nothing.
const gone = watermarkError.code === 'ENOENT'
|| watermarkError.name === 'NoSuchKey'
|| watermarkError.name === 'NotFound'
|| watermarkError.$metadata?.httpStatusCode === 404;
logger.error('Failed to watermark photo for download', {
slug: req.params.slug,
photoId,
eventId: req.event.id,
error: watermarkError.message,
});
return gone
? res.status(404).json({ error: 'Photo file not found' })
: res.status(500).json({ error: 'Failed to download photo' });
}
res.set({
'Content-Type': photo.mime_type || 'image/jpeg',
'Content-Type': resolvePhotoContentType(photo),
'Content-Disposition': contentDisposition,
'Content-Length': watermarkedBuffer.length
});
res.send(watermarkedBuffer);
} else {
// res.download() builds Content-Disposition itself but doesn't emit the
// RFC 5987 filename* parameter, so unicode camera filenames would lose
// their bytes on download. Set the header explicitly and stream the
// file with res.sendFile-equivalent semantics.
res.set({
'Content-Type': photo.mime_type || 'image/jpeg',
'Content-Disposition': contentDisposition,
});
res.sendFile(filePath, (downloadError) => {
if (downloadError) {
logger.error('Error streaming gallery download', {
slug: req.params.slug,
photoId,
eventId: req.event.id,
error: downloadError.message,
});
}
});
return res.send(watermarkedBuffer);
}
const storage = getStorage();
if (storageKey && storage.kind() !== 'local') {
// Deliberately NOT the local path: res.sendFile emits Content-Length,
// Accept-Ranges, ETag and Last-Modified and answers Range requests with
// a 206, and a bare stream.pipe(res) has none of that. On local disk
// sendFile stays the better implementation, so it stays the branch.
//
// On S3 the parts that matter for a download are reproduced: the length
// (browsers need it for the progress indicator, which matters most on
// exactly the large files this route serves) and Range, so an
// interrupted download resumes instead of appending a second full body
// onto the partial file.
const stat = await storage.stat(storageKey);
if (!stat) {
logger.error('Photo not found in storage backend for download', {
slug: req.params.slug,
photoId,
eventId: req.event.id,
storageKey,
});
return res.status(404).json({ error: 'Photo file not found' });
}
const lastModified = stat.mtime ? new Date(stat.mtime).toUTCString() : null;
const headers = {
'Content-Type': resolvePhotoContentType(photo),
'Content-Disposition': contentDisposition,
'Accept-Ranges': 'bytes',
};
if (lastModified) headers['Last-Modified'] = lastModified;
// If-Range: a client resuming an interrupted download sends back the
// validator it was given last time. If the object has been replaced
// since — the watcher re-importing a swapped file, an admin re-upload —
// answering 206 from the NEW bytes lets the client splice two different
// versions into one corrupt file. A validator that doesn't match means
// a full 200, which is the whole point of the header.
const ifRange = req.headers['if-range'];
const staleValidator = !!ifRange && (!lastModified || ifRange.trim() !== lastModified);
const range = staleValidator ? null : parseByteRange(req.headers.range, stat.size);
// Open the stream BEFORE any header is staged or sent. stat() succeeding
// does not mean get() will: a concurrent delete or replace, or a
// transient backend error, lands here. Once writeHead(206) has gone out
// the outer catch can do nothing but throw ERR_HTTP_HEADERS_SENT, and in
// the non-range case it would send its 500 JSON underneath the staged
// image/jpeg attachment headers — a .jpg file full of JSON.
let stream;
try {
stream = range
? await storage.getRange(storageKey, range.start, range.end)
: await storage.get(storageKey);
} catch (fetchError) {
const gone = fetchError.code === 'ENOENT'
|| fetchError.name === 'NoSuchKey'
|| fetchError.name === 'NotFound'
|| fetchError.$metadata?.httpStatusCode === 404;
logger.error('Failed to open photo stream for download', {
slug: req.params.slug,
photoId,
eventId: req.event.id,
storageKey,
error: fetchError.message,
});
return gone
? res.status(404).json({ error: 'Photo file not found' })
: res.status(500).json({ error: 'Failed to download photo' });
}
if (range) {
// status()+set() rather than writeHead(): writeHead commits the
// response immediately, so a stream that resolves and THEN errors
// before its first chunk would leave pipeStreamToResponse able only to
// destroy the connection. Staged headers are flushed by the first body
// write, which means an error at byte zero can still clear them and
// return a clean, retryable status instead of a transport reset.
res.status(206).set({
...headers,
'Content-Range': `bytes ${range.start}-${range.end}/${stat.size}`,
'Content-Length': (range.end - range.start) + 1,
});
} else {
res.set({ ...headers, 'Content-Length': stat.size });
}
pipeStreamToResponse(stream, res, {
context: range ? `download range for photo ${photo.id}` : `download for photo ${photo.id}`,
});
return;
}
let filePath;
try {
filePath = resolvePhotoFilePath(req.event, photo);
} catch (resolveError) {
logger.error('Failed to resolve photo path for download', {
slug: req.params.slug,
photoId,
eventId: req.event.id,
error: resolveError.message,
});
return res.status(404).json({ error: 'Photo file not found' });
}
// res.download() builds Content-Disposition itself but doesn't emit the
// RFC 5987 filename* parameter, so unicode camera filenames would lose
// their bytes on download. Set the header explicitly and stream the
// file with res.sendFile-equivalent semantics.
res.set({
'Content-Type': resolvePhotoContentType(photo),
'Content-Disposition': contentDisposition,
});
res.sendFile(filePath, (downloadError) => {
if (downloadError) {
logger.error('Error streaming gallery download', {
slug: req.params.slug,
photoId,
eventId: req.event.id,
error: downloadError.message,
});
}
});
} catch (error) {
errorResponse(res, error, 500, 'Failed to download photo');
}
@@ -1044,7 +1372,7 @@ router.get('/:slug/download-all', verifyGalleryAccess, denySlideshowToken, async
res.setHeader('Content-Length', zipInfo.size);
res.setHeader('Content-Disposition', `attachment; filename="${req.event.slug}.zip"`);
const stream = await storage.get(zipInfo.key);
stream.pipe(res);
pipeStreamToResponse(stream, res, { context: `prepared zip for event ${req.event.id}`, missingStatus: 410 });
// Log bulk download
db('access_logs').insert({
@@ -1491,25 +1819,34 @@ router.get('/:slug/photo/:photoId',
const parts = range.replace(/bytes=/, '').split('-');
const start = parseInt(parts[0], 10);
const end = parts[1] ? parseInt(parts[1], 10) : fileSize - 1;
const chunksize = (end - start) + 1;
// Validate before writing the 206: a NaN, inverted or out-of-file
// range used to be committed to the headers and then throw while
// streaming (or read past the end).
if (!Number.isInteger(start) || !Number.isInteger(end)
|| start < 0 || end < start || start >= fileSize) {
res.set('Content-Range', `bytes */${fileSize}`);
return res.status(416).end();
}
const boundedEnd = Math.min(end, fileSize - 1);
const chunksize = (boundedEnd - start) + 1;
res.writeHead(206, {
'Content-Range': `bytes ${start}-${end}/${fileSize}`,
'Content-Range': `bytes ${start}-${boundedEnd}/${fileSize}`,
'Accept-Ranges': 'bytes',
'Content-Length': chunksize,
'Content-Type': photo.mime_type || 'video/mp4',
'Content-Type': resolvePhotoContentType(photo),
'Cache-Control': 'private, max-age=1800',
'X-Protection-Level': 'basic'
});
const file = useStorageBackend
? await storage.getRange(storageKey, start, end)
: fs.createReadStream(filePath, { start, end });
file.pipe(res);
? await storage.getRange(storageKey, start, boundedEnd)
: fs.createReadStream(filePath, { start, end: boundedEnd });
pipeStreamToResponse(file, res, { context: `video range for photo ${photo.id}` });
} else {
res.writeHead(200, {
'Content-Length': fileSize,
'Content-Type': photo.mime_type || 'video/mp4',
'Content-Type': resolvePhotoContentType(photo),
'Accept-Ranges': 'bytes',
'Cache-Control': 'private, max-age=1800',
'X-Protection-Level': 'basic'
@@ -1517,7 +1854,7 @@ router.get('/:slug/photo/:photoId',
const file = useStorageBackend
? await storage.get(storageKey)
: fs.createReadStream(filePath);
file.pipe(res);
pipeStreamToResponse(file, res, { context: `video for photo ${photo.id}` });
}
return;
}
@@ -1544,20 +1881,20 @@ router.get('/:slug/photo/:photoId',
const wmStat = await storage.stat(photo.watermark_path);
if (wmStat) {
res.set({
'Content-Type': photo.mime_type || 'image/jpeg',
'Content-Type': resolvePhotoContentType(photo),
'Content-Length': wmStat.size,
'Cache-Control': 'private, max-age=1800',
'ETag': etag,
'X-Protection-Level': 'basic'
});
const wmStream = await storage.get(photo.watermark_path);
return wmStream.pipe(res);
return pipeStreamToResponse(wmStream, res, { context: `watermarked photo ${photo.id}` });
}
} else {
const watermarkFilePath = path.join(getStoragePath(), photo.watermark_path);
if (fs.existsSync(watermarkFilePath)) {
res.set({
'Content-Type': photo.mime_type || 'image/jpeg',
'Content-Type': resolvePhotoContentType(photo),
'Cache-Control': 'private, max-age=1800',
'ETag': etag,
'X-Protection-Level': 'basic'
@@ -1583,7 +1920,7 @@ router.get('/:slug/photo/:photoId',
.catch(err => logger.warn(`Background watermark generation failed for photo ${photo.id}:`, err.message));
res.set({
'Content-Type': photo.mime_type || 'image/jpeg',
'Content-Type': resolvePhotoContentType(photo),
'Cache-Control': 'private, max-age=1800',
'ETag': etag,
'X-Protection-Level': 'basic'
@@ -1598,9 +1935,9 @@ router.get('/:slug/photo/:photoId',
});
if (useStorageBackend) {
res.set('Content-Length', stat.size);
if (photo.mime_type) res.set('Content-Type', photo.mime_type);
res.set('Content-Type', resolvePhotoContentType(photo));
const stream = await storage.get(storageKey);
stream.pipe(res);
pipeStreamToResponse(stream, res, { context: `photo ${photo.id}` });
} else {
const absolutePath = path.isAbsolute(filePath) ? filePath : path.resolve(filePath);
res.sendFile(absolutePath);
@@ -1695,7 +2032,7 @@ router.get('/:slug/thumbnail/:photoId',
} else {
res.setHeader('Content-Length', stat.size);
const stream = await storage.get(thumbnailPath);
stream.pipe(res);
pipeStreamToResponse(stream, res, { context: `thumbnail for photo ${photoId}` });
}
} catch (error) {
errorResponse(res, error, 500, 'Failed to serve thumbnail');
@@ -1781,7 +2118,7 @@ router.get('/:slug/hero/:photoId',
} else {
res.setHeader('Content-Length', stat.size);
const stream = await storage.get(heroPath);
stream.pipe(res);
pipeStreamToResponse(stream, res, { context: `hero for photo ${photoId}` });
}
} catch (error) {
logger.error('Error serving hero image:', {
@@ -1858,7 +2195,12 @@ router.get('/:slug/preview/:photoId',
}
res.set({
'Content-Type': 'image/jpeg',
// From the key, not hard-coded: a preview of a transparent or animated
// source is WebP, because JPEG carries neither. `nosniff` below means
// getting this wrong shows a broken image rather than being silently
// corrected by the browser. Pre-existing keys have no .webp suffix and
// are JPEG, so they keep their old header.
'Content-Type': previewPath.endsWith('.webp') ? 'image/webp' : 'image/jpeg',
// Cache aggressively — preview only changes on photo
// re-upload (which generates a new preview key) or settings
// regenerate (which writes a new mtime + ETag).
@@ -1870,6 +2212,16 @@ router.get('/:slug/preview/:photoId',
});
if (watermarkSettings && watermarkSettings.enabled) {
// No Content-Type override here. applyWatermark PRESERVES the source
// format (watermarkService.js: png -> png, webp -> webp, else jpeg),
// and its input is this preview — so the output format matches the key
// the header was already derived from. Forcing image/jpeg would
// mislabel a watermarked WebP preview, and `nosniff` means the browser
// will not correct it.
//
// What is still lost is the animation: the compositor flattens a
// multi-frame source to one frame while keeping the WebP container.
// That is a separate problem and a much larger one.
const watermarkedBuffer = await withLocalCopy(previewPath, (localPath) =>
watermarkService.applyWatermark(localPath, watermarkSettings)
);
@@ -1877,7 +2229,7 @@ router.get('/:slug/preview/:photoId',
} else {
res.setHeader('Content-Length', stat.size);
const stream = await storage.get(previewPath);
stream.pipe(res);
pipeStreamToResponse(stream, res, { context: `preview for photo ${photoId}` });
}
} catch (error) {
logger.error('Error serving preview image:', {
+8 -1
View File
@@ -367,7 +367,14 @@ router.get('/:slug/my-feedback',
const query = db('photo_feedback')
.join('photos', 'photo_feedback.photo_id', 'photos.id')
.where('photo_feedback.event_id', event.id);
.where('photo_feedback.event_id', event.id)
// Hidden rows are absent for the guest who left them too (#1150). In
// guest identity mode GalleryView builds its Liked/Favorited/Rated
// chips and their filters from THIS array rather than from is_liked,
// so without this a hidden like left an empty heart while the Liked
// chip still counted it and still surfaced the photo. Unapproved rows
// stay: a comment in the moderation queue is still the guest's own.
.where('photo_feedback.is_hidden', false);
// Prefer guest_id lookup when a verified guest token is present
// (per-person identity). Fall back to the device hash otherwise.
+3 -2
View File
@@ -1,4 +1,5 @@
const express = require('express');
const { resolvePhotoContentType } = require('../utils/photoContentType');
const { db } = require('../database/db');
const { formatBoolean } = require('../utils/dbCompat');
const { verifyGalleryAccess } = require('../middleware/gallery');
@@ -165,7 +166,7 @@ router.get('/:slug/photo/:photoId/view', verifyGalleryAccess, async (req, res) =
// Set security headers
res.set({
'Content-Type': photo.mime_type || 'image/jpeg',
'Content-Type': resolvePhotoContentType(photo),
'Content-Length': finalImage.length,
'Cache-Control': 'private, no-cache, no-store, must-revalidate',
'Pragma': 'no-cache',
@@ -335,7 +336,7 @@ router.get('/:slug/photo/:photoId/signed/:token', async (req, res) => {
// Set appropriate headers
res.set({
'Content-Type': photo.mime_type || 'image/jpeg',
'Content-Type': resolvePhotoContentType(photo),
'Content-Length': imageBuffer.length,
'Cache-Control': 'private, max-age=3600',
'X-Content-Type-Options': 'nosniff'
+3 -2
View File
@@ -1,4 +1,5 @@
const express = require('express');
const { resolvePhotoContentType } = require('../utils/photoContentType');
const { db } = require('../database/db');
const { verifyGalleryAccess, denySlideshowToken } = require('../middleware/gallery');
const secureImageService = require('../services/secureImageService');
@@ -236,7 +237,7 @@ router.get('/:slug/secure/:photoId/:token',
// Set content type and security headers
res.set({
'Content-Type': photo.mime_type || 'image/jpeg',
'Content-Type': resolvePhotoContentType(photo),
'Content-Length': processedImage.length,
'X-Protection-Level': protectionSettings.protectionLevel,
'X-Remaining-Uses': tokenValidation.remaining
@@ -425,7 +426,7 @@ router.get('/:slug/secure-download/:photoId/:token',
const downloadName = pickRawDownloadName(photo, useOriginal);
res.set({
'Content-Type': photo.mime_type || 'image/jpeg',
'Content-Type': resolvePhotoContentType(photo),
'Content-Disposition': buildContentDisposition(downloadName),
'Content-Length': fileBuffer.length,
'X-Download-Protected': 'true'
+5 -3
View File
@@ -7,6 +7,8 @@
// rate-limited at the mount point in server.js (authRateLimiter).
const express = require('express');
const { body, validationResult } = require('express-validator');
const { safeValidationErrors } = require('../utils/routeHelpers');
const { MAX_PASSWORD_LENGTH } = require('../utils/passwordValidation');
const setupService = require('../services/setupService');
const { getClientIp } = require('../utils/requestIp');
const { setAdminAuthCookie } = require('../utils/tokenUtils');
@@ -31,7 +33,7 @@ router.post('/verify-token', [
], async (req, res) => {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
try {
const valid = await setupService.verifySetupToken(req.body.token);
@@ -51,11 +53,11 @@ router.post('/verify-token', [
router.post('/admin', [
body('token').notEmpty().withMessage('Setup token is required'),
body('email').isEmail().withMessage('A valid email is required'),
body('password').notEmpty().withMessage('Password is required'),
body('password').isString().notEmpty().isLength({ max: MAX_PASSWORD_LENGTH }).withMessage('Password is required'),
], async (req, res) => {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
return res.status(400).json({ errors: safeValidationErrors(errors) });
}
try {
const { token, email, password } = req.body;
+4 -3
View File
@@ -18,6 +18,7 @@ const crypto = require('crypto');
const multer = require('multer');
const sharp = require('sharp');
const { body, query, validationResult } = require('express-validator');
const { safeValidationErrors } = require('../../utils/routeHelpers');
const { db, logActivity } = require('../../database/db');
const { apiTokenAuth, requireApiScope } = require('../../middleware/apiTokenAuth');
const { requireEventOwnership, scopeEventsQuery } = require('../../middleware/ownership');
@@ -146,7 +147,7 @@ router.post(
async (req, res) => {
try {
const errors = validationResult(req);
if (!errors.isEmpty()) return res.status(400).json({ errors: errors.array() });
if (!errors.isEmpty()) return res.status(400).json({ errors: safeValidationErrors(errors) });
const {
event_name, event_type, event_date,
customer_name = null, customer_email = null, customer_phone = null,
@@ -609,8 +610,8 @@ router.post(
let height = null;
try {
const meta = await sharp(tempPath).metadata();
width = meta.width || null;
height = meta.height || null;
// Oriented, not raw — see imageProcessor.orientedDimensions (#1185).
({ width, height } = require('../../services/imageProcessor').orientedDimensions(meta));
} catch { /* non-fatal */ }
let thumbRel = null;
@@ -94,6 +94,13 @@ async function list(relativePath = '') {
return { path: relFromRoot, entries, canNavigateUp };
}
/**
* A path under an EVENT's configured base directory.
*
* Still the right resolver for anything that means "the folder this event was
* imported from" the import walk in particular. It is NOT how a photo's
* original is found any more: see resolveExternalPhotoPath (#1163).
*/
function resolveExternalPath(event, relpath) {
const root = getExternalMediaRoot();
const base = event?.external_path ? path.join(event.external_path) : '';
@@ -101,9 +108,29 @@ function resolveExternalPath(event, relpath) {
return safePathJoin(root, combined);
}
/**
* A photo's original, from the root (#1163).
*
* photos.external_relpath used to be stored relative to events.external_path,
* which made a row's meaning depend on a column on another table that every
* import overwrites. Importing a second folder into an event therefore rebased
* every row already in it silently, because thumbnails are written to local
* storage during the import and the grid keeps rendering. One reporter had
* 7547 of 8004 rows resolving to files that did not exist.
*
* Root-relative makes a row self-describing: nothing an admin does to the event
* afterwards can move an already-imported photo. Migration 177 rewrote the
* existing rows.
*/
function resolveExternalPhotoPath(photo) {
const root = getExternalMediaRoot();
return safePathJoin(root, photo?.external_relpath || '');
}
module.exports = {
getExternalMediaRoot,
isUnderRoot,
list,
resolveExternalPath,
resolveExternalPhotoPath,
};
+389
View File
@@ -0,0 +1,389 @@
/**
* One row per external file per event (#1162).
*
* The import route used to check for an existing external_relpath and then
* insert, with an fs.stat and a Sharp decode in between wide enough that two
* overlapping imports both walked through it. A reporter's event held 8004 rows
* for 6012 distinct paths.
*
* This lives in a service rather than inside migration 176 because it has two
* callers. The migration is one. The other is a .picpeak restore: the archive
* carries the photos table verbatim, so a backup taken before this fix lands
* duplicate rows into a schema that now has a unique index on them and
* neither Postgres' `session_replication_role = replica` nor SQLite's
* `defer_foreign_keys` disables a UNIQUE index, so batchInsert would abort the
* whole restore after every table had already been emptied.
*
* DELETING dependent rows explicitly, rather than trusting ON DELETE CASCADE,
* is the load-bearing part. Every FK into photos declares CASCADE, but PicPeak
* does not set `PRAGMA foreign_keys = ON` the codebase says so in as many
* words where it deletes an event (adminEvents/helpers.js:245-249) so on
* every SQLite install the cascade is inert and a bare delete would leave
* dangling face embeddings, feedback and marks behind. The same reason
* `hero_photo_id` is repointed by hand: its SET NULL is inert there too, so
* without it a SQLite install keeps a hero pointing at a row that is gone.
*
* Guest and admin state is MOVED to the survivor where it can be, not
* discarded. The duplicates were separate tiles in the grid, so a guest's
* comment or an admin's rating could legitimately be attached to either, and
* silently deleting it inside a fix for silent data loss would be its own bug.
* Where the target already holds an equivalent row the same guest's like on
* the same photo, the same admin's mark, the same transfer's entry the loser
* is dropped instead, because those tables mean "one per (photo, actor)" and
* moving would either violate a unique constraint or double-count.
*
* photo_faces is the deliberate exception: both rows were scanned
* independently, so the survivor already has its own embeddings and moving the
* duplicate's would fabricate a second copy of every face and split the
* person clusters built from them.
*/
const { isUniqueViolation } = require('../utils/dbErrors');
const CHUNK = 400; // SQLite caps a statement at 999 bound parameters.
// Joins the parts of an equivalence key. Escaped, not a literal: a raw NUL in
// the source makes git classify this whole file as binary and hide its diffs.
const KEY_SEP = '\u0000';
const INDEX_NAME = 'photos_event_external_relpath_uniq';
const chunked = (arr) => {
const out = [];
for (let i = 0; i < arr.length; i += CHUNK) out.push(arr.slice(i, i + CHUNK));
return out;
};
/** Pure log rows — nothing is lost by dropping them with the duplicate. */
const LOG_TABLES = [
['image_access_logs', 'photo_id'],
['transfer_downloads', 'photo_id'],
];
/**
* Tables holding one row per (photo, actor). `keys` is what makes two rows
* equivalent, so a move that would collide becomes a delete instead.
*/
const MOVE_TABLES = [
{
table: 'photo_feedback',
// Guest identity, the way feedbackService defines it: guest_id when the
// gallery uses per-person guests (migration 078), guest_identifier
// otherwise — the same COALESCE its own duplicate-check and stats
// aggregate use (feedbackService.js:208, :559). Keying on
// guest_identifier alone would treat two DIFFERENT people sharing a
// device as one and delete one of their ratings.
identity: (row) => (row.guest_id != null ? `id:${row.guest_id}` : `anon:${row.guest_identifier}`),
// is_hidden is part of the identity, not noise: feedbackService lets a
// moderator-hidden row coexist with the guest's visible replacement and
// excludes hidden rows from the counts. Without it the visible row is
// dropped as redundant against the hidden one.
keys: ['feedback_type', 'is_hidden'],
// A comment is distinct content, never a per-guest toggle: two comments
// from one guest are two comments, so they always move.
alwaysMove: (row) => row.feedback_type === 'comment',
},
{
table: 'photo_admin_marks',
keys: ['admin_id'],
// rating and color_label are independently writable, so the same admin can
// have rated one tile and colour-labelled the other. Dropping the loser
// outright would lose a half the survivor's row has no value for.
mergeFields: ['rating', 'color_label'],
},
{ table: 'transfer_files', keys: ['transfer_id'] },
];
const equivalenceKey = (spec, row) => [
spec.identity ? spec.identity(row) : '',
...spec.keys.map((k) => row[k]),
].join(KEY_SEP);
async function repointColumn(knex, table, column, doomedToSurvivor) {
if (!(await knex.schema.hasTable(table))) return;
if (!(await knex.schema.hasColumn(table, column))) return;
for (const [doomed, survivor] of doomedToSurvivor) {
await knex(table).where(column, doomed).update({ [column]: survivor });
}
}
async function moveOrDrop(knex, spec, doomedToSurvivor, touched) {
if (!(await knex.schema.hasTable(spec.table))) return;
for (const [doomed, survivor] of doomedToSurvivor) {
const rows = await knex(spec.table).where('photo_id', doomed);
if (!rows.length) continue;
if (touched) touched.add(survivor);
const existing = await knex(spec.table).where('photo_id', survivor);
const taken = new Set(existing.map((r) => equivalenceKey(spec, r)));
for (const row of rows) {
const key = equivalenceKey(spec, row);
const move = (spec.alwaysMove && spec.alwaysMove(row)) || !taken.has(key);
if (!move) {
// Before dropping the loser, hand over any field the winner has no
// value for — otherwise an independently-set half goes with it.
if (spec.mergeFields) {
const winner = existing.find((r) => equivalenceKey(spec, r) === key);
const fill = {};
for (const field of spec.mergeFields) {
if (winner && winner[field] == null && row[field] != null) fill[field] = row[field];
}
if (winner && Object.keys(fill).length) {
await knex(spec.table).where('id', winner.id).update(fill);
Object.assign(winner, fill);
}
}
await knex(spec.table).where('id', row.id).del();
continue;
}
try {
await knex(spec.table).where('id', row.id).update({ photo_id: survivor });
taken.add(key);
} catch (err) {
// A unique constraint we did not model. The row is redundant with one
// the survivor already has, so dropping it is correct — but anything
// else must surface rather than leave a dangling photo_id behind.
if (!isUniqueViolation(err)) throw err;
await knex(spec.table).where('id', row.id).del();
}
}
}
}
/**
* Remove every photo row in `doomedToSurvivor`, moving or dropping the state
* that hangs off it first. Safe on both engines and on schemas that predate
* any of the dependent tables.
*/
async function deleteDuplicatePhotos(knex, doomedToSurvivor) {
const doomed = [...doomedToSurvivor.keys()];
if (!doomed.length) return 0;
// SET NULL is as inert as CASCADE on SQLite, so an event whose hero happened
// to be the duplicate would silently lose its hero image.
await repointColumn(knex, 'events', 'hero_photo_id', doomedToSurvivor);
await repointColumn(knex, 'photo_categories', 'hero_photo_id', doomedToSurvivor);
const feedbackTouched = new Set();
for (const spec of MOVE_TABLES) {
await moveOrDrop(knex, spec, doomedToSurvivor, spec.table === 'photo_feedback' ? feedbackTouched : null);
}
// photos carries denormalized feedback totals (migration 033:
// feedback_count, like_count, average_rating, favorite_count, and later
// reaction/colour counts). Reparenting rows without recomputing leaves a
// survivor that now OWNS feedback still rendering zero.
if (feedbackTouched.size && await knex.schema.hasColumn('photos', 'feedback_count')) {
const feedbackService = require('./feedbackService');
for (const survivor of feedbackTouched) {
await feedbackService.updatePhotoFeedbackStats(survivor, knex);
}
}
for (const [table, column] of LOG_TABLES) {
if (!(await knex.schema.hasTable(table))) continue;
for (const ids of chunked(doomed)) await knex(table).whereIn(column, ids).del();
}
// Both rows were scanned, so the survivor has its own faces; moving the
// duplicate's would double every embedding and split the person clusters.
//
// Through purgePhotoFaces, not a raw delete: deleting the rows is only half
// of it. event_people counts and centroids are derived from the faces being
// removed, and #1132's separation snapshots hold a COPY of each side's
// centroid — so a bare delete leaves ghost or inflated people and vectors
// built from photos that no longer exist. faceProcessor says as much: it is
// "called from every photo-deletion path".
if (await knex.schema.hasTable('photo_faces')) {
// If the ONLY completed scan of this file belonged to the duplicate, the
// purge below takes the sole embeddings with it and nothing re-queues the
// survivor — it just silently stops having a face. Mark those for a
// rescan; the worker picks up 'pending' on its own.
const needsRescan = [];
for (const [doomedId, survivorId] of doomedToSurvivor) {
if (!(await knex('photo_faces').where('photo_id', doomedId).first())) continue;
if (!(await knex('photo_faces').where('photo_id', survivorId).first())) needsRescan.push(survivorId);
}
let purgePhotoFaces = null;
try {
({ purgePhotoFaces } = require('./faceProcessor'));
} catch (err) {
// Face detection is optional; an install without it still needs the rows
// gone so nothing dangles on SQLite.
purgePhotoFaces = null;
}
if (purgePhotoFaces) {
for (const id of doomed) await purgePhotoFaces(id, knex);
} else {
for (const ids of chunked(doomed)) await knex('photo_faces').whereIn('photo_id', ids).del();
}
if (needsRescan.length && await knex.schema.hasColumn('photos', 'face_status')) {
for (const ids of chunked(needsRescan)) {
await knex('photos').whereIn('id', ids).update({ face_status: 'pending' });
}
}
}
// Real interactions, recorded per row. Deleting the duplicate would quietly
// lower the engagement the admin grid shows for a photo people did view and
// download.
if (await knex.schema.hasColumn('photos', 'view_count')) {
for (const [doomedId, survivorId] of doomedToSurvivor) {
const from = await knex('photos').where('id', doomedId)
.select('view_count', 'download_count').first();
if (!from) continue;
const add = {};
if (from.view_count) add.view_count = knex.raw('COALESCE(view_count, 0) + ?', [from.view_count]);
if (from.download_count) add.download_count = knex.raw('COALESCE(download_count, 0) + ?', [from.download_count]);
if (Object.keys(add).length) await knex('photos').where('id', survivorId).update(add);
}
}
for (const ids of chunked(doomed)) await knex('photos').whereIn('id', ids).del();
// The pre-built "download everything" zip still contains the rows just
// removed. Every ordinary photo-deletion path calls
// downloadZipService.invalidate for this reason (adminPhotos.js:450 and
// friends) — but that service carries debounce timers and a regeneration
// queue, which is not something a migration should be starting. Clearing the
// columns is the durable half of what invalidate does: getZipInfo already
// treats a missing or absent record as a cache miss and rebuilds on the next
// request, so guests stop receiving an archive containing deleted duplicates.
//
// The stale object itself is left for the same reason the duplicates'
// thumbnails are — a migration is the wrong place to reach into storage,
// which may be S3.
if (await knex.schema.hasColumn('events', 'download_zip_path')) {
const affected = [...new Set([...doomedToSurvivor.values()])];
const eventIds = affected.length
? (await knex('photos').whereIn('id', affected).distinct('event_id')).map((r) => r.event_id)
: [];
for (const ids of chunked(eventIds)) {
await knex('events').whereIn('id', ids).update({
download_zip_path: null,
download_zip_generated_at: null,
});
}
}
return doomed.length;
}
/**
* Which rows are duplicates, and which one survives.
*
* Survivor: the lowest id that has a thumbnail_path, else the lowest id.
* Thumbnails are generated per row during import, so on a duplicated pair both
* usually have one and the tie-break never fires but an import killed
* mid-flight leaves rows without, and dropping the one that HAS the thumbnail
* would blank a grid tile for no reason.
*/
async function planDedupe(knex) {
const dupKeys = await knex('photos')
.whereNotNull('external_relpath')
.select('event_id')
.count('* as c')
.groupBy('event_id', 'external_relpath')
.havingRaw('count(*) > 1');
const doomedToSurvivor = new Map();
for (const eventId of new Set(dupKeys.map((r) => r.event_id))) {
const rows = await knex('photos')
.where('event_id', eventId)
.whereNotNull('external_relpath')
.select('id', 'external_relpath', 'thumbnail_path')
.orderBy('id', 'asc');
const byPath = new Map();
for (const row of rows) {
const group = byPath.get(row.external_relpath);
if (group) group.push(row);
else byPath.set(row.external_relpath, [row]);
}
for (const group of byPath.values()) {
if (group.length < 2) continue;
const survivor = group.find((r) => r.thumbnail_path) || group[0];
for (const row of group) {
if (row.id !== survivor.id) doomedToSurvivor.set(row.id, survivor.id);
}
}
}
return doomedToSurvivor;
}
/** @returns {Promise<number>} how many duplicate rows were removed. */
async function dedupeExternalPhotos(knex) {
if (!(await knex.schema.hasTable('photos'))) return 0;
if (!(await knex.schema.hasColumn('photos', 'external_relpath'))) return 0;
return deleteDuplicatePhotos(knex, await planDedupe(knex));
}
/** Is the index actually there? Asked of the catalog, not inferred. */
async function externalRelpathIndexExists(knex) {
const isPg = knex.client && knex.client.config && knex.client.config.client === 'pg';
const row = isPg
? await knex('pg_indexes').where('indexname', INDEX_NAME).first()
: await knex('sqlite_master').where({ type: 'index', name: INDEX_NAME }).first();
return !!row;
}
/**
* The error a failed index MUST raise.
*
* Deliberately carries no `code`. run-migrations-safe.js treats 23505, 42P07,
* 42701 and 42710 as "schema already exists" and marks the migration applied
* (run-migrations-safe.js:138) and a CREATE UNIQUE INDEX that finds
* duplicate rows raises exactly 23505 on Postgres. Letting the driver's error
* through would therefore record 176 as done on an install that never got the
* index, with nothing to trigger a retry: the precise outcome the throw
* exists to prevent.
*/
function indexFailure(detail) {
return new Error(
`Could not create ${INDEX_NAME}: ${detail}. The photos table still holds `
+ 'duplicate (event_id, external_relpath) rows — most likely inserted by a '
+ 'concurrent import while this migration ran. Stop other writers and re-run.'
);
}
/**
* Partial, so the managed rows which all carry NULL are not indexed at
* all. Both engines treat NULLs as distinct in a unique index, so a plain one
* would also be correct, but it would carry every managed photo for no query
* that ever uses it.
*/
async function createExternalRelpathIndex(knex) {
try {
await knex.raw(
`CREATE UNIQUE INDEX IF NOT EXISTS ${INDEX_NAME} `
+ 'ON photos (event_id, external_relpath) WHERE external_relpath IS NOT NULL'
);
} catch (err) {
throw indexFailure(err.message);
}
// IF NOT EXISTS makes the statement itself a poor witness, and a replica
// inserting a duplicate between the dedupe and this lock is a real rolling-
// deploy shape. Ask the catalog.
if (!(await externalRelpathIndexExists(knex))) {
throw indexFailure('the index is absent afterwards');
}
}
async function dropExternalRelpathIndex(knex) {
await knex.raw(`DROP INDEX IF EXISTS ${INDEX_NAME}`);
}
module.exports = {
dedupeExternalPhotos,
externalRelpathIndexExists,
deleteDuplicatePhotos,
planDedupe,
createExternalRelpathIndex,
dropExternalRelpathIndex,
INDEX_NAME,
};
+304
View File
@@ -0,0 +1,304 @@
/**
* Fold each event's base path into its external photo rows (#1163).
*
* photos.external_relpath used to be stored relative to events.external_path
* a column every import overwrites so importing a second folder into an
* event rebased every photo already in it. Root-relative paths make a row
* self-describing.
*
* This lives in a service rather than inside migration 177 because it has two
* callers. The migration is one. The other is a .picpeak restore: knex_migrations
* is excluded from the archive, so restoring a pre-#1163 backup onto an
* already-migrated instance drops base-relative rows into a schema that no
* longer folds them, and every original in the restored library becomes
* unreachable with nothing logged.
*
* REPAIR, and its limits. For a healthy event the correct new value is just
* `external_path + relpath` that is what the app resolves today, so folding
* it in changes nothing and can break nothing. For an event that has ALREADY
* been rebased, that same rule would bake the broken path in permanently, so
* where the current resolution does not exist on disk this walks up the base
* path looking for an ancestor under which the file IS there (the shape the
* bug produces: a parent imported first, a child second). A row it cannot
* place is left resolving exactly where it resolves today preserving current
* behaviour is the floor, never guess below it.
*
* Existence alone is NOT enough to accept an ancestor. A row whose file an
* admin simply deleted would otherwise adopt any same-named file further up
* base `Trip/Sub`, relpath `photo.jpg`, an unrelated `Trip/photo.jpg` and
* downloads would then serve the WRONG original, which is worse than a broken
* link. So an ancestor candidate must also match photos.size_bytes, recorded
* by the import from the very file the row describes. Rows carrying no size
* are never repaired from an ancestor.
*
* ATOMICITY. Probing is read-only and runs first; every rewrite and the marker
* are then committed in ONE transaction. Split across commits, a process
* killed mid-fold would leave converted and unconverted rows behind with no
* marker, and the next run would fold the converted ones a second time
* putting every original one directory deeper, permanently.
*
* The probe is skipped entirely when the media root is unreachable or empty:
* an unmounted share makes every file look missing, and "repairing" off that
* signal would move every original on a healthy install. When it does run it
* is one access() per photo, base path first, so a healthy install pays one
* stat per row and then a single UPDATE per event.
*
* Idempotency is recorded explicitly in app_settings rather than inferred from
* the data. The tempting inference "does the relpath already start with the
* base path?" is wrong for any event with a subfolder named after its parent
* (base 'Trip', row 'Trip/x.jpg'), and being wrong there corrupts a path in an
* operation that has no undo.
*/
const path = require('path');
const fsp = require('fs').promises;
const { deleteDuplicatePhotos } = require('./externalPhotoDedupe');
const MARKER = 'external_relpath_root_relative';
// Per-row parking value for the two-pass rewrite below.
//
// NOT a NUL-prefixed string, which is what this was first written as: Postgres
// rejects U+0000 in a `text` column outright ("invalid byte sequence for
// encoding UTF8"), so the rewrite would abort on exactly the installs that
// need the two-pass repair — and only on Postgres, which SQLite-only tests
// cannot see. The prefix below is ordinary text, cannot collide with a real
// relative path (no import writes a leading dot-segment like this), and stays
// obviously wrong if a crash ever leaves one behind.
const STAGING_PREFIX = '.picpeak-fold-staging/';
const CHUNK = 400; // SQLite caps a statement at 999 bound parameters.
const chunk = (arr) => {
const out = [];
for (let i = 0; i < arr.length; i += CHUNK) out.push(arr.slice(i, i + CHUNK));
return out;
};
function normalizeBase(externalPath) {
return String(externalPath || '').replace(/\\/g, '/').replace(/^\/+|\/+$/g, '');
}
/** Every prefix of `base`, longest first, then '' (the root itself). */
function ancestorPrefixes(base) {
const segs = base.split('/').filter(Boolean);
const out = [];
for (let i = segs.length; i > 0; i--) out.push(segs.slice(0, i).join('/'));
out.push('');
return out;
}
async function exists(p) {
try { await fsp.access(p); return true; } catch { return false; }
}
/**
* Is `p` plausibly the file this row was imported from?
*
* Size is the provenance signal available without re-reading every original:
* photos.size_bytes was written by the import from the file the row describes.
* Returns false when it cannot be verified, so an unverifiable candidate is
* never adopted from somewhere the row did not previously point.
*/
async function fileMatchesSize(p, expectedSize) {
if (expectedSize == null || Number(expectedSize) <= 0) return false;
try {
const stats = await fsp.stat(p);
return stats.isFile() && stats.size === Number(expectedSize);
} catch {
return false;
}
}
/**
* Joined without the safePathJoin the app serves through: this is reading, and
* a stored path that escapes the root is damage worth detecting rather than
* throwing on.
*/
const under = (root, ...parts) => path.join(root, ...parts.filter(Boolean));
async function rootUsable(root) {
if (!root) return false;
try {
// An unmounted NFS/SMB share usually leaves the mountpoint behind as an
// ordinary empty directory, so readdir succeeds on storage that is gone.
return (await fsp.readdir(root)).length > 0;
} catch {
return false;
}
}
/**
* @param {import('knex')} knex a knex instance, or a transaction from a caller
* that is already inside one (the restore).
* @param {(msg: string) => void} [log]
* @returns {Promise<{skipped?: string, folded?: number, repaired?: number, stranded?: number, collided?: number}>}
*/
async function foldExternalRelpaths(knex, log = () => {}) {
if (!(await knex.schema.hasTable('photos'))) return { skipped: 'no photos table' };
if (!(await knex.schema.hasColumn('photos', 'external_relpath'))) return { skipped: 'no external_relpath column' };
if (!(await knex.schema.hasTable('events'))) return { skipped: 'no events table' };
if (!(await knex.schema.hasTable('app_settings'))) return { skipped: 'no app_settings table' };
if (await knex('app_settings').where('setting_key', MARKER).first()) {
return { skipped: 'already folded' };
}
const events = await knex('events').select('id', 'external_path');
const byId = new Map(events.map((e) => [e.id, normalizeBase(e.external_path)]));
const eventRows = await knex('photos').whereNotNull('external_relpath').distinct('event_id');
let root = null;
try {
root = require('./externalMediaService').getExternalMediaRoot();
} catch (e) {
log(`media root unavailable (${e.message}) — folding without on-disk repair`);
}
const canProbe = await rootUsable(root);
if (!canProbe) log('media root unreachable or empty — folding base paths in without on-disk repair');
// ---- Phase 1: decide, writing nothing. -------------------------------
// Read-only, so the transaction below stays short. Probing a cold NAS can
// take minutes and holding a write transaction open for that would block the
// app for the duration.
let folded = 0; let repaired = 0; let stranded = 0; let collided = 0;
const plan = [];
for (const { event_id: eventId } of eventRows) {
// No base path means the rows are already relative to the root.
const base = byId.get(eventId);
if (!base) continue;
const rows = await knex('photos')
.where('event_id', eventId)
.whereNotNull('external_relpath')
.select('id', 'external_relpath', 'size_bytes');
// Deciding health from a SAMPLE was the tempting shortcut and it is not
// safe: a rebased event whose first few rows happen to come from the most
// recent import reads as healthy, and every older row is baked in wrong.
const prefixes = ancestorPrefixes(base);
const placements = [];
let allUnderBase = true;
for (const row of rows) {
if (!canProbe) { placements.push([row, base]); continue; }
let chosen = null;
// The current base first, on existence alone — nothing is inferred
// there, it is where the row already resolves.
if (await exists(under(root, base, row.external_relpath))) {
chosen = base;
} else {
// Anywhere else has to prove itself: the name AND the size the import
// recorded. Without that a row whose file an admin deleted would adopt
// an unrelated same-named file one directory up, and downloads would
// serve the wrong original.
for (const prefix of prefixes) {
if (prefix === base) continue;
if (await fileMatchesSize(under(root, prefix, row.external_relpath), row.size_bytes)) {
chosen = prefix;
break;
}
}
}
if (chosen === null) { chosen = base; stranded++; }
if (chosen !== base) allUnderBase = false;
placements.push([row, chosen]);
}
if (allUnderBase) {
folded += rows.length;
plan.push({ eventId, base, bulk: true, rows: [], ids: rows.map((r) => r.id) });
continue;
}
// Two rows can now target the same path — the same file imported under two
// different bases really is one file. Resolved HERE rather than by letting
// the write fail: a caught write error cannot tell a genuine duplicate from
// a lock or I/O fault, and continuing past one would certify a partial
// conversion by writing the marker anyway.
//
// The loser is DELETED, not skipped. Skipping leaves it holding a
// base-relative path that the root-only resolver then reads as
// `<root>/<relpath>` — permanently pointing at the wrong place, or
// nowhere, with the marker saying the conversion is done. And it is a
// duplicate by construction: two rows that resolve to one file is exactly
// what migration 176 removes, so it goes through the same helper, which
// reparents the feedback and marks and reconciles the face clusters.
const claimed = new Map();
const resolved = [];
const losers = new Map();
for (const [row, chosen] of placements) {
const next = chosen ? `${chosen}/${row.external_relpath}` : row.external_relpath;
const winner = claimed.get(next);
if (winner != null) { losers.set(row.id, winner); collided++; continue; }
claimed.set(next, row.id);
if (chosen === base) folded++; else repaired++;
resolved.push([row.id, next]);
}
plan.push({ eventId, base, bulk: false, rows: resolved, losers });
}
// ---- Phase 2: write, all or nothing. ---------------------------------
// The marker rides in the same transaction as the rewrites, so there is no
// window where some rows are folded, the marker is absent, and a second run
// folds them again — which would put every original one directory deeper,
// permanently.
const apply = async (trx) => {
for (const step of plan) {
if (step.bulk) {
// BY ID, not by event. Phase 1 runs outside the transaction and can
// take minutes probing a cold mount; an import completing in that
// window inserts an already root-relative row, and `where event_id`
// would prefix it a second time with the stale base.
for (const ids of chunk(step.ids)) {
await trx('photos')
.whereIn('id', ids)
.whereNotNull('external_relpath')
.update({ external_relpath: trx.raw('? || external_relpath', [`${step.base}/`]) });
}
continue;
}
// Losers first: while they still hold their old path, the survivor has
// not taken the value they would collide with.
if (step.losers && step.losers.size) {
await deleteDuplicatePhotos(trx, step.losers);
}
// Two passes, through a per-row temporary value. The FINAL values are
// all distinct, but a final value can equal another row's CURRENT one —
// `photo.jpg` repairing to `Trip/photo.jpg` while the existing
// `Trip/photo.jpg` is still waiting to fold — so a single pass violates
// migration 176's unique index halfway through. And on Postgres that
// surfaces as 23505, which run-migrations-safe.js mistakes for "schema
// already exists" and records the migration as applied after the
// rollback, leaving every path unconverted with no retry.
for (const [id] of step.rows) {
await trx('photos').where('id', id).update({ external_relpath: `${STAGING_PREFIX}${id}` });
}
for (const [id, next] of step.rows) {
// No catch: collisions were resolved above, so anything failing here is
// a real fault and must roll the whole fold back rather than leave a
// half-converted table certified by the marker.
await trx('photos').where('id', id).update({ external_relpath: next });
}
}
await trx('app_settings').insert({
setting_key: MARKER,
setting_value: JSON.stringify(true),
setting_type: 'system',
updated_at: new Date().toISOString(),
});
};
// A caller already inside a transaction (the restore) passes its trx in as
// `knex`; opening a nested one would deadlock SQLite.
if (knex.isTransaction) await apply(knex);
else await knex.transaction(apply);
log(`${folded} folded, ${repaired} repaired, ${stranded} left unresolved, ${collided} skipped as duplicates`);
return { folded, repaired, stranded, collided };
}
module.exports = { foldExternalRelpaths, MARKER };
+59 -11
View File
@@ -124,7 +124,12 @@ class FeedbackService {
*/
async countGuestFeedback(eventId, feedbackType, guestId, guestIdentifier) {
const query = db('photo_feedback')
.where({ event_id: eventId, feedback_type: feedbackType });
// Hidden rows do not count against the guest's cap (#1150). They are
// absent everywhere else — the heart is empty, the tallies skip them,
// and submitFeedback now treats one as room for a fresh row. Counting
// them here would meet that fresh row with limit_reached and leave the
// control dead until the guest un-likes something they can still see.
.where({ event_id: eventId, feedback_type: feedbackType, is_hidden: false });
if (guestId) {
query.where('guest_id', guestId);
} else {
@@ -152,6 +157,13 @@ class FeedbackService {
photo_id: photoId,
event_id: eventId,
feedback_type,
// A hidden row is not there (#1150). Without this the guest saw an
// empty heart — every read surface treats hidden as absent — and
// clicking it found the hidden row and TOGGLED IT OFF, so the
// click appeared to do nothing and it took two more to get back to
// a filled heart. Skipping it makes the click create a fresh,
// visible row, which is what the guest is asking for.
is_hidden: false,
});
if (guest_id) {
duplicateQuery.where('guest_id', guest_id);
@@ -297,6 +309,11 @@ class FeedbackService {
const totalStats = await db('photo_feedback')
.where('event_id', eventId)
// Hidden rows do not count, the same rule the photo counters above
// already apply — without this the two halves of THIS response
// disagreed, and a hidden row preserved beside its replacement (#1150)
// is counted twice.
.where('is_hidden', false)
.select(
db.raw('COUNT(DISTINCT CASE WHEN feedback_type = ? THEN guest_identifier END) as unique_raters', ['rating']),
db.raw('COUNT(CASE WHEN feedback_type = ? THEN 1 END) as total_ratings', ['rating']),
@@ -317,25 +334,30 @@ class FeedbackService {
}
/**
* Update photo feedback statistics
* Update photo feedback statistics.
*
* `trx` so a caller running outside the request path the duplicate-photo
* dedupe (#1162), which reparents feedback rows and must leave the
* survivor's denormalized totals correct can recompute on its own
* connection.
*/
async updatePhotoFeedbackStats(photoId) {
async updatePhotoFeedbackStats(photoId, trx = db) {
try {
// Get aggregated stats
const stats = await db('photo_feedback')
const stats = await trx('photo_feedback')
.where('photo_id', photoId)
.where('is_hidden', false)
.select(
db.raw('COUNT(CASE WHEN feedback_type = ? AND is_approved = ? THEN 1 END) as comment_count', ['comment', formatBoolean(true)]),
db.raw('COUNT(CASE WHEN feedback_type = ? THEN 1 END) as like_count', ['like']),
db.raw('COUNT(CASE WHEN feedback_type = ? THEN 1 END) as favorite_count', ['favorite']),
db.raw('AVG(CASE WHEN feedback_type = ? THEN rating END) as average_rating', ['rating']),
db.raw('COUNT(DISTINCT COALESCE(CAST(guest_id AS VARCHAR), guest_identifier)) as feedback_count')
trx.raw('COUNT(CASE WHEN feedback_type = ? AND is_approved = ? THEN 1 END) as comment_count', ['comment', formatBoolean(true)]),
trx.raw('COUNT(CASE WHEN feedback_type = ? THEN 1 END) as like_count', ['like']),
trx.raw('COUNT(CASE WHEN feedback_type = ? THEN 1 END) as favorite_count', ['favorite']),
trx.raw('AVG(CASE WHEN feedback_type = ? THEN rating END) as average_rating', ['rating']),
trx.raw('COUNT(DISTINCT COALESCE(CAST(guest_id AS VARCHAR), guest_identifier)) as feedback_count')
)
.first();
// Update photo table
await db('photos')
await trx('photos')
.where('id', photoId)
.update({
feedback_count: stats.feedback_count || 0,
@@ -379,7 +401,33 @@ class FeedbackService {
await db('photo_feedback')
.where('id', feedbackId)
.update(updates);
// Unhiding can collide with a replacement (#1150). A hidden row reads as
// absent, so the guest may well have re-added the same feedback in the
// meantime; making the original visible again would leave TWO visible
// rows for one guest on one photo — double-counted in the tallies, and
// needing two toggles to clear because each one deletes a single row.
//
// Needs a stable identity to scope by. With neither id nor identifier
// the fallback degrades to `guest_identifier IS NULL`, which is every
// identifier-less row on the photo — other people's, deleted. Nothing to
// converge in that case, so leave it alone. Comments are exempt: several
// from one guest on one photo is normal.
const collapseIdentity = feedback.guest_id || feedback.guest_identifier;
if (updates.is_hidden === false && feedback.feedback_type !== 'comment' && collapseIdentity) {
const superseded = db('photo_feedback')
.where({
photo_id: feedback.photo_id,
event_id: feedback.event_id,
feedback_type: feedback.feedback_type,
is_hidden: false,
})
.whereNot('id', feedbackId);
if (feedback.guest_id) superseded.where('guest_id', feedback.guest_id);
else superseded.where('guest_identifier', feedback.guest_identifier);
await superseded.delete();
}
// Update photo stats if visibility changed
await this.updatePhotoFeedbackStats(feedback.photo_id);
+4 -2
View File
@@ -101,8 +101,10 @@ async function processNewPhoto(filePath) {
if (!isVideo) {
try {
const metadata = await sharp(filePath).metadata();
if (metadata.width && metadata.height) {
dimensions = { width: metadata.width, height: metadata.height };
// Oriented, not raw — see imageProcessor.orientedDimensions (#1185).
const dims = require('./imageProcessor').orientedDimensions(metadata);
if (dims.width && dims.height) {
dimensions = { width: dims.width, height: dims.height };
}
} catch (err) {
logger.debug(`Could not read image dimensions for ${filename}: ${err.message}`);
+12 -1
View File
@@ -168,8 +168,19 @@ async function formatEventDate(value) {
}
}
// Draft, archived and deactivated galleries are refused by /info; the OG
// preview must not leak their name, date and welcome message to crawlers.
function isPubliclyVisible(event) {
if (!event) return false;
const truthy = (v) => v === true || v === 1 || v === '1' || v === 'true';
if (truthy(event.is_draft) || truthy(event.is_archived)) return false;
if (event.is_active === false || event.is_active === 0 || event.is_active === '0') return false;
return true;
}
async function buildOgMetadata(slug, requestPath) {
const event = await resolveSlug(slug);
const resolved = await resolveSlug(slug);
const event = isPubliclyVisible(resolved) ? resolved : null;
const branding = await fetchBranding();
const base = frontendBase();
const siteName = branding.companyName || 'PicPeak';
+174 -25
View File
@@ -123,6 +123,31 @@ const contentTypeFor = (format) => {
* import path passes a per-photo unique basename so two events both
* referencing `IMG_0001.jpg` don't clobber each other's thumbnail.
*/
/**
* The dimensions a viewer actually sees, given EXIF orientation (#1185).
*
* sharp reports `metadata.width`/`height` as the pixels are stored, not as
* they are displayed. Orientation values 5-8 carry a 90° rotation, so for
* those the two are swapped which is why a portrait photo from a body that
* tags rather than rotates was landing in the database as landscape, and why
* masonry and justified layouts sized its tile with the wrong aspect ratio on
* top of the image itself being unrotated.
*
* Everything that renders these photos now applies `.rotate()`, so the stored
* numbers have to describe the rotated result to match.
*
* @param {Object} metadata - a sharp metadata object
* @returns {{ width: number|null, height: number|null }}
*/
function orientedDimensions(metadata) {
if (!metadata || !metadata.width || !metadata.height) return { width: null, height: null };
const swap = metadata.orientation >= 5 && metadata.orientation <= 8;
return {
width: swap ? metadata.height : metadata.width,
height: swap ? metadata.width : metadata.height,
};
}
async function generateThumbnail(imagePath, options = {}) {
const sourceBasename = path.basename(imagePath);
const outputBasename = options.outputBasename || sourceBasename;
@@ -133,10 +158,18 @@ async function generateThumbnail(imagePath, options = {}) {
// Get thumbnail settings
const settings = await getThumbnailSettings();
// Force regeneration: drop the existing object before writing the new one
if (options.regenerate) {
await storage.delete(thumbnailRelKey).catch(() => {});
}
// `options.regenerate` deliberately does NOT delete the existing object first
// (#1129).
//
// It used to, and the delete ran BEFORE sharp had even opened the source — so
// a source that could not be read (a NAS mount that blipped, a corrupt file)
// left the old thumbnail already gone and returned null, with the database
// still pointing at it. One bulk regeneration during a mount outage could
// therefore strip every canonical thumbnail in a reference gallery.
//
// Nothing is lost by dropping it: LocalFsStorage.put stages to a temp file and
// renames over the target, which replaces atomically, and an S3 put overwrites
// by key. The delete only added a window with no thumbnail at all.
try {
// First, verify the source image is complete and valid
@@ -152,6 +185,18 @@ async function generateThumbnail(imagePath, options = {}) {
failOn: 'none'
});
// Apply EXIF orientation before resizing (#1185). Without this a photo
// whose Orientation tag is not 1 — routine for portrait shots on bodies
// that tag rather than rotate the sensor data — is resized from the raw
// pixels and comes out sideways. `.withMetadata(false)` below then strips
// the tag, so the browser has no hint left to correct it either.
//
// Unconditional: this pipeline never passes `animated: true`, so it
// already flattens a multi-frame source. Guarding on `pages` would protect
// an animation that was being discarded anyway while leaving the output in
// raw orientation against corrected dimensions.
sharpInstance = sharpInstance.rotate();
// Strip EXIF/metadata from thumbnails (privacy: prevent GPS leak etc.)
sharpInstance = sharpInstance.withMetadata(false);
@@ -192,9 +237,12 @@ async function generateThumbnail(imagePath, options = {}) {
const msg = (error && error.message) ? error.message : String(error);
logger.error(`Failed to generate thumbnail for ${sourceBasename}: ${msg}`);
// Clean up any partially uploaded object
await storage.delete(thumbnailRelKey).catch(() => {});
// No cleanup delete here either, for the same reason (#1129). This was
// "clean up any partially uploaded object", but there cannot be one:
// storage.put is the LAST statement in the try, every throw above it
// happens before anything is written, and put unlinks its own temp file on
// failure. The only object this could remove is the PREVIOUS, valid
// rendition — exactly the thumbnail a failed regeneration must leave alone.
return null;
}
}
@@ -366,7 +414,10 @@ async function generateVideoPlaceholder(originalFilename, options = {}) {
* Outputs a 1920x1080 image suitable for full-width hero sections
*/
async function generateHeroImage(imagePath, options = {}) {
const filename = path.basename(imagePath);
// outputBasename lets callers disambiguate sources that share a basename —
// two events referencing the same NAS filename would otherwise clobber each
// other's hero. Same contract as generateThumbnail and generatePreviewImage.
const filename = options.outputBasename || path.basename(imagePath);
const heroFilename = `hero_${filename}`;
const heroRelKey = path.posix.join('heroes', heroFilename);
const storage = getStorage();
@@ -392,6 +443,10 @@ async function generateHeroImage(imagePath, options = {}) {
failOn: 'none'
});
// EXIF orientation, same reasoning as generateThumbnail (#1185) — and
// unconditional for the same reason: no `animated: true` on the input.
sharpInstance = sharpInstance.rotate();
// Strip EXIF/metadata from hero images (privacy: prevent GPS leak etc.)
sharpInstance = sharpInstance.withMetadata(false);
@@ -448,16 +503,13 @@ async function isHeroValid(heroPath) {
* Ensure a hero image exists for a photo, regenerate if needed
*/
async function ensureHeroImage(photo) {
const { resolvePhotoStorageKey } = require('./photoResolver');
const { resolvePhotoStorageKey, resolvePhotoFilePath } = require('./photoResolver');
let sourceKey;
let event;
try {
const event = await db('events').where('id', photo.event_id).first();
sourceKey = resolvePhotoStorageKey(event, photo);
logger.info(`Ensuring hero image for photo ${photo.id} from key: ${sourceKey}`);
event = await db('events').where('id', photo.event_id).first();
} catch (e) {
const msg = (e && e.message) ? e.message : String(e);
logger.error(`Failed to resolve original key for hero image (photo ${photo.id}): ${msg}`);
logger.error(`Failed to load event for hero image (photo ${photo.id}): ${e.message}`);
return null;
}
@@ -469,7 +521,56 @@ async function ensureHeroImage(photo) {
logger.warn(`Invalid hero image detected for photo ${photo.id}, regenerating...`);
}
const newHeroPath = await withLocalCopy(sourceKey, (localPath) =>
// External sources never reach the managed backend, so resolvePhotoStorageKey
// returns null for them by design — and this function used to feed that null
// straight to withLocalCopy, which throws, so the hero route fell back to
// redirecting at the full original. #1078 fixed exactly this for
// ensurePreviewImage and nobody carried it across; it only became visible
// when the Story hero started asking for hero_url instead of the original
// (#1166), which on a reference-mode gallery quietly changed nothing.
const heroIsExternal = photo.source_origin === 'external' || photo.source_origin === 'reference';
let newHeroPath;
if (heroIsExternal) {
// Mirrors the external branch in ensurePreviewImage: a direct fs read off
// the mount, so no withLocalCopy, and a per-photo outputBasename so two
// events referencing the same NAS basename cannot clobber each other.
let localPath;
try {
localPath = resolvePhotoFilePath(event, photo);
} catch (e) {
logger.error(`Failed to resolve external file for hero image (photo ${photo.id}): ${e.message}`);
return null;
}
const sourceBasename = path.basename(photo.external_relpath || photo.filename || `photo-${photo.id}`);
newHeroPath = await generateHeroImage(localPath, {
regenerate: true,
outputBasename: `ext${photo.id}_${sourceBasename}`,
});
if (newHeroPath) {
await db('photos').where({ id: photo.id }).update({ hero_path: newHeroPath });
}
return newHeroPath;
}
let sourceKey;
try {
sourceKey = resolvePhotoStorageKey(event, photo);
logger.info(`Ensuring hero image for photo ${photo.id} from key: ${sourceKey}`);
} catch (e) {
const msg = (e && e.message) ? e.message : String(e);
logger.error(`Failed to resolve original key for hero image (photo ${photo.id}): ${msg}`);
return null;
}
if (!sourceKey) {
// Reference-mode event holding a row with no source_origin: the mode falls
// back to the event's and resolvePhotoStorageKey returns null. Honour the
// null-on-failure contract rather than feeding it to withLocalCopy.
logger.warn(`No managed storage key for hero image (photo ${photo.id}); skipping generation`);
return null;
}
newHeroPath = await withLocalCopy(sourceKey, (localPath) =>
generateHeroImage(localPath, { regenerate: true })
);
@@ -496,22 +597,50 @@ async function ensureHeroImage(photo) {
* Output to `previews/preview_<filename>` so an admin who flips the
* setting back off can wipe the folder cleanly without touching
* thumbnails or heroes.
*
* ENCODING follows the source, it is not always JPEG. JPEG has no alpha
* channel and no second frame, so encoding everything as JPEG flattened a
* transparent PNG onto a solid background and reduced an animated GIF to its
* first frame for every consumer of this tier, not just the lightbox.
* Sources with alpha or more than one page are encoded as WebP instead, which
* carries both and is still far smaller than the original.
*
* The output extension is rewritten to match what was actually written.
* Previously the source basename was kept verbatim, so a PNG source produced
* `preview_foo.png` holding JPEG bytes harmless while the route hard-coded
* image/jpeg, and actively wrong now that the encoding varies. Old keys keep
* working: they are still JPEG and still served as such.
*/
async function generatePreviewImage(imagePath, options = {}) {
// outputBasename lets callers disambiguate sources that share a basename
// (external mounts, see ensurePreviewImage) — same contract as
// generateThumbnail.
const filename = options.outputBasename || path.basename(imagePath);
const previewFilename = `preview_${filename}`;
const previewRelKey = path.posix.join('previews', previewFilename);
const storage = getStorage();
// Probed BEFORE the key is built: the extension has to match the encoding,
// and the encoding depends on what the source turns out to be.
let probe;
try {
probe = await sharp(imagePath).metadata();
} catch (error) {
const msg = (error && error.message) ? error.message : String(error);
logger.error(`Failed to read metadata for preview of ${filename}: ${msg}`);
return null;
}
const isAnimated = (probe.pages || 1) > 1;
const needsWebp = isAnimated || probe.hasAlpha === true;
const base = filename.replace(/\.[^./\\]+$/, '');
const previewFilename = `preview_${base}.${needsWebp ? 'webp' : 'jpg'}`;
const previewRelKey = path.posix.join('previews', previewFilename);
if (options.regenerate) {
await storage.delete(previewRelKey).catch(() => {});
}
try {
const metadata = await sharp(imagePath).metadata();
const metadata = probe;
if (!metadata.width || !metadata.height) {
throw new Error('Invalid image metadata - file may be incomplete');
}
@@ -523,8 +652,27 @@ async function generatePreviewImage(imagePath, options = {}) {
limitInputPixels: 268402689, // ~16k x 16k max
sequentialRead: true,
failOn: 'none',
// Without this an animated source is opened as its first frame only, and
// every later frame is discarded before the resize ever sees it.
// limitInputPixels still applies, and sharp counts an animated input as
// width x (height x pages) — so a pathological GIF is rejected rather
// than decoded, and the caller falls back to the original.
animated: isAnimated,
});
// EXIF orientation (#1185). Guarded here and not in the thumbnail/hero
// generators because this one DOES open multi-frame sources with
// `animated: true` above, and `.rotate()` would flatten them to a single
// frame — trading an animation for an orientation.
//
// Which leaves one corner unsolved: a multi-frame source that also carries
// an orientation tag keeps its raw orientation here while the thumbnail
// and the stored dimensions describe the rotated one. GIF has no EXIF at
// all and animated WebP effectively never sets it.
if (!isAnimated) {
sharpInstance = sharpInstance.rotate();
}
// Strip EXIF — same privacy reasoning as thumbnails/heroes.
sharpInstance = sharpInstance.withMetadata(false);
@@ -536,18 +684,18 @@ async function generatePreviewImage(imagePath, options = {}) {
fit: 'inside',
});
sharpInstance = sharpInstance.jpeg({
quality,
progressive: true,
mozjpeg: true,
});
sharpInstance = needsWebp
? sharpInstance.webp({ quality })
: sharpInstance.jpeg({ quality, progressive: true, mozjpeg: true });
const buffer = await sharpInstance.toBuffer();
if (!buffer || buffer.length === 0) {
throw new Error('Generated preview image is empty');
}
await storage.put(previewRelKey, buffer, { contentType: 'image/jpeg' });
await storage.put(previewRelKey, buffer, {
contentType: needsWebp ? 'image/webp' : 'image/jpeg',
});
logger.info(`Generated preview image for ${filename}${previewRelKey}`);
return previewRelKey;
@@ -703,6 +851,7 @@ async function extractCaptureDate(imagePath) {
}
module.exports = {
orientedDimensions,
generateThumbnail,
isThumbnailValid,
ensureThumbnail,
+216
View File
@@ -0,0 +1,216 @@
/**
* What PicPeak actually occupies on this machine (#1164).
*
* The dashboard's "Storage used" tile summed photos.size_bytes, which is the
* catalogued size of the ORIGINALS a number with no relationship to the disk
* PicPeak runs on:
*
* - in reference mode the originals are never copied. Those bytes are on the
* NAS. The reporter's tile read ~80 GB against 21 GB of real local usage.
* - duplicate rows counted the same file twice (#1162).
* - it ignored everything PicPeak genuinely does write: thumbnails, previews,
* hero renditions, and the per-event download cache an 11.8 GB
* `.download-cache/all.zip` sat outside the figure entirely.
*
* So the one number an admin reaches for when asking "am I running out of
* disk" pointed away from the answer and omitted exactly the things filling
* the disk. This walks the storage root instead and reports what is there.
*
* Walking rather than summing DB columns is deliberate: thumbnail/preview/hero
* rows record a key, never a byte count, and orphans (a deleted event's
* leftovers, an interrupted import's thumbnails) are real bytes on a real
* disk. A `du` is the only honest answer, and the only one that notices what
* PicPeak has forgotten about.
*
* The external media root is EXCLUDED, and that is the whole point rather than
* a detail. Its compose default is `<storage>/external-media`, where the NAS is
* bind-mounted a plain directory, not a symlink so walking it would add
* every referenced original back into a figure that exists to leave them out,
* and compare NAS bytes against statfs() of the local disk. That is the
* over-count this replaces, reintroduced by the fix for it.
*
* Cached, because it is one stat per file. On a large install that is seconds,
* and the dashboard is polled and concurrent misses share one walk rather
* than each starting their own.
*/
const path = require('path');
const fsp = require('fs').promises;
const logger = require('../utils/logger');
const { getStoragePath } = require('../config/storage');
const TTL_MS = 5 * 60 * 1000;
let cache = null;
// The walk in flight, if any. Two admins loading the dashboard, or the sidebar
// and the storage tab on one page, hit the same cold cache and would otherwise
// each stat every file on the disk.
let inFlight = null;
/**
* The external media root, resolved, when it lies inside the storage root.
* Returns null when it is elsewhere (the usual production case) or cannot be
* resolved nothing to exclude then.
*/
function nestedExternalRoot(storageRoot) {
let externalRoot;
try {
externalRoot = require('./externalMediaService').getExternalMediaRoot();
} catch (err) {
return null;
}
if (!externalRoot) return null;
const resolvedExternal = path.resolve(externalRoot);
const resolvedStorage = path.resolve(storageRoot);
if (resolvedExternal === resolvedStorage) return null;
return resolvedExternal.startsWith(resolvedStorage + path.sep) ? resolvedExternal : null;
}
/**
* Which line of the breakdown a path belongs to.
*
* The names are the ones the writers actually use `heroes` from
* imageProcessor, `watermarks` from watermarkService, and so on rather than
* the ones the layout docs imply. Getting one wrong is not a crash, it is a
* silent 30 MB in "other", which is the least useful place for it to land.
*
* `.download-cache` is the case that needs the explicit check: it lives INSIDE
* an event directory, so the naive rule files an 11.8 GB zip as photography
* and it is the one bucket that is pure disposable cache, which makes it the
* one an admin most wants to see on its own.
*
* There is no external-media bucket: that subtree is not walked at all (see
* nestedExternalRoot). Those bytes live on the media share, and counting them
* is the exact over-count this measurement exists to end.
*/
function categorize(relPath) {
const segments = relPath.split(path.sep);
if (segments.includes('.download-cache')) return 'downloadCache';
switch (segments[0]) {
case 'thumbnails': return 'thumbnails';
case 'previews': return 'previews';
case 'heroes': return 'heroes';
case 'watermarks': return 'watermarks';
case 'uploads': return 'uploads';
case 'temp': return 'temp';
case 'business-docs': return 'businessDocs';
case 'events':
return segments[1] === 'archived' ? 'archives' : 'originals';
default:
return 'other';
}
}
const EMPTY_BREAKDOWN = () => ({
originals: 0,
archives: 0,
thumbnails: 0,
previews: 0,
heroes: 0,
watermarks: 0,
uploads: 0,
businessDocs: 0,
downloadCache: 0,
temp: 0,
other: 0,
});
async function walk(absDir, relDir, acc) {
// The media share, bind-mounted under the storage root. Walking it would put
// every referenced original back into a local-usage figure, and on a real
// NAS the traversal alone would take far longer than the measurement is
// worth.
if (acc.excludeRoot && path.resolve(absDir) === acc.excludeRoot) {
acc.excludedExternalRoot = acc.excludeRoot;
return;
}
let entries;
try {
entries = await fsp.readdir(absDir, { withFileTypes: true });
} catch (err) {
// A directory that is not there yet (a fresh install has no /previews) is
// not an error. Anything else is worth knowing about but must not abort
// the measurement — a partial number beats no number, and `partial` says
// so to the caller.
if (err.code !== 'ENOENT') {
acc.partial = true;
logger.debug?.(`localStorageUsage: skipped ${absDir}: ${err.message}`);
}
return;
}
for (const entry of entries) {
const abs = path.join(absDir, entry.name);
const rel = relDir ? path.join(relDir, entry.name) : entry.name;
// Symlinks are not followed: a link into the external media mount would
// otherwise add the NAS to the local total, which is the exact confusion
// this replaces.
if (entry.isDirectory()) {
await walk(abs, rel, acc);
} else if (entry.isFile()) {
try {
const stats = await fsp.stat(abs);
acc.total += stats.size;
acc.files += 1;
acc.breakdown[categorize(rel)] += stats.size;
} catch (err) {
// Raced with a delete, most likely. Nothing to add.
if (err.code !== 'ENOENT') acc.partial = true;
}
}
}
}
/**
* @param {{ force?: boolean }} [opts] force skips the TTL cache.
* @returns {Promise<{total:number, files:number, breakdown:object, partial:boolean, measuredAt:string, root:string}>}
*/
async function measureLocalStorageUsage(opts = {}) {
const now = Date.now();
if (!opts.force && cache && now - cache.at < TTL_MS) return cache.value;
// Share a walk already underway rather than starting a second one.
if (!opts.force && inFlight) return inFlight;
inFlight = runMeasurement()
.then((value) => { cache = { at: Date.now(), value }; return value; })
.finally(() => { inFlight = null; });
return inFlight;
}
async function runMeasurement() {
const root = getStoragePath();
const acc = {
total: 0,
files: 0,
breakdown: EMPTY_BREAKDOWN(),
partial: false,
excludeRoot: nestedExternalRoot(root),
excludedExternalRoot: null,
};
await walk(root, '', acc);
return {
total: acc.total,
files: acc.files,
breakdown: acc.breakdown,
partial: acc.partial,
// Set when the media share sits inside the storage root and was skipped,
// so the UI can say why the figure is smaller than `du` would report.
excludedExternalRoot: acc.excludedExternalRoot,
measuredAt: new Date().toISOString(),
root,
};
}
/** Test seam — the TTL cache would otherwise outlive a temp storage root. */
function resetLocalStorageUsageCache() {
cache = null;
inFlight = null;
}
module.exports = {
measureLocalStorageUsage,
resetLocalStorageUsageCache,
};
+178
View File
@@ -0,0 +1,178 @@
/**
* Shared run state for the photo maintenance sweeps (#1181).
*
* These jobs used to keep `{ isRunning, lastResult }` in a module-level
* variable. That is invisible to every other replica, so on a multi-replica
* install the status endpoint answers from whichever process the poll happens
* to reach and a second POST can start a duplicate pass over the whole
* library. Moving the state into the database makes both the claim and the
* reporting shared.
*
* The claim is a conditional UPDATE whose affected-row count is the answer
* the same shape backgroundProcessor uses to hand a photo to exactly one
* worker (backgroundProcessor.js:110-116). Two replicas issuing it
* concurrently cannot both match: the row is locked for the duration of each
* UPDATE, so the loser sees is_running already true and gets 0 rows back.
*
* A lease that can be taken over needs fencing, which is what claim_token is
* for. Taking over a stale claim does not stop the old runner it is a
* process nobody can signal, quite possibly still walking the library. So
* every write it attempts carries the token it was issued:
*
* - heartbeat() reports whether the renewal landed. It returns false once
* the claim has moved on, and the run loops treat that as "stop".
* - release() only clears the row if the token still matches, so a
* superseded runner finishing late cannot clear the new owner's flag or
* overwrite its result.
*
* Without both of those, a takeover produces two live runners and the loser
* ends up stomping the winner's state on its way out.
*
* Timestamps are written as ISO strings rather than Date objects. Production
* stores Dates fine, but inside jest the sqlite3 binding turns them into the
* literal string "[object Object]" (see CLAUDE.md), which would silently break
* every staleness comparison in the tests. ISO-8601 also compares correctly
* under SQLite's lexicographic text ordering, so the `<` below means the same
* thing on both engines.
*/
const os = require('os');
const crypto = require('crypto');
const { db } = require('../database/db');
const logger = require('../utils/logger');
const JOB_DIMENSION_REPAIR = 'photo_dimension_repair';
const JOB_CAPTURE_DATE_BACKFILL = 'photo_capture_date_backfill';
// How long a run may go without renewing its lease before another replica is
// allowed to take it over. Generous on purpose: these jobs walk the whole
// library and a single slow original on a stalled NAS mount can block the loop
// for a while. The cost of being too eager is a duplicate pass; the cost of
// being too patient is a button that stays disabled after a crash.
const DEFAULT_STALE_MS = 15 * 60 * 1000;
// How often a running job renews. Time-based, and comfortably inside the stale
// window: tying renewal to a photo counter meant a job whose photos were slow
// — a stalled mount, a handful of very large originals — could be declared
// abandoned while it was still working.
const HEARTBEAT_INTERVAL_MS = 60 * 1000;
const OWNER = `${os.hostname()}:${process.pid}`;
const nowIso = () => new Date().toISOString();
const cutoffIso = (staleAfterMs) => new Date(Date.now() - staleAfterMs).toISOString();
/**
* Try to become the one runner of `jobName`.
*
* Returns a claim token on success, or null when another replica holds it and
* is still renewing the caller should answer 409. The token must be passed
* to every subsequent heartbeat/release for this run.
*/
async function claim(jobName, { staleAfterMs = DEFAULT_STALE_MS } = {}) {
const stamp = nowIso();
const cutoff = cutoffIso(staleAfterMs);
const token = crypto.randomBytes(16).toString('hex');
const claimed = await db('maintenance_jobs')
.where({ job_name: jobName })
.where(function () {
// Free, or held by a run that has stopped renewing. heartbeat_at is
// always written by the claim below, so a running job cannot have a null
// heartbeat — no third case to handle here.
this.where('is_running', false).orWhere('heartbeat_at', '<', cutoff);
})
.update({
is_running: true,
started_at: stamp,
heartbeat_at: stamp,
finished_at: null,
owner: OWNER,
claim_token: token,
});
return claimed > 0 ? token : null;
}
/**
* Renew the lease.
*
* Returns false when this run no longer owns the claim it was declared stale
* and taken over. The caller must stop working at that point: the new owner is
* already walking the same rows, and two runners writing is exactly what the
* lock exists to prevent.
*/
async function heartbeat(jobName, token) {
try {
const renewed = await db('maintenance_jobs')
.where({ job_name: jobName, is_running: true, claim_token: token })
.update({ heartbeat_at: nowIso() });
return renewed > 0;
} catch (err) {
// A failed renewal query is not proof the claim is gone, and aborting a
// long sweep over one transient database blip is the worse trade. Say the
// claim still holds; if it really has moved on, the next renewal says so.
logger.warn(`maintenanceJobState: heartbeat failed for ${jobName}: ${err.message}`);
return true;
}
}
/**
* Give up the claim.
*
* Scoped to the token, so a runner that was superseded while it was working
* cannot clear the new owner's flag or overwrite its result on the way out.
* Returns false when the claim had already moved on.
*
* `result` is stored as the job's last outcome. Pass null (the "nothing to do"
* and error paths) to release without overwriting what the previous real run
* reported.
*/
async function release(jobName, token, result = null) {
const update = { is_running: false, finished_at: nowIso() };
if (result !== null && result !== undefined) {
update.last_result = JSON.stringify(result);
}
const released = await db('maintenance_jobs')
.where({ job_name: jobName, claim_token: token })
.update(update);
return released > 0;
}
/**
* Current state, in the shape the status endpoints hand to the frontend.
*
* A run whose lease has gone stale is reported as not running: the owning
* replica is gone, nothing is going to release the claim, and the operator
* needs the button back. The next claim() takes the row over on the same
* condition, so the two agree.
*/
async function read(jobName, { staleAfterMs = DEFAULT_STALE_MS } = {}) {
const row = await db('maintenance_jobs').where({ job_name: jobName }).first();
if (!row) return { isRunning: false, lastResult: null };
const alive = row.heartbeat_at && new Date(row.heartbeat_at).getTime() > Date.now() - staleAfterMs;
let lastResult = null;
if (row.last_result) {
try {
lastResult = JSON.parse(row.last_result);
} catch (err) {
// Never let a malformed row take the status endpoint down with it.
logger.warn(`maintenanceJobState: unreadable last_result for ${jobName}: ${err.message}`);
}
}
return { isRunning: Boolean(row.is_running) && Boolean(alive), lastResult };
}
module.exports = {
claim,
heartbeat,
release,
read,
JOB_DIMENSION_REPAIR,
JOB_CAPTURE_DATE_BACKFILL,
DEFAULT_STALE_MS,
HEARTBEAT_INTERVAL_MS,
};

Some files were not shown because too many files have changed in this diff Show More