1387 Commits

Author	SHA1	Message	Date
Paul Nothaft	5db0a76cce	chore(main): release 3.131.7-beta.0	2026-09-11 15:37:41 +02:00
Paul Nothaft	415497ad06	chore(main): release 3.131.6-beta.0 (#1420)	2026-09-11 11:42:49 +00:00
Paul Nothaft	98d25601b4	fix(gallery): cap how many cached zips rebuild at once in the background (#1418) ... invalidateAll() invalidates every event holding a cached zip, and each invalidate() arms its own debounce timer in the same tick — so they all fired together and every one of them started building at once. Each build opens its own storage reads, so flipping a global setting across 25 events was enough to exhaust the S3 agent pool and stall uploads, thumbnails and gallery reads until the burst drained. Nothing capped it. Background rebuilds now run two at a time. The cap is deliberately only on that path: a foreground generateZip — a guest actually waiting on a download — is never queued behind a settings-change burst, which would trade one stall for another. stop() drains anything parked for a slot, so shutdown cannot hang on a queue that will never move. This is the second half of the problem. The first half — an individual build opening one unbounded read per photo — is the storage-read guard that already landed for the guest download routes and the job builder; the cached-zip builder's own copy is still in an open PR. Relates to issue 1399 Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-11 13:36:27 +02:00
Paul Nothaft	2a04137a75	chore(main): release 3.131.5-beta.0 (#1419)	2026-09-11 11:34:31 +00:00
Trung-Tin Pham	f094cc06a7	fix(gallery): stop the pre-zip build leaking storage reads (#1402) ... Building the download-all archive opened one storage read per photo and handed each stream to archiver, which uses them one at a time. Every read past the one being written parked an S3 socket with a full receive buffer, and both early exits walked away from all of them: archiver's abort() does not touch its source streams, and the error path only removed the temp directory. Nothing else reclaims those sockets either, because the SDK arms its socket timeout on a 3s delay and clears it as soon as the response headers land. On a live server 43 of the 50 pooled sockets ended up stuck for days and photo uploads stopped completing, with nothing logged. The common way in is an ordinary upload. invalidate() runs on every photo upload, delete and bulk edit, and it aborted an in-flight build. Track every open read and destroy them on every exit path, cancel the in-flight build from invalidate() rather than waiting for the loop to reach its next version check, and cap reads in flight at 2. A build of 120 photos peaked at 50 concurrent GETs before, the whole agent pool, which starved uploads and thumbnails on its own. Only S3 deployments are affected. Local filesystem installs take the archive.file() branch and open no sockets.	2026-09-11 13:28:09 +02:00
Paul Nothaft	ed7ddd1e60	chore(main): release 3.131.4-beta.0 (#1413)	2026-09-11 08:53:37 +00:00
Paul Nothaft	70f5a8c54e	fix(gallery): bound and reclaim storage reads in the remaining zip builders (#1410) ... * fix(gallery): bound and reclaim storage reads in the remaining zip builders PR 1402 fixes the cached-zip builder. The same unguarded pattern — one storage read appended per photo, archiver draining one at a time, nothing destroying the rest — is still present in three other places, two of which a gallery guest reaches with no admin credentials: - routes/gallery/downloads.js, download-all and download-selected - services/downloadJobService.js, the custom-resolution job builder, which is worse in one respect: its per-photo catch skips a bad source without ever destroying the stream it had already opened, so every skipped photo leaked a socket for the life of the process An unread S3 response body holds its socket open indefinitely — the SDK arms its socket timeout on a 3s delay and clears it the moment response headers land, so a fast response never gets one — and archiver's abort() does not touch its source streams. That is the mechanism behind the incident described in PR 1402: 43 of 50 pooled sockets held with unread bytes, uploads and gallery reads starved behind them, a process restart the only way out. utils/archiveStreamGuard.js caps reads in flight at 2 and destroys whatever is still open on every exit: an error, a failed append, and — for the two guest routes — the client closing the tab mid-download, which previously left every appended-but-undrained read parked forever. Deliberately does not touch downloadZipService.js, so there is no conflict with 1402. Once that lands, its inline equivalent can move onto this helper. Local-filesystem installs are unaffected either way: they take archiver's archive.file(path) branch and open no sockets. Relates to issue 1399 * fix(gallery): survive a cancelled download and a read that dies while queued Two failures found reviewing the previous commit, both reachable on an ordinary download. A client hanging up mid-download aborted the archive, but the append loop's `break` still fell through to archive.finalize(), which rejects with ABORTED. The catch then called errorResponse over a response whose ZIP headers had already gone out, throwing ERR_HTTP_HEADERS_SENT from an async Express 4 handler with nothing to catch it — an unhandled rejection, on a cancelled download, which can take the process down. Both bulk routes now return without finalizing, and the catch stays quiet once headers are sent. A read that errored while still QUEUED behind another was absorbed by the guard's own error listener. archiver had not attached its source listener yet, so it never learned the stream had died, and the dead stream stayed in the queue: the archive hung when it reached it, and in downloadJobService the build held its concurrency slot with it. The guard now reports such a failure through onFatalError so the caller aborts the archive and reclaims the rest. Streams the guard destroyed itself are deliberately not reported — that is the caller's own teardown, already running. Relates to issue 1399 --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-11 10:42:22 +02:00
Trung-Tin Pham	7c0c5c1cda	fix(upload): let the csrf gate pass application/octet-stream chunks (#1401) ... The chunked upload route reads the raw request body and its only client sends application/octet-stream, but the CSRF gate on /api rejected every such request with 415 before it reached the route. The endpoint had never accepted a chunk. The gate's origin check is the CSRF defence. Its Content-Type list only has to keep out what a cross-site page can send without a preflight, and octet-stream is not one of those: a form cannot produce it and fetch() with it is not CORS-safelisted. Accept it. express.json leaves an octet-stream body unread, so JSON routes see an empty body as before. Fixes PicPeak/picpeak#1377	2026-09-11 10:42:07 +02:00
Paul Nothaft	4622478e44	fix(upload): stop buffering a chunk body before anything checks its size (#1406) ... * fix(upload): stop buffering a chunk body before anything checks its size The chunk route drained the whole request into an array and concatenated it before calling uploadChunk — which is where every check lives. So a 300MB body sent against an unknown upload id was read in full, cost ~300MB of heap, and was only then answered with an error. The per-file cap was real but applied after the damage, and nothing looked at Content-Length at all. uploadChunk now takes the request stream itself and consumes nothing until the upload id, the chunk index and the declared Content-Length have all been checked against the remaining allowance. A body that clears those is streamed straight to the chunk file under a hard byte cap, so a sender that lies about its length — or sends none, the Transfer-Encoding: chunked case — is cut off at the allowance instead of being read to the end. A Buffer is still accepted, so the existing callers and chunkedUploadSizeCap tests are untouched. Worth noting the old code silently did nothing when handed a stream: fs.promises.writeFile accepts an async iterable, so the body was written while `chunkData.length` was undefined and the cap comparison was NaN > max, i.e. always false. Reachable only for an authenticated admin holding photos.upload on an event they own, and only once the CSRF gate lets application/octet-stream through. Relates to issue 1403 * fix(upload): harden the chunk stream against abort, retry and cap failures Three failure paths the streaming rewrite introduced, all found by external review. None existed in the buffered version: the async iterator it replaced rejected a dead request on its own, and never opened the chunk file until it already held the whole body. - An already-destroyed request hung the call forever. If the client hangs up while auth and ownership are awaiting the database, pipe() emits neither `end` nor `error`, so the promise never settled and the write descriptor stayed open. Checked up front now, alongside `aborted` and a `close` without `readableEnded` for a body cut short mid-flight. - A failed re-send destroyed the chunk it was replacing. createWriteStream truncates on open, so re-sending an index and then failing left receivedChunks and chunkSizes still claiming the old copy: status reported 100% and completeUpload died on ENOENT. Chunks are staged through a sibling .part file and renamed only on success. - Tripping the cap stopped the 413 from reaching the client. `source` is the IncomingMessage, so destroying it destroyed the socket under the response and the client saw a connection reset instead of the size-limit JSON. The read is paused instead, which is all the cap needs. Relates to issue 1403 * fix(upload): isolate chunk staging per attempt and close before cleanup Two races found by a second review round, both in the staging logic added by the previous commit. - Two in-flight sends of the same chunk index shared one `.part` path, so whichever renamed first published bytes the other had already truncated. An acknowledged 10-byte chunk could end up 2 bytes. The staging suffix is now per-attempt rather than per-index. - Unlinking the partial file raced the write stream's pending open(). destroy() does not await it, so the unlink failed with ENOENT and the open then recreated the `.part` file after cleanup had supposedly finished — reported reproducible in 121 of 300 immediately-failing streams. Cleanup now waits for the stream to close. Relates to issue 1403 * fix(upload): revalidate the per-file cap before publishing a chunk `allowance` is computed before the body arrives, so a chunk that completed while this one was still streaming was not counted in it. Two overlapping 0.75MB chunks under a 1MB cap were therefore both accepted, leaving 1.5MB on disk; enough concurrent streams could go well past the cap before anyone asked to complete the upload. The buffered version got this right for free, because it only ever checked after reading the whole body. The streaming version keeps its pre-read check — that is what makes a too-large request cheap — and asks again with the current aggregate before renaming the staging file into place. Relates to issue 1403 * fix(upload): answer client-caused chunk states with their own status code An unknown, finished or expired upload id, and a complete call with chunks missing, were plain Errors with no statusCode, so both routes fell through to the blanket 500 and logged at error level. All four are the client's mistake: they read as a backend fault in monitoring and invite a retry that can never succeed. They now carry 404, 409, 410 and 400 respectively, and both routes pass a tagged status through instead of matching on the two they happened to know about. Only genuinely unexpected errors reach the 500 and the error log. No client is affected: uploadLargeFile, the only caller of this endpoint family, still has no callers of its own. Relates to issue 1403 * fix(upload): retire the connection after an early refusal, clean up a failed publish Two more from review. Refusing a body before reading it is the point of the streaming cap, but the unread bytes are still in flight on a connection the response advertises as keep-alive. Node does not drain them, so the NEXT request on that socket hangs until it times out — reproducible with an 8MB body against a 1MB cap. The error response now sets Connection: close whenever the request was not read to the end. A failed rename — ENOSPC, a vanished directory — left the fully written staging file behind. Staging names are per-attempt, so a client that retries instead of aborting accumulates one per try until the upload expires. The partial is removed on that path too. Relates to issue 1403 --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-11 10:41:50 +02:00
Paul Nothaft	f92d4bb2d9	fix(gallery): let an admin preview a draft through its short share URL (#1405) ... fix(gallery): keep an admin draft preview out of the guest share-login flow Making verify-token pass for a draft preview opened a path that did not exist before it: the gallery bootstrap then called shareLinkLogin, which refuses a draft AND records a failed login attempt against the caller's IP while doing it. Five preview opens inside the attempt window therefore locked share-link logins out for that IP — including for real guests, and including after the gallery was published. An admin preview does not need a guest session at all. The admin cookie plus admin_preview=1 already authorizes every gallery call, which is exactly how preview works on a published gallery, so the preview path loads the gallery directly and never touches the login endpoint. Deliberately not fixed by relaxing shareLinkLogin's draft check: that endpoint mints a guest token, and a draft should not be handing those out. Relates to issue 1386 fix(gallery): let an admin preview a draft through its short share URL /info has honoured admin_preview since issue 868, but two sibling routes on the short-URL path never did: - GET /resolve/:identifier filtered drafts out through ACTIVE_EVENT_FILTER (shareLinkService.js), with no escape for a verified admin. - GET /:slug/verify-token/:token repeated the same filter inline, so clearing the first would only have moved the 404 one step later. With "use short gallery URLs" off the View Gallery link carries the slug, GalleryPage never calls /resolve, and the preview worked. With it on the link is the token form, GalleryPage resolves it first, and the draft answered "Gallery Not Found". resolveShareIdentifier takes an includeDrafts option, and /resolve reaches for it only after the published lookup misses AND verifyAdminPreview accepts the caller — so the published path keeps its single query and an unverified caller never learns the draft exists. The frontend already sends admin_preview=1 (EventDetailsHeader.tsx:203, forwarded by config/api.ts:81); only the backend had to change. GHSA-rh8r's rule is unchanged and now pinned by test: a bare slug lookup still never returns share_token, draft or not. Relates to issue 1386	2026-09-11 10:41:25 +02:00
Paul Nothaft	1080388f28	fix(gallery): keep videos playable under enhanced and maximum protection (#1404) ... Once an event left `standard` protection, both halves of the video path were routed through /api/secure-images, and neither half can carry a video. galleryQueryService emitted the secure template as a video's `url`. The lightbox drops that straight into a <video> element; nothing substitutes the `{{token}}` placeholder (the helper that could, secureToken.service.ts, has no importers), so the request answered 403 "Invalid or expired token". Even with a valid token it would still have failed — the secure-images route pipes every byte through secureImageService.processProtectedImage, which calls sharp() and throws on an mp4. routes/gallery/media.js bounced the JWT route to that same endpoint before reaching its own video branch, so there was no way through. Videos now keep the JWT route at every protection level, on both sides. That is not a new exposure: thumbnails of those same videos have always been served from it, and a valid gallery token is still required to reach it. Still images are unaffected and keep bouncing to the secure endpoint. VideoPlayer had no `error` listener, so all of this rendered as a poster frozen at "0:00 / 0:00" behind a play button that did nothing — indistinguishable from a codec the browser cannot decode, which is the other common cause (HEVC/H.265 phone footage plays in Safari and nowhere else). It now surfaces the failure and names the codec case, since the answer there is to download the file. Relates to issue 1370 Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-11 10:41:17 +02:00
Paul Nothaft	316bcbd679	fix(backend): contain and sanitize the SQLite restore source path (#1384) ... The restore flow accepted an unvalidated database.backup_file from the manifest (absolute paths and traversal both worked, and no containment check enforced the configured backup root), then interpolated it unescaped into a `sqlite3 .restore '<path>'` command, letting an attacker-chosen source file replace the live database. Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-11 10:40:54 +02:00
Paul Nothaft	ec03089d57	fix(backend): validate the S3 endpoint host before the restore download (#1383) ... * fix(backend): validate the S3 endpoint host before the restore download downloadFileFromS3() built an S3StorageAdapter and called .download() directly, skipping the isHostAllowed() private-IP/DNS-rebinding guard that testConnection() applies elsewhere — an admin with backup.restore could point the configured S3 endpoint at an internal/metadata address for unauthenticated egress via the server. * fix(backend): pin the restore S3 download to its validated DNS resolution isHostAllowed() was check-then-connect: the AWS SDK re-resolves the endpoint hostname independently when it actually connects, so a DNS rebinding condition between the preflight check and the real connection could still reach a private/internal address. Reuses the same pinnedRequestOptions() primitive webhookDeliveryWorker.js already uses, wired into the S3Client's requestHandler. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-11 10:40:50 +02:00
Paul Nothaft	b798d8e4c1	fix(backend): use the strong password generator for resets and enforce must_change_password (#1387) ... Admin password reset generated a ~2^21-entropy password from a small wordlist instead of the already-available generateSecurePassword(16), and must_change_password was written on reset but never checked by any route-blocking logic — a reset user could keep using the old session/password indefinitely. Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-11 10:40:46 +02:00
Paul Nothaft	59ea83c84e	fix(backend): require actor to hold every permission of a role they grant (#1378) ... * fix(backend): require actor to hold every permission of a role they grant Any admin with `users.edit` could grant an arbitrary non-super_admin role — including one carrying far more permissions than they themselves hold — via PUT /api/admin/users/:id. The role-change path never called the existing assertActorMayGrant() guard that already protects role create/edit. * fix(backend): apply the same role-grant guard to admin invitations createInvitation() only blocked granting super_admin — the same users.create-holder-can-invite-into-any-role escalation that updateAdminUser() was fixed for (GHSA-rv8w-m6mx-7j4q) was still open via POST /admin/users/invite. Reuses assertActorMayGrant(). --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-11 10:40:42 +02:00
Paul Nothaft	38b0e1d584	fix(backend): validate event id before using it in the logo storage filename (#1382) ... * fix(backend): validate event id before using it in the logo storage filename The multer filename callback built the stored path directly from req.params.id with no integer validation, letting a traversal payload in the route param escape the intended uploads/logos/events/ directory — most directly reachable via a super_admin session, since requireEventOwnership short-circuits with no DB lookup for that role. * fix(backend): validate contract id before using it in the signed-PDF storage filename Same pattern as the event-logo fix (GHSA-9q5j-vqfw-32hr) in a different file adminContracts.js never touched: multer's filename callback ran before express-validator's :id check, letting a traversal payload escape uploads/contracts/signed/. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-11 10:40:38 +02:00
Paul Nothaft	abc960170b	fix(backend): validate business-profile logo uploads by content, not filename (#1381) ... * fix(backend): validate business-profile logo uploads by content, not filename The upload route skipped the shared validateFileType() helper every sibling upload route uses, and derived the stored extension from the client-supplied filename. A file could declare an image MIME type while carrying an executable/HTML extension and arbitrary content, then be served same-origin via the mass-assignable logoPath field. * fix(backend): content-sniff business-profile logo uploads too fileFilter paired the claimed MIME type against the extension but never verified the actual bytes matched, unlike other upload routes that already call validateFileContent(). Defense-in-depth: the extension-confusion XSS itself was already closed (stored extension is derived from the validated MIME, not client input), this closes the remaining gap where declared-vs-actual content can still diverge. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-11 10:40:34 +02:00
Paul Nothaft	cdde937d7f	fix(backend): reject a replayed TOTP code within its validity window (#1389) ... * fix(backend): reject a replayed TOTP code within its validity window verifyTotp() was stateless — otplib's window:1 tolerance meant the same 6-digit code could complete two independent logins inside its ~90s validity window. Track each admin's last-consumed step and reject a code that doesn't advance past it. * fix(backend): make the TOTP replay-tracking persist atomic verifyTotpEncryptedStep() read two_factor_last_used_step, then a plain UPDATE wrote the new step with no conditional guard — two concurrent requests carrying the same captured code could both pass the check before either UPDATE landed. The persist is now a conditional UPDATE (only advances the step, checked via affected-row count), so a losing concurrent request is correctly treated as a replay. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-11 10:40:27 +02:00
Paul Nothaft	e3247911a0	fix(backend): shorten payment-check token TTL and notify admin on use (#1385) ... The unauthenticated payment-check magic link (intentional, matches publicQuotes.js) had a 30-day token lifetime and wrote to the invoice ledger silently. Shortened the TTL and added a best-effort admin notification on every write via this route, so the no-login convenience stays but an admin always sees the action happen. Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-11 10:40:24 +02:00
Paul Nothaft	e290207934	fix(backend): enforce event ownership on short URL deletion (#1379) ... GET and POST for an event's short URLs both required requireEventOwnership; DELETE only checked events.edit permission, letting any admin holding that permission delete another tenant's branded gallery short URL. Resolve the short URL's event first, then apply the same ownership check the other routes use. Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-11 10:40:20 +02:00
Paul Nothaft	f6b81fabf0	fix(backend): bump sharp, nodemailer, multer, js-yaml, joi for security fixes (#1374) ... * fix(backend): bump sharp, nodemailer, multer, js-yaml, joi for security fixes Resolves 12 open code-scanning alerts (#589-600): sharp libheif RCE, nodemailer address-parser ReDoS + domain-validation bypasses, multer upload DoS/race conditions, js-yaml parsing DoS, and joi prototype pollution. All patch/minor bumps within the currently used major version. * fix(backend): set multer's fieldArrayIndexLimit to actually close CVE-2026-82333 The advisory is explicit that the 2.3.0 version bump alone doesn't remediate the array-index DoS — an app must also set limits.fieldArrayIndexLimit. Set it on every multer instance, sized to what each route's form actually needs. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-11 10:40:17 +02:00
Paul Nothaft	c4d89c9d64	chore(main): release 3.131.3-beta.0 (#1373)	2026-09-10 11:54:50 +00:00
Paul Nothaft	a2bf1f644c	fix(video): try metadata extraction and thumbnail generation independently (#1371) ... * fix(video): try metadata extraction and thumbnail generation independently processUploadedVideo() gated everything behind isValidVideo(), which rejects the whole video if ffprobe can't read even one of duration/width/height -- common on some iPhone/Lightroom-exported MP4s (issue 1370). Callers (photoProcessor.js's processPhoto and processUploadedPhotos) already catch that throw and fall back to a static placeholder thumbnail plus a metadata-only retry (codex review of #845), but that fallback never got a REAL thumbnail even when generateVideoThumbnail() would have succeeded on its own -- thumbnailing doesn't need valid duration/width/height, it just seeks and grabs a frame. processUploadedVideo now tries metadata extraction and thumbnail generation independently, keeping whichever succeeds instead of discarding both on a single failed field. The callers' existing throw handling stays as a backstop. Also: extractVideoMetadata stored duration as 0 (not null) whenever ffprobe had no duration field, masking "unknown" as a fake real zero-second clip and defeating downstream `duration != null` checks meant to skip an untrustworthy value. Relates to issue 1370 * fix(video): fall back to the SVG placeholder when thumbnail generation fails processUploadedVideo could return success with thumbnailKey: null when only thumbnail generation failed. The gallery grid (GridGalleryLayout/JustifiedGalleryLayout) falls back to `photo.thumbnail_url || photo.url` when there's no thumbnail, so AuthenticatedImage downloaded the full original video and tried to render it as an <img> -- a broken tile and a potentially huge fetch just from opening the gallery. Falls back to the same ffmpeg-free SVG placeholder the callers already generate for a total processing failure, so a bare thumbnail-generation failure degrades to that placeholder too, never to "no thumbnail at all". Found by codex review. * fix(video): avoid a SQLite connection deadlock in the placeholder fallback generateVideoPlaceholder() unconditionally called getThumbnailSettings(), which queries the database directly (not through any active transaction). videoProcessor.js's new placeholder fallback can run from inside processUploadedPhotos' open per-file SQLite transaction (chunked video upload) -- knex's default SQLite pool has exactly one connection, so that second, un-transacted query deadlocks against the transaction holding it, timing out after acquireConnectionTimeout (60s). Reproduced directly against an isolated SQLite db. generateVideoPlaceholder now skips the settings lookup entirely when the caller supplies explicit width/height, and the video fallback passes the same DEFAULT_THUMBNAIL_WIDTH/HEIGHT the settings lookup would have fallen back to anyway (now exported for reuse). Found by codex review. * fix(video): throw when neither a real thumbnail nor the placeholder can be produced processUploadedVideo returned success with thumbnailKey: null when both the real thumbnail AND the SVG placeholder failed -- a total, systemic failure (storage backend down, disk full), not a quirk of one file. On stable, which doesn't have the #845 call-site fallback, this silently completed the video with no thumbnail at all instead of the retryable 'failed' status a throw here produces. On main, the pre-existing #845 fallback already absorbed this exact case (no behavior change there) -- verified against codex's own git-blame check of the pre-PR stable code before applying this. Now throws in that case, restoring the pre-existing "let the caller mark it failed and retryable" behavior for a genuinely unrecoverable video, while keeping every partial-failure case (the vast majority) resolving with whatever succeeded. Found by codex review. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-10 13:48:48 +02:00
Paul Nothaft	443ec91de9	chore(main): release 3.131.2-beta.0 (#1368)	2026-09-09 20:35:16 +00:00
Paul Nothaft	15cd5ede82	fix(backup): honor the configured database-backup destination path (#1366) ... * fix(backup): stop ignoring the configured database-backup destination path databaseBackupService.getBackupConfig() returns the raw database_backup_*-prefixed setting keys, but backup() and startScheduledBackups() destructured unprefixed names off that object (destinationPath, compress, enabled, schedule, retentionDays, emailOnSuccess/Failure). None of those keys ever existed on the config object, so every read silently fell through to its hardcoded default. The visible symptom (reported in issue 1365): the inline database dump that runs before every file backup (default ON) always tried to create /backup/database, regardless of what an admin configured, and died with EACCES on the read-only default path — before the file backup's own (correctly wired) backup_destination_path was ever reached. The standalone scheduled database-backup runner had the same bug: config.enabled was always undefined, so it silently never started regardless of database_backup_enabled. Also fixes saveManifestToLocal's manifest-directory fallback, which hardcoded /backup instead of matching the sane getStoragePath()/backups default used everywhere else for a missing backup_destination_path. Relates to issue 1365 * fix(backup): reject a database-backup destination inside a public static mount Making database_backup_destination_path actually take effect reopens a GHSA-jw8m-43r2-jqrm-class exfiltration path: that setting is writable via PUT /api/admin/database-backup/config under backup.create alone (the built-in admin role has it without settings.edit or backup.restore), with no path validation. Before this fix the setting was silently ignored (the destructuring bug), so pointing it at the public uploads/logos or fonts mount was harmless; now that it is honored, it needed the same defense GHSA-jw8m already applies to the per-request override. Rejects the setting at both the config write (immediate 400) and, defensively, at backup() time before mkdir. Found by codex review. * fix(backup): close two gaps codex round 2 found in the destination guard - The public-roots list missed the bundled fallback fonts dir (backend/assets/fonts, also mounted at /fonts, and nodejs-owned per the Dockerfile's COPY --chown so it's writable at runtime). - The comparison was case-sensitive; on a case-insensitive-but- preserving filesystem (APFS, NTFS, Docker Desktop bind mounts of either) STORAGE_PATH/UPLOADS/Logos names the same directory as uploads/logos on disk. Now compares lowercased. - database_backup_retention_days reached cleanupOldBackups unvalidated. A value <= 0 pushes the cutoff to today or the future, deleting every completed backup on the next scheduled run -- a backup.create holder achieving what backup.delete gates on the manual /cleanup route. Rejected at config-write time (400) and defensively inside cleanupOldBackups itself. - The scheduled-backup cron callback closed over retention_days from schedule-start time; a retention-only /config update (which doesn't restart the schedule) ran stale until restart. Re-reads it on every tick instead. Found by codex review, round 2. * fix(backup): resolve symlinks and add the all-in-one frontend dir to the destination guard Codex round 3 found two more bypasses of the public-root guard, both specific to the all-in-one image (Dockerfile.aio): - /app/frontend/dist (FRONTEND_DIR) ships nodejs-owned and is served unauthenticated as the built SPA -- missing from the protected-roots list. - /app/storage is a symlink to /data/storage (the actual STORAGE_PATH). A destination given as /app/storage/uploads/logos passed the guard's lexical path.resolve() comparison while resolving, on disk, to the exact same directory as the protected STORAGE_PATH/uploads/logos. isUnderPubliclyServableRoot now resolves symlinks in whatever prefix of each path already exists (resolveRealish) before comparing, rather than relying on path.resolve() alone. Also restores three fs.mkdir spies in the test file that were never un-spied, which silently leaked a rejected mock into any later test doing a real fs.mkdir -- exactly what the new symlink test needed to set up its fixture. Found by codex review, round 3. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-09 22:27:18 +02:00
Paul Nothaft	6d717544ec	chore(main): release 3.131.1-beta.0 (#1364)	2026-09-08 20:08:38 +00:00
Paul Nothaft	de3ae1f176	chore(main): release 3.131.0-beta.0 (#1362)	2026-09-08 17:32:31 +00:00
Paul Nothaft	59ef2ee9af	Merge pull request #1361 from PicPeak/feat/usage-reporting-update-prompt ... feat(usage): prompt existing admins once for usage reporting after an update	2026-09-08 19:23:54 +02:00
Paul Nothaft	ff23efec81	chore(usage): renumber prompt_shown migration to 212 ... #1359 independently added 211_revocations_without_expiry.js against the same main baseline. Renumbering this one to 212 keeps the migrations directory sequentially numbered once both land, regardless of merge order. No functional change — same up()/down(), same column.	2026-09-08 19:22:01 +02:00
Paul Nothaft	9a437ee9e1	fix(usage): preserve consent choices and make the prompt accessible	2026-09-08 17:22:47 +02:00
Paul Nothaft	d20f80112f	feat(usage): prompt existing admins once for usage reporting after an update ... An admin who already had PicPeak installed before the opt-in reporting feature existed never gets asked — the setup wizard only runs once, on a brand-new instance. Adds a one-time modal, shown on the admin's next dashboard visit after updating, offering the same choice the wizard gives a new install. - New `product_usage_state.prompt_shown` column (migration 211) and UsageService.markPromptShown(), set on either outcome (enable or decline) from both this modal and the wizard step, so an installation is never asked twice regardless of which path it took. - New POST /admin/usage/prompt-seen endpoint. - Extracted the wizard's three-point pitch (UsageReportingPitch.tsx) so the modal and the wizard step share identical copy instead of drifting apart. - The modal never shows once participation is already active, and never shows a second time after either the wizard or the modal has been through it once. Depends on #1360 (the setup wizard step this reuses).	2026-09-08 16:25:51 +02:00
Paul Nothaft	662516a5ad	fix: retain revocations for tokens without expiry	2026-09-08 15:54:01 +02:00
Paul Nothaft	a31a2e25e2	fix: interrupt idle worker waits during shutdown	2026-09-08 15:54:01 +02:00
Paul Nothaft	20a291eb34	chore(main): release 3.130.2-beta.0 (#1358)	2026-09-08 13:40:19 +00:00
Paul Nothaft	f0e6d2dfb1	fix: enforce gallery access and consolidate gallery workflows (#1357) ... Harden gallery authentication and authorization, consolidate gallery workflows, and prevent token-bearing URLs from leaking through nginx request error logs.	2026-09-08 15:34:09 +02:00
Paul Nothaft	895e5ab3cc	chore(main): release 3.130.1-beta.0 (#1354)	2026-09-08 06:53:51 +00:00
Paul Nothaft	acb25a9a1c	fix(images): probe and clean up preview tiers under the extension the encoder actually wrote (#1355) ... generatePreviewImage rewrites the output extension to match the encoding it chose, .jpg or .webp for alpha and multi-frame sources. The tier lookup in ensurePreviewImageAtWidth and the cleanup list in previewTierKeys kept the SOURCE extension instead, so for anything but a lowercase .jpg source the stat never matched: every tier request for a .png, .JPG, .jpeg, .heic or RAW photo re-ran Sharp, and cleanup never found the files it left behind, which accumulated for the life of the install. Both now derive every key the tier can live under: the .jpg and .webp candidates, plus the source-extension key last so tiers written before the rewrite are still found by lookup and by cleanup. Follow-up to issue 1020, where the mismatch was identified during review. Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-08 08:41:53 +02:00
Paul Nothaft	c97341e454	fix(images): single-flight lazy rendition generation and keep the old rendition during replacement (#1350) ... * fix(images): single-flight lazy rendition generation and keep the old rendition during replacement The lazy generators in imageProcessor are check-then-generate, and the check reads the path off the photo row the route already fetched. N concurrent cold requests for one photo all held a snapshot with the path still null, all missed, and all ran the same sharp pipeline. Only the thumbnail tier path had a guard; the canonical thumbnail it falls back to, heroes, previews and preview tiers had none. One process-local map now covers every rendition, keyed by photo id and rendition (`thumbnail:<id>`, `thumbnail:<id>:w<width>`, `hero:<id>`, `preview:<id>`, `preview:<id>:w<width>`). Concurrent callers share one promise; the entry is cleared in a finally on success and failure alike so a rejection cannot poison the key. The tier stat moved inside the flight so a request arriving as the previous flight clears finds the written tier instead of missing on a stale probe. Heroes and previews also deleted the existing object before generating its replacement, and again in the catch. Both are gone, mirroring what the thumbnail generator already does: put is the last statement in the try and replaces atomically on local storage and by key on S3, so the delete only ever opened a window with no rendition at all, and a source that failed to read stripped the old rendition with the row still pointing at it. No re-read of the photo row inside the flight: the admin regenerate endpoints force a rebuild by passing a row with the path nulled, and a re-read would hand back the persisted rendition untouched. Fixes the single-flight half of issue 1020. The preview cache-key extension mismatch and any server-wide work queue remain separate. * fix(images): keep the snapshot validity check outside the single-flight With the check inside the flight, an admin regeneration (row passed with the path nulled) could join a viewer's flight for the same photo that was merely confirming an already-good rendition, and be handed back the very file it was asked to replace while the endpoint counted a success. Only a miss enters the flight now; inside it everything is a regeneration. * fix(images): forced rebuilds run after an in-flight lazy generation instead of adopting it The admin regenerate endpoints could still join a lazy flight that was already generating for the same photo. That flight read the thumbnail settings when it started, so after a settings change it produces exactly the rendition the regenerate was invoked to replace; adopting it counted a success while the old size stayed cached. ensureThumbnail and ensurePreviewImage take `{ force: true }`: skip the snapshot check and, if a flight is pending, start after it settles. Lazy misses arriving meanwhile join the forced flight, and an older flight settling late no longer evicts the newer entry from the map. * fix(images): key rendition flights by source as well as photo id replacePhoto keeps the photo id and changes path and filename. Keyed by id and width alone, a request carrying the replacement row joined a flight still rendering the file it replaced and was handed the old image, which the gallery caches for 30 minutes. The tier map this replaced was keyed by storage key and so already told the two apart. * test(thumbnails): wait for the regenerate loop's completion line instead of a fixed 150 ms The loop runs in setImmediate after the response. Under a loaded machine (fifteen suites in parallel, each booting a migrated SQLite) it took longer than 150 ms once and the assertions ran against a half-finished mock call list. Poll the logger spy for the "regeneration complete" line with a 10 s deadline; the suite also finishes sooner because the wait ends as soon as the loop does. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-08 08:33:04 +02:00
Paul Nothaft	7e9a61c245	chore(main): release 3.130.0-beta.0 (#1347)	2026-09-07 20:36:18 +00:00
Paul Nothaft	810801a9ab	fix(events): drop non-canonical keys from the event update before any check runs (#1346) ... PUT /admin/events/:id spreads the body into the UPDATE. SQLite resolves quoted identifiers case-insensitively, so `{ "Event_Name": ... }` lands on event_name there — while every check in the handler (validators, the field-level permission guards, the deny-set) keys on the exact lowercase name. The deny-set already case-folded for its own columns; every other column was reachable through a spelling variant. Every events column and every input-only key the handler accepts is lowercase snake_case, so a key with any uppercase in it is not something a legitimate client sends. Such keys are now removed before anything looks at the body. Postgres was unaffected (quoted identifiers are case-sensitive there; a variant produced a 500 instead). Surfaced by the Codex review of the folder-watcher change, where a photos.upload guard on external_watch could be walked around this way. Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-07 22:30:13 +02:00
Paul Nothaft	8cc7d7d14a	feat(external-media): watch reference folders and import new files automatically (#1345) ... * feat(external-media): watch reference folders and import new files automatically Managed uploads dropped into storage/events/active are picked up by the chokidar watcher; external media had no equivalent, so a NAS folder that keeps growing needed an admin to open the event and press Import every time. Relates to issue 1187. - The import pass moves out of the route into services/externalImportService.js. The watcher and the Import button now run the identical function; the route only validates and maps errors to status codes. - Mutual exclusion is the per-event claim from maintenanceJobState (`external_import:<id>`, seeded on demand by the new ensure()) instead of the in-process Set. The Set stopped a double-click in one process; the claim also stops the watcher on a second replica, or an admin clicking while the watcher is mid-run elsewhere. The run heartbeats so a claim from a dead process is taken over. - services/externalMediaWatcher.js: per-event opt-in via the new events.external_watch column (migration 208), chokidar with awaitWriteFinish so a copy in flight is not imported half-written, debounced full pass per change, a timer sweep every 15 minutes as the fallback for NFS/SMB mounts that deliver no inotify events, optional stat-polling via EXTERNAL_MEDIA_WATCH_POLLING. The set of watched events is re-read every minute, so the toggle works from any replica. A watcher that just started runs one pass immediately. - Deletions are ignored on purpose: a file vanishing from a NAS is at least as likely to be a reorganisation or a dropped mount as an intentional removal, and acting on it would delete a guest-visible photo. Rows whose file is gone stay, as they do today. - Not gated on STORAGE_BACKEND: EXTERNAL_MEDIA_ROOT is always local. - Quiet system passes stay out of the activity log; runs that imported something are logged with actor external-media-watcher. - Frontend: "Watch folder for new files" checkbox under the external folder picker, status line in view mode, EN/DE strings. * fix(external-media): close the review gaps in the folder watcher Codex review of the watcher, round 1. All six findings were real: - Enabling the watcher, or pointing an enabled one at another folder, now requires photos.upload — the permission the manual Import already requires. events.edit alone was a way around it. Only the transition is checked, so a role without photos.upload can still edit an already-watched event. The checkbox is disabled for such roles. - Automatic passes defer files that are still changing: anything modified inside the stability window, or whose size moves across one wait of that window, is left for the next pass. chokidar's awaitWriteFinish only settles the file that fired the event, and the sweep sees no events at all, so a sibling still being copied could be inserted half-written and then skipped forever. - Photos an admin deleted are not brought back by the sweep. The delete routes record the file in external_import_exclusions (migration 209); automatic passes skip the list, the manual Import ignores it and clears it for what it imports. - The six EXTERNAL_MEDIA_WATCH* variables are forwarded in all three compose files; they were documented but the backend services use explicit environment lists, so the kill switch did nothing. - A pass re-checks is_active / is_archived at run time, not only in the minutely reconcile. - The lease is renewed on a timer for the whole run, walk included, and ownership is checked before the event row is touched. * fix(external-media): make automatic passes follow the row, not rewrite it Codex review round 2, four findings, all applied: - The event update route drops non-canonical spellings of external_watch and external_path before the permission guard. SQLite resolves column names case-insensitively, so `External_Watch` reached the column while the guard only looked at the lowercase key. - Exclusions are checked per file at insert time, not against a snapshot taken before the settle wait. A photo deleted during the wait was present in the snapshot and got re-inserted by the loop. - An automatic pass no longer writes source_mode / external_path. It re-reads the row after the walk and the settle wait and stops if the folder changed or the event went managed; the manual Import is the only writer. The options are now `automatic` + `settleMs`. - A pass that deferred files re-arms the debounced import, so a file copied just before the watcher started is not stranded when the sweep is disabled. * fix(external-media): keep exclusions for replaced photos, stop a pass whose event stopped qualifying Codex review round 3, both findings applied: - recordExclusions keys on external_relpath alone. A replaced external photo becomes managed but keeps its relpath on purpose, and deleting that replacement must not republish the NAS original. - An automatic pass checks the full watcher predicate (reference mode, same folder, watch on, active, not archived) before it inserts and on every heartbeat tick during the loop, and stops as soon as the event no longer qualifies. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-07 22:29:55 +02:00
Paul Nothaft	1b3f721d10	chore(main): release 3.129.0-beta.0 (#1344)	2026-09-07 18:47:26 +00:00
Paul Nothaft	fb9da72f14	feat(security): opt-in recoverable gallery passwords (#1341) ... * feat(security): opt-in recoverable gallery passwords Gallery passwords are bcrypt hashes, so an admin who needs to hand a password to a client a second time has to reset it, which invalidates what the client already has. This adds a security setting, security_gallery_password_recoverable, off by default, that keeps an AES-256-GCM encrypted copy of each gallery password and client PIN next to the hash. The key is derived from GALLERY_PASSWORD_ENCRYPTION_KEY or JWT_SECRET. While the setting is on: - create, publish, send-later, edit, reset and the v1 API write the copy alongside the hash; turning a gallery's password requirement off clears it - GET /api/admin/events/:id/password returns the copy to admins with events.edit and ownership, and writes a gallery_password_viewed activity entry on every real reveal - resend-email uses the stored password instead of the "set at creation" sentinel, so the client receives what already works Switching the setting off purges every stored copy. Login and hash verification are untouched; the copy is never read on the gallery side. The Security tab carries the toggle with a warning that stays visible, and the event page shows "Show password" with copy buttons only while the setting is on and the gallery has a secret. Relates to issue 1271 * fix(security): close the write-versus-switch-off race in the password vault The recoverable setting is read while an event insert is assembled and the client-PIN hash awaits after that, so a settings request that switched the feature off and purged in that gap was overtaken by the insert. Every write site now re-reads the setting right after its statement and clears its own row when the setting is off; the settings writer flips the value before it purges, so either the purge or the re-check catches the row. * fix(security): resend carries the stored client PIN and link; deterministic tamper test The creation mail includes the client-access link and PIN; a resend only sent the gallery password even when a stored PIN was available. The ciphertext tamper assertion replaced the last two characters with a constant, which was a no-op roughly once in 4096 runs. * fix(security): drop the revealed password after Send gallery email The send-later route can replace the password; the share card keys its revealed copy on the event query's refetch time, so invalidate the event after the send like the other password-changing mutations do. * fix(security): purge leftovers before the setting write when turning recovery on Switching on wrote the setting first and purged after, so a password write that read the new "on" in between stored a copy the purge then deleted. Turning on now purges before the write; turning off keeps purging after it, which together with the write-site re-check leaves the vault holding exactly what was written while the setting was on. * chore(security): drop the duplicate rateLimitService import left by the rebase * chore(usage): register the password recovery routes in the v5 coverage inventory The inventory moved from v4 to v5 on main; the entry added by this branch followed it. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-07 20:36:48 +02:00
Paul Nothaft	16b79ee119	chore(main): release 3.128.0-beta.0 (#1343)	2026-09-07 18:10:48 +00:00
Paul Nothaft	5c1e38d921	feat(usage): distinguish real edits and template delivery with v5 consent (#1339) ... * feat(usage): distinguish real edits and template delivery with v5 consent * fix(usage): exclude queued test messages and count reorders as edits - queueEmail carries usageEligible: false into email_data and the queue processor passes it on, so the dev tools' send-test-email no longer records email_template_delivery once the worker sends it. - event-types/reorder and categories/reorder-global compare the persisted order before and after and record the v5 edit markers only when it changed, matching the display_order edit already counted on PUT. - normalized() builds arrays with Array.from so a row array from the sqlite binding compares equal under Jest's separate realm. * fix(usage): cover per-gallery category order and workflow test runs - categories/reorder records category_editing when an event's override changes; reorder/:eventId records it when an override was actually removed. - send_email and the collections handoff pass usageEligible: false for a workflow test run (engine.testRun sets __test), so a non-dry test send is not counted as template delivery. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-07 20:03:15 +02:00
Paul Nothaft	69754f8a2c	fix(email): scrub gallery passwords from the sent-mail archive (#1340) ... * fix(email): scrub gallery passwords from the sent-mail archive The email queue kept every gallery password and client PIN in clear text in email_data and rendered_html after the mail was sent, and the Messages reading pane handed them back to any admin with the messaging flag. A password hash in the events table bought nothing while the plaintext sat next to it. Once a mail is out, or its retries are exhausted, the processor now masks secret-looking variables (password, passcode, pin) in email_data and replaces their values in the rendered body, plain and HTML-escaped. The reading pane applies the same masking to rows archived before this change. Pending rows keep the real values so a retry still sends them. Relates to issue 1271 * fix(email): keep a quoted ">" from cutting an attribute value out of redaction The tag splitter stopped at the first ">", so a template attribute such as title="{{gallery_password}} > details" left the password unmasked in the archived HTML while email_data was already masked. The tokenizer is now quote-aware; a tag with an unbalanced quote falls through as text and is scrubbed there. * fix(email): scrub secrets inside HTML comments in the archived body A comment such as <!-- PIN: {{client_password}} --> was split off as a tag and its body, which has no attribute, was never scrubbed. Comments are now one segment and their content is masked whole. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-07 20:02:57 +02:00
Paul Nothaft	8017370271	feat(settings): expose the API rate limiter in the Security tab (#1338) ... * feat(settings): expose the API rate limiter in the Security tab The general per-IP limiter had six settings in app_settings and a backend route to write them, and no screen. Installs ran on the code fallback — 300 requests per 15 minutes per IP — with no way to see it, which is how issue 1287 played out: a 546-photo gallery exhausted the budget for one viewer and the operator learned about the setting from a grep of the backend log. Security tab: a card with the six settings, the validation ranges the route enforces, a one-line explanation per field, and a note that the unit is the client IP — an office or household behind one NAT shares a budget, and behind a proxy TRUST_PROXY has to cover the proxy or every visitor shares its address. The tab's Save button saves the limiter through its own route. The limiter values are checked against the route's ranges before anything is written and the limiter is written first, so a rejected value cannot leave the password/session settings half-saved behind a failure toast. Backend, three things the screen needed: - The settings read fills the six keys with the code defaults when they have no row, so the form shows the budget in force rather than an empty field; the defaults live in one exported constant the limiter itself reads. - The write route upserts instead of updating: on a fresh install, which has no rows, the old UPDATE matched nothing and the route answered 200 while changing nothing. - The live limiter instances move into rateLimitService and the write route rebuilds them. express-rate-limit fixes windowMs when an instance is built — max and skip re-read the settings per request, the window does not — so a saved window used to apply only after a restart. The gates in server.js resolve the instance per request through the service's getters. A rebuild starts fresh counters, which on a settings change is acceptable. The limiters get explicit MemoryStores and a rebuild shuts the superseded ones down, because a store keeps a cleanup interval alive for as long as it exists and dropping the reference alone would leak one timer per save. Tests: the read surfaces defaults and honours the key filter; the write creates rows on a fresh database, the limiter sees the values immediately and hands the gates a fresh instance; existing rows are updated not duplicated; out-of-range values are rejected. The tab renders the values, edits through the hook state, carries the ranges, saves with the tab's button, and the pre-write validation accepts the bounds and rejects outside them and cleared fields. Relates to issue 1337 * docs(security): point the rate limiter doc at the Security tab and the upsert --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-07 20:02:40 +02:00
Paul Nothaft	834111fc66	chore(main): release 3.127.2-beta.0 (#1342)	2026-09-07 14:40:56 +00:00
Paul Nothaft	9bbdca9fc5	docs(security): correct the rate limiter defaults and how they are set (#1336) ... * docs(security): correct the rate limiter defaults and how they are set SECURITY_LOGGING.md said rate_limit_max_requests defaults to 1000 and that the settings are configurable via the admin panel. The code fallback when no app_settings row exists is 300 (19e125d8), a fresh install has no row, and there is no admin screen: the settings are written by PUT /api/admin/settings/security/rate-limit, which nothing in the frontend calls. The reporter of issue 1287 ran on the 300 default without any way to see it. The table now carries the real defaults, what the auth budget counts, the exemptions including the gallery-image one from v3.127.0-beta.0, and the TRUST_PROXY and shared-NAT caveats. Relates to issue 1287 * docs(security): note the rate-limit route only updates existing rows --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-07 16:35:13 +02:00
Paul Nothaft	e43216311b	chore(main): release 3.127.1-beta.0 (#1334)	2026-09-07 07:48:14 +00:00