Video uploads on production fail with "missing ffmpeg" because the
backend container ships nothing usable for the video pipeline.
Two compounding causes:
1. **Alpine + glibc mismatch.** The npm `@ffmpeg-installer/ffmpeg`
dependency added with the video-support PR (commit 68a9dc5)
ships per-platform binaries via optionalDependencies. The Linux
binaries are built against glibc, but the backend image runs on
`node:22-alpine` (musl libc) — known to either fail to execute
or fail on shared-library lookups on Alpine.
2. **`ffprobe` missing entirely.** `@ffmpeg-installer/ffmpeg`
bundles only the `ffmpeg` binary. There's a separate
`@ffprobe-installer/ffprobe` package that the codebase never
depended on. But `videoProcessor.js:21` calls
`ffmpeg.ffprobe(videoPath, …)` — the very first step of the
video pipeline shells out to a `ffprobe` binary that doesn't
exist in the image. Even if (1) worked, every video upload
would 500 here.
The fix is to install Alpine's `ffmpeg` package via apk. It ships
both `ffmpeg` and `ffprobe` built natively against musl, ~70MB
extra image size, single line in the Dockerfile, no per-arch
handling needed (apk pulls the right binary for both linux/amd64
and linux/arm64 — works with the multi-arch infra from #349).
- `backend/Dockerfile`: add `ffmpeg` to the apk install line.
- `backend/Dockerfile.dev`: same for dev parity.
- `backend/src/services/videoProcessor.js`: remove the
`setFfmpegPath(require('@ffmpeg-installer/ffmpeg').path)` line
— without removing it, fluent-ffmpeg would prefer the broken
bundled binary over the working apk one. Letting fluent-ffmpeg
fall back to PATH lookup picks up the apk binary in the
container and the developer's locally-installed binary on dev
hosts (Homebrew on macOS, apt on Debian).
- `backend/package.json`: drop the now-unused
`@ffmpeg-installer/ffmpeg` dependency. `npm install` removes
2 packages from the lockfile.
Verified: `videoProcessor.js` still loads cleanly (`node -e
"require('./src/services/videoProcessor')"`); lint clean.
Two follow-ups to PR #372 (external-URL toggle for imprint /
privacy CMS pages):
1. **i18n.** PR #372 added 6 new `cms.*` keys to the en + de
locales but the project ships 5 locales total. Adds the missing
nl / pt / ru translations so the admin CMS page renders in the
active language for those users instead of falling back to
English literals next to the German/Dutch/Portuguese/Russian
surrounding strings.
2. **API shape.** `publicCMS.js` returned `external_url`
unconditionally — even when `use_external_url` is false the URL
value was still emitted in the public response. The frontend
correctly gated on both flags so it worked, but the API surface
was leaking a value the admin had explicitly disabled. The
value still lives in the DB (so the toggle can be flipped back
on without losing it), but the public endpoint now returns
`null` whenever the toggle is off.
Note: kept the existing `logo_url` shape unchanged. Its semantics
are different — null means "fall back to global branding" and
consumers rely on always having the field, so emitting it
unconditionally is intentional there.
No frontend change needed: both `GalleryLayout` and `LegalPage`
already gate on `use_external_url && external_url`, so the
short-circuit handles `external_url: null` correctly.
The rebuilt modal in this PR shipped with hard-coded English strings.
That made the reset flow untranslated for German/Dutch/Portuguese/
Russian customers — toasts, confirm dialog, success screen all
fell back to English regardless of the active locale.
- New `events.passwordReset.*` namespace in en/de/nl/pt/ru with 22
keys covering both modal screens, the warning banner, validation
errors, and the toast messages.
- Modal uses `useTranslation()` for every previously hard-coded
string. Reuses `common.cancel`, `events.copy`, `events.copied`
where they already exist across all locales.
- The {{eventName}} interpolation uses i18next's standard variable
syntax so the description line reads naturally in each language.
No behaviour change. TypeScript clean (`npx tsc --noEmit`), ESLint
clean. JSON validity checked for all 5 locale files.
Two related defects on the same gallery-email surface that PR #367
opened, addressed together:
1. Reset-password endpoint was a one-way auto-generate.
`POST /admin/events/:id/reset-password` always called
`generateReadablePassword()` and ignored any client-supplied value;
the modal only offered a confirm + a forced auto-generated result.
Admins who wanted to set a memorable customer-supplied password
had no way to do it.
Backend: route now reads optional `password` from the body. If
present, validates with `validatePasswordInContext('gallery', …)`
(same rules as create-event) and uses it; if absent, falls back to
the existing generator, so old callers / cron stay functional.
Switched the bcrypt rounds from a hard-coded `10` to
`getBcryptRounds()` to match the create flow.
Frontend: rebuilt `PasswordResetModal.tsx`. Typed input with
show/hide, confirm-password field that appears on type, the same
`<PasswordGenerator>` used by `CreateEventPage` (event-context-
aware, fills both fields when used), send-email checkbox,
client-side validation, server-side validation feedback inline.
Submit empty → server auto-generates and the success screen shows
the value with a copy button (legacy one-click flow preserved);
submit with a typed password → success toast + close (no need to
re-show what the admin already typed).
Service layer: `events.service.resetPassword(id, sendEmail,
password?)` only sends `password` in the body when set.
Caller: `EventDetailsPage` now passes `eventDate` + `eventType`
into the modal so the generator has event context.
2. `gallery_link` was the path-only `event.share_link` in three
email-queue sites, so customer mail showed
`/gallery/<slug>/<token>` instead of the full
`https://example.com/gallery/<slug>/<token>` URL.
- `adminEvents.js` reset-password queue (#1437)
- `adminEvents.js` resend-creation-email queue (#1502)
- `expirationChecker.js` expiration_warning queue (#82)
All three now derive `shareUrl` from `buildShareLinkVariants`
(the same helper already used by create-event, publish-from-
draft, and event-rename). The other 4 callers
(`adminEvents.js:651/913`, `events.js:187`,
`eventRenameService.js:231`) already used the full URL — this
closes the gap.
Verified: TypeScript clean (`npx tsc --noEmit`), ESLint clean on
every touched file (the 4 lint errors that remain in
`adminEvents.js` are pre-existing and predate this branch).
Bundle of email-renderer and email-caller fixes triggered by a
reproducer on picpeak.nothaft.cloud (gallery_created mail showing
literal `{{#if welcome_message}}` markers and `Passwort: (set at
creation)`). The audit that followed surfaced six more user-visible
defects in the same surface; all are fixed here so customer-facing
mail renders cleanly.
Renderer (`backend/src/services/emailProcessor.js`)
- `safeTemplateReplace` now resolves `{{#if VAR}}…{{/if}}` blocks
before flat `{{var}}` substitution. The shipped templates have used
Handlebars-style conditionals since migration 026; the renderer
ignored them, so the markers leaked verbatim into every mail with
an empty welcome_message. Lifted to module scope and exported so
the conditional contract is unit-testable. Single-pass, non-nested
(commented).
- Added `passwordSetAtCreationI18n` next to the existing two i18n
password sentinels so `(set at creation)` (sent by the publish-
from-draft flow when only the bcrypt hash remains) is localised
to "Das bei der Erstellung der Galerie gesetzte Passwort" /
equivalent in EN/DE/NL/PT/RU instead of the raw English string.
- Added an opt-in `{ escapeHtml: true }` mode to `safeTemplateReplace`
so admin-supplied free text (`event_name`, `host_name`, …) is
HTML-escaped on substitution into the HTML body. Allowlist of
passthrough keys (`welcome_message` already-HTML, server-generated
URLs `gallery_link` / `client_link`). Subject and text body keep
the legacy unescaped behaviour. `formatWelcomeMessage` now escapes
before nl2br so the welcome_message allowlist is safe.
- New `htmlToText()` strips `<style>` and `<script>` blocks (and
their content) before tag-stripping, decodes common entities, and
collapses whitespace. Used by the textBody fallback in
`sendTemplateEmail` — without this, every template missing a
`body_text` produced a "plain-text" mail starting with the 100+
lines of CSS embedded by `wrapEmailHtml()`.
- The client-access section (#172) now mirrors its HTML block into
`textBody` using the same per-language strings, so plain-text
recipients see the link / PIN / warning. `pinLabel = 'PIN'` moved
into `clientAccessI18n` (RU uses ПИН-код).
- Added `getSupportEmail()` exported helper that reads
`branding_support_email` from `app_settings` (JSON-decoded), with
the SMTP from-address as fallback. Used by the gallery_expired and
archive_complete callers below.
- Removed dead `require('handlebars')` (unused since the regex
renderer landed; pre-existing lint error in this file).
Callers (data the templates already reference)
- `expirationChecker.js queueExpirationWarning`: send `expiry_date`
(templates use this, the old code sent `expiration_date` —
typo'd key, never read), drop the hard-coded `.de`/`en` sniff
(the processor formats with the recipient's resolved language),
add the `{{password_security_message}}` sentinel for
`gallery_password` (plaintext is gone by warning time, so
customers used to see literal `{{gallery_password}}` in the mail).
- `expirationChecker.js handleExpiredEvent`: both queueEmail calls
now supply `host_name`, `event_date`, `expiry_date`,
`support_email` so the EN/DE/NL/PT/RU `gallery_expired` template
doesn't render literal `{{host_name}}, your gallery expired on
{{expiry_date}}`. Skip the duplicate admin send when
admin_email == customer_email.
- `archiveService.js`: `archive_complete` queue now supplies
`host_name`, `photo_count` (from `photoEntries.length`),
`archive_date`, `support_email` — the previous payload had only
`event_name` and `archive_size`, so most of the mail was
unfilled placeholders.
Tests
- `__tests__/services/emailProcessor.safeTemplateReplace.test.js`:
16 cases — flat substitution, conditional truthy/falsy/missing/
multi-line/sibling/numeric-0, plus 5 cases for the new
`escapeHtml` option (default off, escape on, allowlist
passthrough for welcome_message and gallery_link).
- `__tests__/services/emailProcessor.htmlToText.test.js`: 7 cases —
the regression scenario (full wrapped body with embedded `<style>`
block), tag-stripping, entity decoding, paragraph spacing.
- `__tests__/utils/formatters.test.js`: 12 cases for `escapeHtml`,
`nl2br`, and the now-escaping `formatWelcomeMessage`.
35 cases total, all green. Lint clean on every touched file
(also fixes a pre-existing `no-prototype-builtins` warning in the
process). Pre-existing failures in
`__tests__/services/backupService.enhanced.test.js` are unrelated
and pre-date this branch.
Cover the two new pieces of the async pipeline:
backgroundProcessor.claimNextPhoto
- returns null when no pending rows
- returns the row + flips status under postgres FOR UPDATE SKIP LOCKED
- returns null when SQLite UPDATE-with-guard loses the race
- returns the row when the SQLite guard wins
photoProcessor.processPhoto
- happy path: writes thumbnail / dimensions / EXIF capture date and
marks 'complete'; fires watermark queue + photo.uploaded webhook
with the right payload
- video path: writes ffmpeg duration / codec / dimensions; does NOT
queue watermark (image-only)
- throws cleanly when the photo row no longer exists
Mocks db / imageProcessor / videoProcessor / storage / sharp /
watermarkGeneratorService / webhookService / logger so the tests run
without a real DB or any image library calls — fast and deterministic.
Live processing-state UI that complements the backend async pipeline.
Modal stays open through the processing phase and surfaces real
progress (X of N photos processed); the admin grid renders placeholder
cards for in-flight photos and auto-refreshes via polling until the
queue drains.
services/uploads.service.ts (new)
- getStatus(uploadId) — JSON snapshot from /admin/uploads/:id/status
- retryPhoto(photoId) — POST /admin/photos/:id/retry
- streamUrl(uploadId) — SSE upgrade URL
hooks/useUploadProgress.ts (new)
- Tracks N concurrent upload IDs (one per chunk POST) and merges
counters into a single aggregate.
- Always polls every 1.5s; opportunistic SSE upgrade on top of that.
SSE failure (proxy buffering, etc.) silently downgrades to polling
only — no reconnect storms.
- Auto-stops both channels when every tracked group is in a terminal
(complete/failed) state.
components/admin/PhotoUpload.tsx
- Captures upload_id from each chunk's 202 response, feeds them into
useUploadProgress.
- Phase machine extended: stays in 'processing' until the worker
drains the queue (not just until bytes-on-wire). Progress UI shows
real "X of N done" with a determinate bar fed by the aggregate.
- "You can leave this page" hint kept — closing the modal is now
actually safe, work continues server-side.
- Side-effect refactor: invokes onUploadComplete twice — once early
so the user sees photos appearing immediately, once on terminal
so the parent grid sees final state.
components/admin/AdminPhotoGrid.tsx
- Photos with processing_status pending/processing render an amber
placeholder card with a spinning Cog instead of the missing
thumbnail.
- Photos with status='failed' render a red card with the error message
and a "Retry" button that POSTs /admin/photos/:id/retry.
pages/admin/EventDetailsPage.tsx
- Photo list query gains refetchInterval that polls every 2s while
any photo is non-terminal, then stops. Keeps the grid auto-fresh
during ongoing processing.
Move thumbnail / EXIF / dimensions / watermark / webhook work off the
upload request thread and into a background worker pool. Upload
requests now return 202 in seconds even on NFS-backed storage; the
worker(s) drain the pending queue independently and update each
photo's processing_status to 'complete' or 'failed' on its own.
Schema (migration 085_async_photo_processing.js):
- photos.processing_status enum default 'complete' (existing
rows are already done)
- photos.processing_error populated on 'failed'
- photos.processing_started_at timestamp for janitor recovery
- photos.upload_id groups all photos from one upload
request so the frontend can poll
status by group
- indexes on processing_status and upload_id for queue lookups
services/photoProcessor.js
- queueFilesForProcessing(files, options) — shared helper used by
the admin and gallery upload routes. Moves files to final storage
+ inserts pending rows; returns { uploadId, photos, errors }.
- processPhoto(photoId) — worker-mode: reads original from storage
via withLocalCopy (transparent local/S3), generates thumbnail and
EXIF/dimensions or video metadata, queues watermark, fires
photo.uploaded webhook, marks 'complete'. Throws => caller marks
'failed' with the error message.
- processUploadedPhotos kept untouched — chunkedUploadService still
uses the synchronous path.
services/backgroundProcessor.js (new)
- N independent worker loops per backend instance (default 2,
UPLOAD_PROCESSOR_CONCURRENCY env override).
- Multi-pod safe: postgres SELECT FOR UPDATE SKIP LOCKED, sqlite
UPDATE-with-status-guard. Pods race on rows, exactly one wins.
- Janitor every minute resets photos stuck in 'processing' for >10
minutes (worker died, pod restarted) back to 'pending'.
- UPLOAD_PROCESSOR_DISABLED=true opt-out for CI/test.
- Started from server.js after the other long-running workers.
routes/adminPhotos.js — POST /:eventId/upload
- Replaced batch-of-25 sync processing loop with per-file
move-to-storage + insert-pending. Response is now 202 with
upload_id, count, photo_ids in addition to the legacy
successCount / replacedCount fields the existing frontend reads.
- Per-request temp directory cleanup is now a single idempotent
handler on res.finish/res.close (was three inline blocks for
error paths only, leaking dirs on success — original bug from
contributor analysis).
- GET /uploads/:upload_id/status — JSON snapshot of pending /
processing / complete / failed counts plus per-photo state.
- GET /uploads/:upload_id/stream — SSE upgrade. Polls internally
every 1.5s, emits on snapshot change, ends when all photos
reach a terminal state.
- POST /photos/:photoId/retry — flips a 'failed' photo back to
'pending' so the worker picks it up again.
- GET /:eventId/thumbnail/:photoId now returns 503 with Retry-After
while the photo is still pending/processing, and 422 on 'failed'.
The admin grid renders placeholders accordingly.
routes/gallery.js — POST /:eventId/upload (guest)
- Refactored to use queueFilesForProcessing instead of the synchronous
processUploadedPhotos. Same 202 + upload_id shape.
- GET /:slug/photos now filters processing_status to 'complete' (or
NULL for pre-migration rows) so guests never see in-flight photos.
Side-effect timing change:
- photo.uploaded webhook now fires from the worker after the photo
is actually processed (thumbnail + dimensions populated) instead
of from inside the upload request. Same payload fields. Worth a
one-line note in the changelog.
Phase 1 of the upload-progress redesign. Two changes that ship UX wins
without any architectural surgery — they're a stepping stone for the
full async-processing rework that follows in subsequent commits.
1. Two-state progress bar (PhotoUpload.tsx, UserPhotoUpload.tsx)
When axios.onUploadProgress reports loaded === total, the request is
on the server and the bytes have left the browser. Today the bar sits
at 100% for the chunk while the backend runs sharp/ffmpeg/EXIF (often
minutes on NFS-backed storage) and users assume the upload froze.
The component now distinguishes two phases:
- 'transferring' — bytes-on-wire, determinate progress bar.
- 'processing' — bytes done, waiting for response. Indeterminate
spinner + an explanatory hint that the backend is
generating thumbnails / reading metadata and the
user can leave the page.
Same pattern in UserPhotoUpload (gallery): the per-file checkmark
icon is replaced by a Loader2 spinner while the request is in flight
after bytes-on-wire finished.
2. Temp directory cleanup (adminPhotos.js)
Multer creates temp/upload_<ts>_<rand>/ per request. Files inside it
are individually unlinked after they're moved to storage on the
success path, but the empty directory was never removed. On error
paths three different inline blocks each tried to clean up; the
success path was missed entirely. Result: the orphan-empty-dirs
accumulation reported in the issue (70+ on the affected instance).
Replace the inline cleanup blocks with a single idempotent
cleanupTempDir() registered on res.finish + res.close, so it fires
exactly once on every exit path (validation 4xx, server 5xx, multer
error, success).
New translation keys (en/de): upload.transferring, upload.processing,
upload.processingHint, upload.processingProgress, upload.processingFailed,
upload.retryFailed.
Second loop fix in the same /admin/login → /admin/dashboard → /admin/login
pattern as #355. The frontend trusts /auth/session as the source of
truth for "is the user authenticated?". When that endpoint is more
lenient than the protected middleware, every admin endpoint 401s
right after /auth/session said valid:true, the response interceptor
hard-redirects to /admin/login, /auth/session says valid again, and
the cycle closes — exactly the loop reported on v3.32.4-beta.0.
#355 fixed the issuer-claim asymmetry. This commit fixes the
remaining asymmetries: /auth/session was missing the admin-existence,
admin-active, password-change-after-iat, and gallery-existence /
gallery-archived / gallery-expired checks that adminAuth and
galleryAuth perform on every protected request.
The fix is to mirror those checks in /auth/session, scoped by token
type, and degrade gracefully when the underlying tables aren't
present (test fixtures, early bootstrap) so the endpoint never
fails-closed because of a missing table.
Reproducer that the new test covers:
1. Admin logs in (token issued at T).
2. Admin (or another admin) changes their own password at T+1.
3. Browser still has the cookie from T.
4. /auth/session says valid:true (no password-change check).
5. /admin/dashboard fires queries; adminAuth rejects with
PASSWORD_CHANGED 401.
6. Frontend redirects to /admin/login.
7. /auth/session says valid:true again. → loop.
Other surfaces this also covers:
- admin user deactivated (admin_users.is_active = false)
- admin user deleted
- gallery token whose event is archived
- gallery token whose event has expired
Tests live in __tests__/routes/authSession.symmetry.test.js — 9 cases,
mocking db / tokenRevocation / tokenUtils / recaptcha / sessionTimeout
so the suite runs without a real database.
The root devDependencies still pinned an older Playwright (1.48.2)
plus a stray `dotenv` that nothing in the e2e suite or root scripts
actually requires (verified via grep across tests/). Updates the
Playwright version to match the current upstream stable and removes
the unused dotenv to keep the root install lean.
Originated from a local stash that picked up these changes; landing
them as a small dedicated commit so they don't blend into the auth
fix that follows.
Two further fixes for the gallery loading sequence shown in
@Rekoo-PS's frame breakdown on issue #358 — both about colours that
didn't track the active theme.
1. Initial white frame (frame f1)
The pre-React bootstrap script in #359 sets the cached background
on documentElement, but the browser may paint the very first frame
*before* that <script> tag runs (synchronous parse-time JS in the
<head> is still slightly later than CSS apply-time). On first-visit
dark-OS devices that meant a single white frame before the script
resolved.
Fix: move the OS-preference default into a <style> block that
precedes the script. CSS @media (prefers-color-scheme) is applied
before paint, so dark-OS devices land on dark from frame zero.
The script keeps the per-gallery cache hit on top, and now also
stamps the colour onto document.body in case the body element has
already mounted by the time the script runs.
2. "Most annoying" skeleton tile frame (frame f4)
Skeleton placeholders rendered as bright `bg-neutral-200` light grey
regardless of theme. On a dark gallery that's the highest-contrast
thing on screen during loading — the exact frame Rekoo-PS labelled
"the most annoying" in the issue.
Fix: the Skeleton component's background now reads
`var(--color-surface-border)`, which ThemeContext already wires up
per active theme (`#e5e5e5` light / `#2e2e2e` dark by default; per-
event themes can override). The bare `<div>` no longer carries any
colour utility class — the inline style supplies the active value.
Also dropped the leftover `bg-white` on SkeletonCard / SkeletonTable
in favour of `var(--color-surface)` for the same reason.
Tests:
New src/components/common/__tests__/Skeleton.test.tsx covers
- Skeleton uses var(--color-surface-border, ...)
- bg-neutral-200 is no longer present
- SkeletonGalleryGrid tiles all inherit the theme colour
- SkeletonCard surface uses var(--color-surface)
Paul Nothaft