96818c7ae8de0d8fd478cd901ea25a3272eee85d
12
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Paul Nothaft
|
e8052adf1d |
fix(email): render conditionals, localise password placeholders, fix caller/template variable drift
Bundle of email-renderer and email-caller fixes triggered by a
reproducer on picpeak.nothaft.cloud (gallery_created mail showing
literal `{{#if welcome_message}}` markers and `Passwort: (set at
creation)`). The audit that followed surfaced six more user-visible
defects in the same surface; all are fixed here so customer-facing
mail renders cleanly.
Renderer (`backend/src/services/emailProcessor.js`)
- `safeTemplateReplace` now resolves `{{#if VAR}}…{{/if}}` blocks
before flat `{{var}}` substitution. The shipped templates have used
Handlebars-style conditionals since migration 026; the renderer
ignored them, so the markers leaked verbatim into every mail with
an empty welcome_message. Lifted to module scope and exported so
the conditional contract is unit-testable. Single-pass, non-nested
(commented).
- Added `passwordSetAtCreationI18n` next to the existing two i18n
password sentinels so `(set at creation)` (sent by the publish-
from-draft flow when only the bcrypt hash remains) is localised
to "Das bei der Erstellung der Galerie gesetzte Passwort" /
equivalent in EN/DE/NL/PT/RU instead of the raw English string.
- Added an opt-in `{ escapeHtml: true }` mode to `safeTemplateReplace`
so admin-supplied free text (`event_name`, `host_name`, …) is
HTML-escaped on substitution into the HTML body. Allowlist of
passthrough keys (`welcome_message` already-HTML, server-generated
URLs `gallery_link` / `client_link`). Subject and text body keep
the legacy unescaped behaviour. `formatWelcomeMessage` now escapes
before nl2br so the welcome_message allowlist is safe.
- New `htmlToText()` strips `<style>` and `<script>` blocks (and
their content) before tag-stripping, decodes common entities, and
collapses whitespace. Used by the textBody fallback in
`sendTemplateEmail` — without this, every template missing a
`body_text` produced a "plain-text" mail starting with the 100+
lines of CSS embedded by `wrapEmailHtml()`.
- The client-access section (#172) now mirrors its HTML block into
`textBody` using the same per-language strings, so plain-text
recipients see the link / PIN / warning. `pinLabel = 'PIN'` moved
into `clientAccessI18n` (RU uses ПИН-код).
- Added `getSupportEmail()` exported helper that reads
`branding_support_email` from `app_settings` (JSON-decoded), with
the SMTP from-address as fallback. Used by the gallery_expired and
archive_complete callers below.
- Removed dead `require('handlebars')` (unused since the regex
renderer landed; pre-existing lint error in this file).
Callers (data the templates already reference)
- `expirationChecker.js queueExpirationWarning`: send `expiry_date`
(templates use this, the old code sent `expiration_date` —
typo'd key, never read), drop the hard-coded `.de`/`en` sniff
(the processor formats with the recipient's resolved language),
add the `{{password_security_message}}` sentinel for
`gallery_password` (plaintext is gone by warning time, so
customers used to see literal `{{gallery_password}}` in the mail).
- `expirationChecker.js handleExpiredEvent`: both queueEmail calls
now supply `host_name`, `event_date`, `expiry_date`,
`support_email` so the EN/DE/NL/PT/RU `gallery_expired` template
doesn't render literal `{{host_name}}, your gallery expired on
{{expiry_date}}`. Skip the duplicate admin send when
admin_email == customer_email.
- `archiveService.js`: `archive_complete` queue now supplies
`host_name`, `photo_count` (from `photoEntries.length`),
`archive_date`, `support_email` — the previous payload had only
`event_name` and `archive_size`, so most of the mail was
unfilled placeholders.
Tests
- `__tests__/services/emailProcessor.safeTemplateReplace.test.js`:
16 cases — flat substitution, conditional truthy/falsy/missing/
multi-line/sibling/numeric-0, plus 5 cases for the new
`escapeHtml` option (default off, escape on, allowlist
passthrough for welcome_message and gallery_link).
- `__tests__/services/emailProcessor.htmlToText.test.js`: 7 cases —
the regression scenario (full wrapped body with embedded `<style>`
block), tag-stripping, entity decoding, paragraph spacing.
- `__tests__/utils/formatters.test.js`: 12 cases for `escapeHtml`,
`nl2br`, and the now-escaping `formatWelcomeMessage`.
35 cases total, all green. Lint clean on every touched file
(also fixes a pre-existing `no-prototype-builtins` warning in the
process). Pre-existing failures in
`__tests__/services/backupService.enhanced.test.js` are unrelated
and pre-date this branch.
|
||
|
Paul Nothaft
|
b30010eb6f |
test(upload): unit tests for backgroundProcessor + processPhoto
Cover the two new pieces of the async pipeline:
backgroundProcessor.claimNextPhoto
- returns null when no pending rows
- returns the row + flips status under postgres FOR UPDATE SKIP LOCKED
- returns null when SQLite UPDATE-with-guard loses the race
- returns the row when the SQLite guard wins
photoProcessor.processPhoto
- happy path: writes thumbnail / dimensions / EXIF capture date and
marks 'complete'; fires watermark queue + photo.uploaded webhook
with the right payload
- video path: writes ffmpeg duration / codec / dimensions; does NOT
queue watermark (image-only)
- throws cleanly when the photo row no longer exists
Mocks db / imageProcessor / videoProcessor / storage / sharp /
watermarkGeneratorService / webhookService / logger so the tests run
without a real DB or any image library calls — fast and deterministic.
|
||
|
Paul Nothaft
|
f905f7e733 |
fix(auth): /auth/session must reject tokens that adminAuth/galleryAuth would reject
Second loop fix in the same /admin/login → /admin/dashboard → /admin/login pattern as #355. The frontend trusts /auth/session as the source of truth for "is the user authenticated?". When that endpoint is more lenient than the protected middleware, every admin endpoint 401s right after /auth/session said valid:true, the response interceptor hard-redirects to /admin/login, /auth/session says valid again, and the cycle closes — exactly the loop reported on v3.32.4-beta.0. #355 fixed the issuer-claim asymmetry. This commit fixes the remaining asymmetries: /auth/session was missing the admin-existence, admin-active, password-change-after-iat, and gallery-existence / gallery-archived / gallery-expired checks that adminAuth and galleryAuth perform on every protected request. The fix is to mirror those checks in /auth/session, scoped by token type, and degrade gracefully when the underlying tables aren't present (test fixtures, early bootstrap) so the endpoint never fails-closed because of a missing table. Reproducer that the new test covers: 1. Admin logs in (token issued at T). 2. Admin (or another admin) changes their own password at T+1. 3. Browser still has the cookie from T. 4. /auth/session says valid:true (no password-change check). 5. /admin/dashboard fires queries; adminAuth rejects with PASSWORD_CHANGED 401. 6. Frontend redirects to /admin/login. 7. /auth/session says valid:true again. → loop. Other surfaces this also covers: - admin user deactivated (admin_users.is_active = false) - admin user deleted - gallery token whose event is archived - gallery token whose event has expired Tests live in __tests__/routes/authSession.symmetry.test.js — 9 cases, mocking db / tokenRevocation / tokenUtils / recaptcha / sessionTimeout so the suite runs without a real database. |
||
|
Paul Nothaft
|
e232f9f2cf |
fix(backup): incremental backups against S3 + jsonb stats parsing
Three fixes uncovered while bringing the backup-s3 integration suite to 12/12 against MinIO + Postgres: - backupService.getDatabaseBackupInfo: pg's jsonb driver auto-parses `statistics` / `table_checksums` to objects; the old JSON.parse() then threw "[object Object]" is not valid JSON and the manifest dropped database info silently. Accept both string and object inputs. - backupService.runBackup: incremental path called backupManifest.loadManifest() with an s3:// URI directly, which falls through to fs.readFile() and ENOENTs — every "incremental" backup silently downgraded to a full one. Added loadManifestFromAnywhere() helper that downloads s3:// to a tmp file before delegating. - backupManifest.generateIncrementalManifest: attached the `incremental` section AFTER generateManifest() had already stamped verification.total_checksum, so every incremental manifest failed validateManifest() on read-back. Recompute the checksum after. Test side: updated assertions to the current manifest shape (`incremental.changes.modified_files_count`), Number()-coerce bigint columns from pg, and gate the logger mock on UNMOCK_LOGGER for diagnosing similar silent-failure modes in the future. |
||
|
Paul Nothaft
|
ab4095f592 |
fix(backup): cron schedule mapping + manifest format detection + bigint coerce
Three pre-existing bugs surfaced by re-running the backup-s3 integration
suite. backup-s3 went 0/12 → 7/12 (storage-refactor session bootstrap
fixes) → 10/12 with this commit.
1. Backup service crashes on backend startup with
`TypeError: Cannot read properties of undefined (reading 'replace')`
from node-cron's expression parser.
Root cause: `backup_schedule` stores a UI label like "weekly", while
`backup_schedule_cron` stores the actual cron expression. Startup
code read the label and passed it straight to cron.schedule() —
"weekly" is not a cron expression.
Fix in startBackupService(): read backup_schedule_cron first; fall
back to mapping known labels (hourly/daily/weekly/monthly) to cron
expressions; back-compat for deployments that wrote a cron expression
into the legacy backup_schedule field.
2. Backup manifest retrieval fails with
`SyntaxError: Unexpected token 'a', "applicatio"...` when the
manifest format is YAML.
Root cause: getBackupManifest() downloads the s3:// manifest to a
tmp file hardcoded as `manifest-N.json`. loadManifest() then
detects format from extension only — sees .json, runs JSON.parse on
YAML content (which starts with "application: …"), fails.
Fix in backupManifest.loadManifest(): detect format from BOTH the
extension AND the content's first non-whitespace character. JSON
starts with { or [; anything else falls through to yaml.load.
Backwards compatible — extension is still authoritative when present
AND content matches.
3. Test assertion `expect(backupRun.total_size_bytes).toBeGreaterThan(0)`
fails with "received value must be a number or bigint" because pg
driver returns bigint columns as strings. Coerce via Number() in
the test.
Remaining 2 failures (out of scope here, both are spec-level drift):
- "should include database backup" expects the runBackup() flow to
upload the database backup file at S3 key `database/db-backup.sql`.
Current implementation reads db backup metadata for the manifest but
does not upload the file itself. Missing feature, not a test bug.
- "should only upload changed files" expects manifest.incremental.
modified_files_count. Implementation writes backupType: 'incremental'
on the run row but no per-run incremental subobject in the manifest.
Field shape mismatch.
|
||
|
Paul Nothaft
|
c488f481ca |
feat: outbound webhooks for event/photo lifecycle (#327)
PicPeak POSTs lifecycle notifications to admin-configured URLs. Each delivery is signed HMAC-SHA256 in the X-PicPeak-Signature header. Verified end-to-end: 1/1 Playwright spec, 8/8 backend integration tests, full UI click-through via Chrome DevTools. Schema (migration 082) - webhooks: id, name, url, secret (plaintext — required to compute HMAC for every outbound POST), secret_preview, events[], active, filter, template, created_by, timestamps, last_success_at/last_failure_at. - webhook_deliveries: webhook_id (FK CASCADE), event_type, payload, attempt_count, status (pending|success|failed), response_status, response_body (truncated to 1KB), latency_ms, next_retry_at, last_error, created_at, completed_at. Composite index (status, next_retry_at) serves the worker's hot-path query. Service + worker - webhookService.fire(eventType, data) — non-throwing entry point used by lifecycle hooks. Looks up active webhooks subscribed to the event and applies their per-webhook filter (dot-path equality predicate) before enqueueing one webhook_deliveries row per match. Filter and template logic ship in this commit; admin surfaces in the follow-up. - webhookDeliveryWorker — setInterval(5s) poller; fetches up to 5 pending rows; per delivery: re-validates URL via networkValidation (DNS-rebinding mitigation, opt-out via WEBHOOK_ALLOW_PRIVATE_URLS), signs body with HMAC-SHA256, POSTs with 10s timeout, records outcome. Backoff schedule: 1m → 5m → 30m → 2h → 12h, max 5 attempts. Response body truncated to 1KB before storage. If a webhook has a template, the rendered string replaces the JSON envelope as the request body (signature is computed over the bytes actually sent). Lifecycle wiring - adminEvents.js POST /events → event.created (+ event.published when not draft); POST /:id/publish → event.published. - routes/events.js (legacy public POST) → event.created + event.published. - routes/v1/events.js (#322 API) → event.created + event.published on create, photo.uploaded on photo POST. - archiveService.archiveEvent() → event.archived. Per-photo photo.deleted intentionally NOT fired during cascade — receivers infer from event.archived to avoid flooding (issue spec). - expirationChecker.handleExpiredEvent() → event.expired BEFORE the cascading archive (so receivers see expired→archived in order). - adminPhotos.js — photo.uploaded on each batch row, photo.deleted on single + bulk delete. - photoProcessor.js — photo.uploaded for guest uploads + auto-import (covers all entry paths). - fileWatcher.js — photo.uploaded on add, photo.deleted on unlink (local mode only). Admin endpoints (mirrors adminApiTokens.js pattern) - /api/admin/webhooks: GET list, POST create (returns plaintext secret exactly once), GET :id, PUT :id, DELETE :id, POST :id/test (synthetic fire), GET :id/deliveries (paginated, filter by status), GET :id/deliveries/:deliveryId, POST :id/deliveries/:deliveryId/replay. Frontend - Settings → Webhooks tab (mirrors API Tokens layout): name + URL + event checkboxes + "Advanced" expander for filter (JSON) and template. Plaintext secret shown once on creation with a Copy button. Active/ Disabled toggle button per row. - /admin/webhooks/:id/deliveries — operational debug surface. Table with timestamp/event/status/attempts/HTTP/latency. Status filter chips (all/pending/success/failed). Row click → slide-over with payload + signature + response body. Replay button on failed rows. Send-test-event dialog. Auto-refresh every 10s. Dev infrastructure - dev/webhook-receiver/ — tiny node:alpine HTTP server (~100 LOC) that records every POST to an in-memory ring buffer. Exposes GET /requests for the E2E spec to assert deliveries landed with the right HMAC. Sibling pattern to MinIO. Reachable from the backend at http://webhook-receiver:8888 inside the picpeak network. Tests - backend/__tests__/integration/webhookDelivery.test.js (8/8) — signature verification, headers, retry/backoff, max-attempts → failed, response truncation, disabled-mid-flight, SSRF block, start/stop idempotency. - tests/e2e/webhooks-roundtrip.spec.ts (1/1) — create webhook → trigger event.published → assert receiver got POST with valid HMAC → visit deliveries page → row visible with status=success → API test event → API replay → disable webhook → assert no new delivery. Docs - README §"Webhooks" — event catalog, payload shape, HMAC verification in Node + Python + bash, retry semantics, SSRF protection. - .env.example — WEBHOOK_ALLOW_PRIVATE_URLS, WEBHOOK_DELIVERY_INTERVAL_MS, WEBHOOK_DELIVERY_CONCURRENCY, WEBHOOK_HTTP_TIMEOUT_MS, WEBHOOK_MAX_ATTEMPTS. Out of scope for v1 (per issue): webhook templates' code-eval (the ${dot.path} substitution that ships is pure string replacement, no expression engine — see follow-up commit), per-webhook rate limiting beyond the global concurrency cap, synchronous "ask before delete" webhooks. Spanning files - App.tsx pulls in this commit with both the AnalyticsBootstrap (#325 dedup) and the WebhookDeliveriesPage route registration. Splitting via git add -p was forfeit for sanity; the single 92-line diff is honest about both contributions. - adminEvents.js diff bundles the webhook fires AND the allow_presigned_download field plumbing (#328 follow-up). Same reasoning. - The new webhookService/Worker/adminWebhooks files include the filter and template logic from the follow-up — they were authored in one pass; splitting them post-hoc would have produced fragile partial files. The follow-up commit covers the migration and the UI for these. |
||
|
Paul Nothaft
|
1b717ce5ed |
feat: native S3 storage backend (#328) + presigned download follow-up
Lets PicPeak write photos, thumbnails, hero images, watermarks, and archive zips to any S3-compatible bucket (AWS S3, MinIO, Cloudflare R2, Backblaze B2, Wasabi, DigitalOcean Spaces) instead of the local filesystem. Selected via STORAGE_BACKEND=local|s3. Architecture - backend/src/services/storage/StorageBackend.js — abstract interface (put/get/exists/stat/delete/list/copy/rename/signedUrl/putFromFile/ getToFile) — typedef-only, documents the contract. - LocalFsStorage.js — wraps fs with atomic-write-via-tmp-rename, path traversal protection, list-as-walker. - S3StorageBackend.js — thin wrapper around the existing S3StorageAdapter (used by backupService) mapping it onto the canonical interface; supports optional STORAGE_S3_PREFIX namespace. - index.js — factory selected by STORAGE_BACKEND with startup ping (HEADs sentinel key on S3, fs.stat on local) so misconfig fails fast before the first request. Consumer refactors (~12 services + routes), each parametrized over the abstraction: - imageProcessor / videoProcessor — pipe Sharp/ffmpeg output through storage.put; expose withLocalCopy() helper for S3-mode regeneration paths that need a local file for sharp/ffmpeg. - archiveService / downloadZipService — finalize zip in tmp dir, then storage.putFromFile. Atomic-rename pattern preserved on local; S3 emulates via copy + delete (worker prunes orphaned .tmp.* on startup). - photoProcessor / photoReplacementService / adminPhotos upload+delete / routes/v1/events.js POST /events/:id/photos / routes/events.js — every upload path now goes storage.putFromFile(temp) → unlink temp. - gallery.js bulk-download (cached + on-the-fly + selected) — managed photos via storage.get, external-mode unchanged. - protectedImages / secureImages / photoResolver — read via storage.get; resolvePhotoStorageKey returns the canonical key. - watermarkService / watermarkGeneratorService — persistent watermarks via storage.put. - fileWatcher — bails out with a clear log warning when STORAGE_BACKEND=s3 (chokidar can't watch S3); auto-import lands via the S3 prefix walker introduced in the follow-up commit. - expirationChecker — small touch (event.expired webhook fire from #327 shipping in the next commit). Migration tooling - backend/scripts/migrate-storage.js — one-shot --dry-run capable script that walks photos.path, thumbnail_path, hero_path, watermark_path and events.archive_path/download_zip_path; streams local → S3; sha256 size-match skip for idempotent re-run; failures CSV. Presigned-URL "Download All" (#328 follow-up shipped in this commit) - routes/gallery.js — when STORAGE_BACKEND=s3 + event.allow_presigned_download + downloads enabled + watermark NOT enabled, /download-all returns a 302 redirect to a 5-minute presigned S3 URL. Per-event opt-in surface ships in the next commit's UI. Tests - backend/__tests__/integration/storageBackend.test.js — parametrized contract suite running against BOTH LocalFs AND MinIO (18 tests, both backends — 36 cases total). - backend/__tests__/integration/imageProcessor.storage.test.js — same parametrized pattern for the image processor (10 tests × 2 backends). - backend/__tests__/integration/backup-s3.test.js — bootstrap fix: drop the redundant initDb() (001_init handles it) and remove schema-drift in configureS3Backup (app_settings has no created_at anymore and the unique constraint is on setting_key alone, not composite). 0/12 → 7/12 (5 remaining are unrelated assertion drift). - backend/src/services/photoResolver.js — mixed-source events (reference mode with managed-uploaded photos) now fall back to managed when external_relpath is missing instead of throwing. - tests/e2e/s3-storage-roundtrip.spec.ts — Playwright spec that auto-skips against local backend; full upload → serve → delete round-trip when run against an S3-mode backend. Server wiring (server.js) - initStorage() called after database init, before rate limiters. - This commit's diff also includes the webhook delivery worker startup and the S3 auto-importer startup. Those features ship in the next two commits — co-located here for one bisectable diff per file. Docs + ops - README §"Storage Backends" — capability matrix, switching playbook, IAM policy snippet, MinIO/R2/B2 examples. - README §"Webhooks" — also added here (full diff bundled). - .env.example — STORAGE_BACKEND + STORAGE_S3_* + STORAGE_AUTO_IMPORT documented; WEBHOOK_* added in the same diff. - .gitignore — re-anchor the existing `storage/` rule to `/storage/` so backend/src/services/storage/ (the new abstraction code) is trackable. The runtime ./storage/ data dir stays ignored. Out of scope for v1 (per the issue): presigned URLs for individual photo display (always streamed for protection middleware), CDN integration, hybrid hot/cold tiers, S3 → local migration, multi-bucket per-event. |
||
 paul
|
775e417e55 | Fix admin reference mode regressions | ||
|
paul
|
fc1bf53412 | fix: harden gallery downloads and per-gallery auth | ||
|
paul
|
5f8c8c5508 | Fix branding asset storage path | ||
|
paul
|
2a4d38813f | feat: overhaul public landing page and backup tooling | ||
|
paulandClaude
|
f6a79c815e |
feat: implement comprehensive backup and restore system with S3 support
- Add S3/MinIO storage adapter with multipart upload support - Implement database backup service for SQLite and PostgreSQL - Create backup manifest generator for tracking backup contents - Enhance backup service with S3 integration and incremental backups - Add restore service with safety measures and rollback capability - Create comprehensive test suite for all backup functionality - Add admin API endpoints for backup/restore management - Implement frontend UI with dashboard, configuration, and restore wizard - Add roadmap section to README with implemented backup feature This implementation provides: - Multiple backup destinations (local, rsync, S3/MinIO) - Intelligent change detection to minimize backup frequency - Full database backups with compression - Manifest-based restore with integrity validation - Pre-restore safety backups with rollback - Comprehensive error handling and monitoring - User-friendly admin interface 🤖 Generated with Claude Code Co-Authored-By: Claude <[email protected]> |