paul/picpeak
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity
1,710 Commits 86 Branches 516 Tags
64e4925fbb60052eb3bc29d3774a04414d249120
Commit Graph
9 Commits
Author SHA1 Message Date
Paul Nothaft e4e79a0b3a fix(archives): stream-extract restore for >2 GiB + preserve original_filename via manifest (#640) 
Two related backup-integrity fixes from 8digit's fork (issue #640 items
#3 + #4), bundled because they touch the same two files and ship better
together than apart.

### Stream-extract restore for >2 GiB archives

`adminArchives.js:170` was using `adm-zip`, which loads the entire ZIP
into a Node Buffer before extracting. Node has a hard 2 GiB Buffer cap,
so any restore over that limit fails with `ERR_FS_FILE_TOO_LARGE` — and
since the frontend `onError` toast is the generic "Something went wrong",
the cause stays invisible. Real-world wedding archives routinely cross
2 GiB; affected restores have likely been silent failures.

Swapped `adm-zip` for `node-stream-zip` which streams each entry to disk
as it's processed — no full-file Buffer, no 2 GiB ceiling. API shape:

```js
const zip = new StreamZip.async({ file: archivePath });
const entries = Object.values(await zip.entries());
await zip.extract(null, eventDir);
await zip.close();
```

Re-import logic (photos, categories, sizes) unchanged; only field rename
`entry.entryName` → `entry.name`. Credit: 8digit/picpeak@69033c6.

### Preserve `original_filename` via photos manifest

Archive → restore round-trip currently loses `original_filename` (the
post-#508 column tracking the camera-side name) because the gallery
filenames are renamed on upload and can't be derived from the extracted
files. This matters now that the Lightroom export (#623) depends on
`original_filename` — a restored event lost that signal.

- **`archiveService.js`**: writes `photos_manifest.json` into the archive
  containing per-photo `{filename, original_filename, type, uploaded_at,
  category_name}`. Non-fatal: a manifest write failure falls through to
  legacy behaviour (filename used as original_filename, same as before).
- **`adminArchives.js`**: reads the manifest on restore, builds a
  `Map<filename → manifest>`, and assigns
  `original_filename = manifest?.original_filename || filename`.
  Archives produced before this lands have no manifest — restore logs a
  one-shot notice and falls back to filename, preserving backward compat.

Credit: 8digit/picpeak@eb018aa.

### Deps

- Removed `adm-zip ^0.5.16`
- Added `node-stream-zip ^1.15.0`

### What's NOT in this PR

8digit's commit also fixed the production compose healthcheck (`curl`
isn't in our Alpine image); that's already been addressed upstream in
the meantime. The frontend `onError` swallow on the restore toast is a
separate small follow-up.

### Test plan

- [x] `node -c` on both files clean
- [x] `node-stream-zip` async API verified at load time
- [ ] Manual: archive a multi-GB event → restore → confirm photos
      re-import with original_filename preserved
- [ ] Manual: restore an archive produced before this lands → confirm
      fallback to filename works (no manifest path crashes)
- [ ] Manual: confirm the new photos_manifest.json is inside the
      generated archive (`unzip -l <archive>.zip | grep manifest`)
 2026-06-18 22:18:54 +02:00
Paul Nothaft 7eeef2ba98 feat(downloads): preserve original camera filenames on download (opt-in) (#493) 
New Settings → General toggle `Use original filenames on download` (off by
default). When on, single-photo downloads, bulk/selection zips, and per-event
archive zips surface `photos.original_filename` instead of the sanitized
storage filename. Storage paths are unchanged.

- Content-Disposition uses RFC 5987 (`filename=` ASCII + `filename*=UTF-8''…`)
  so unicode camera filenames survive while header-injection bytes are stripped.
- Zip entries are deduplicated with a deterministic `_1` / `_2` suffix on
  collision (folder structure preserved in archive zips).
- Pre-generated download-all zips and the in-memory setting cache are
  invalidated when the toggle flips so the next download rebuilds with the
  new names.
- Falls back to the storage filename whenever `original_filename` is null
  (legacy uploads predating migration 062).
 2026-05-14 23:11:00 +02:00
Paul Nothaft 61f1d13210 feat(lightbox): medium-resolution preview tier (#492) 
Adds an opt-in lightbox preview tier so guests open photos against an
aspect-preserved ~1920px JPEG (~200–500 KB) instead of the full original
(often 5–12 MB). Originals are still served on Download.

Backend:
  - imageProcessor: generatePreviewImage / isPreviewValid / ensurePreviewImage
    using fit:'inside' + withoutEnlargement (longEdge 1920, q85, mozjpeg)
  - migration 104: photos.preview_path + lightbox_preview_enabled setting
    (off by default, JSON-stringified for SQLite/Postgres parity)
  - GET /api/gallery/:slug/preview/:photoId — gallery-auth, lazy generation,
    ETag based on mtime+photoId+watermarkHash
  - preview_url surfaced in the photo response only when the toggle is on
  - admin /thumbnails/regenerate-previews mirrors regenerate-thumbnails,
    skipping videos
  - backup walk + archive cleanup + photo-delete now include previews/

Frontend:
  - PhotoLightbox uses photo.preview_url ?? photo.url (null-safe fallback)
  - ThumbnailsTab gets a Lightbox Preview Tier card: opt-in toggle +
    Regenerate All Previews button (gated until the toggle is on)
  - en/de locale strings; nl/pt/ru/fr fall back to en

Tested end-to-end: 11 MB / 4000×3000 source → 985 KB / 1920×1440 preview,
381 ms first call, 7 ms cached, ~91% byte reduction.
 2026-05-14 22:30:39 +02:00
Paul Nothaft e8052adf1d fix(email): render conditionals, localise password placeholders, fix caller/template variable drift 
Bundle of email-renderer and email-caller fixes triggered by a
reproducer on picpeak.nothaft.cloud (gallery_created mail showing
literal `{{#if welcome_message}}` markers and `Passwort: (set at
creation)`). The audit that followed surfaced six more user-visible
defects in the same surface; all are fixed here so customer-facing
mail renders cleanly.

Renderer (`backend/src/services/emailProcessor.js`)

- `safeTemplateReplace` now resolves `{{#if VAR}}…{{/if}}` blocks
  before flat `{{var}}` substitution. The shipped templates have used
  Handlebars-style conditionals since migration 026; the renderer
  ignored them, so the markers leaked verbatim into every mail with
  an empty welcome_message. Lifted to module scope and exported so
  the conditional contract is unit-testable. Single-pass, non-nested
  (commented).

- Added `passwordSetAtCreationI18n` next to the existing two i18n
  password sentinels so `(set at creation)` (sent by the publish-
  from-draft flow when only the bcrypt hash remains) is localised
  to "Das bei der Erstellung der Galerie gesetzte Passwort" /
  equivalent in EN/DE/NL/PT/RU instead of the raw English string.

- Added an opt-in `{ escapeHtml: true }` mode to `safeTemplateReplace`
  so admin-supplied free text (`event_name`, `host_name`, …) is
  HTML-escaped on substitution into the HTML body. Allowlist of
  passthrough keys (`welcome_message` already-HTML, server-generated
  URLs `gallery_link` / `client_link`). Subject and text body keep
  the legacy unescaped behaviour. `formatWelcomeMessage` now escapes
  before nl2br so the welcome_message allowlist is safe.

- New `htmlToText()` strips `<style>` and `<script>` blocks (and
  their content) before tag-stripping, decodes common entities, and
  collapses whitespace. Used by the textBody fallback in
  `sendTemplateEmail` — without this, every template missing a
  `body_text` produced a "plain-text" mail starting with the 100+
  lines of CSS embedded by `wrapEmailHtml()`.

- The client-access section (#172) now mirrors its HTML block into
  `textBody` using the same per-language strings, so plain-text
  recipients see the link / PIN / warning. `pinLabel = 'PIN'` moved
  into `clientAccessI18n` (RU uses ПИН-код).

- Added `getSupportEmail()` exported helper that reads
  `branding_support_email` from `app_settings` (JSON-decoded), with
  the SMTP from-address as fallback. Used by the gallery_expired and
  archive_complete callers below.

- Removed dead `require('handlebars')` (unused since the regex
  renderer landed; pre-existing lint error in this file).

Callers (data the templates already reference)

- `expirationChecker.js queueExpirationWarning`: send `expiry_date`
  (templates use this, the old code sent `expiration_date` —
  typo'd key, never read), drop the hard-coded `.de`/`en` sniff
  (the processor formats with the recipient's resolved language),
  add the `{{password_security_message}}` sentinel for
  `gallery_password` (plaintext is gone by warning time, so
  customers used to see literal `{{gallery_password}}` in the mail).

- `expirationChecker.js handleExpiredEvent`: both queueEmail calls
  now supply `host_name`, `event_date`, `expiry_date`,
  `support_email` so the EN/DE/NL/PT/RU `gallery_expired` template
  doesn't render literal `{{host_name}}, your gallery expired on
  {{expiry_date}}`. Skip the duplicate admin send when
  admin_email == customer_email.

- `archiveService.js`: `archive_complete` queue now supplies
  `host_name`, `photo_count` (from `photoEntries.length`),
  `archive_date`, `support_email` — the previous payload had only
  `event_name` and `archive_size`, so most of the mail was
  unfilled placeholders.

Tests

- `__tests__/services/emailProcessor.safeTemplateReplace.test.js`:
  16 cases — flat substitution, conditional truthy/falsy/missing/
  multi-line/sibling/numeric-0, plus 5 cases for the new
  `escapeHtml` option (default off, escape on, allowlist
  passthrough for welcome_message and gallery_link).

- `__tests__/services/emailProcessor.htmlToText.test.js`: 7 cases —
  the regression scenario (full wrapped body with embedded `<style>`
  block), tag-stripping, entity decoding, paragraph spacing.

- `__tests__/utils/formatters.test.js`: 12 cases for `escapeHtml`,
  `nl2br`, and the now-escaping `formatWelcomeMessage`.

35 cases total, all green. Lint clean on every touched file
(also fixes a pre-existing `no-prototype-builtins` warning in the
process). Pre-existing failures in
`__tests__/services/backupService.enhanced.test.js` are unrelated
and pre-date this branch.
 2026-05-03 22:06:06 +02:00
Paul Nothaft 1e69d5ff71 feat(webhooks): enrich event.* payloads with customer contact + share_token (#341) 
The event.published webhook reporter wired into n8n to send WhatsApp
gallery links was missing the data needed to actually message the
customer — only event_name + share_url were in the payload, no
customer_name / customer_email / customer_phone, and no bare share
token to construct alternate URLs.

Adds a single canonical event subject helper (webhookService.buildEventSubject)
so every event.* webhook returns the same shape:

  { id, slug, event_name, event_type, event_date,
    share_url, share_token,
    customer_name, customer_email, customer_phone }

Fields the caller does not have in scope come back as null — keys are
always present so receivers do not have to distinguish "field missing"
from "field null". Pure addition: existing receivers continue to work,
existing templates ${data.event.event_name} keep working, and new
templates can now reference ${data.event.customer_phone} etc.

Wired into all five firing sites:
- routes/events.js — public event create (created + published)
- routes/adminEvents.js — admin create + draft→publish
- routes/v1/events.js — public v1 API (created + published)
- services/expirationChecker.js — event.expired (extra: expires_at)
- services/archiveService.js — event.archived (extra: archive_path)

PII surface area widens (customer email/phone now flow to webhook
receivers), so:
- Settings → Webhooks UI gets an amber Callout above the create form
  warning admins to only point webhooks at receivers they trust.
- Docs page updated with the new payload sample, the always-present
  null contract, and a Callout warning.

Verified end-to-end against the local dev webhook receiver — delivered
payload contains all 10 fields. webhookDelivery integration suite
remains 8/8 green.
 2026-04-29 20:32:20 +02:00
Paul Nothaft 1b717ce5ed feat: native S3 storage backend (#328) + presigned download follow-up 
Lets PicPeak write photos, thumbnails, hero images, watermarks, and
archive zips to any S3-compatible bucket (AWS S3, MinIO, Cloudflare R2,
Backblaze B2, Wasabi, DigitalOcean Spaces) instead of the local
filesystem. Selected via STORAGE_BACKEND=local|s3.

Architecture
- backend/src/services/storage/StorageBackend.js — abstract interface
  (put/get/exists/stat/delete/list/copy/rename/signedUrl/putFromFile/
  getToFile) — typedef-only, documents the contract.
- LocalFsStorage.js — wraps fs with atomic-write-via-tmp-rename, path
  traversal protection, list-as-walker.
- S3StorageBackend.js — thin wrapper around the existing
  S3StorageAdapter (used by backupService) mapping it onto the canonical
  interface; supports optional STORAGE_S3_PREFIX namespace.
- index.js — factory selected by STORAGE_BACKEND with startup ping
  (HEADs sentinel key on S3, fs.stat on local) so misconfig fails fast
  before the first request.

Consumer refactors (~12 services + routes), each parametrized over the
abstraction:
- imageProcessor / videoProcessor — pipe Sharp/ffmpeg output through
  storage.put; expose withLocalCopy() helper for S3-mode regeneration
  paths that need a local file for sharp/ffmpeg.
- archiveService / downloadZipService — finalize zip in tmp dir, then
  storage.putFromFile. Atomic-rename pattern preserved on local; S3
  emulates via copy + delete (worker prunes orphaned .tmp.* on startup).
- photoProcessor / photoReplacementService / adminPhotos upload+delete /
  routes/v1/events.js POST /events/:id/photos / routes/events.js — every
  upload path now goes storage.putFromFile(temp) → unlink temp.
- gallery.js bulk-download (cached + on-the-fly + selected) — managed
  photos via storage.get, external-mode unchanged.
- protectedImages / secureImages / photoResolver — read via
  storage.get; resolvePhotoStorageKey returns the canonical key.
- watermarkService / watermarkGeneratorService — persistent watermarks
  via storage.put.
- fileWatcher — bails out with a clear log warning when STORAGE_BACKEND=s3
  (chokidar can't watch S3); auto-import lands via the S3 prefix walker
  introduced in the follow-up commit.
- expirationChecker — small touch (event.expired webhook fire from #327
  shipping in the next commit).

Migration tooling
- backend/scripts/migrate-storage.js — one-shot --dry-run capable script
  that walks photos.path, thumbnail_path, hero_path, watermark_path and
  events.archive_path/download_zip_path; streams local → S3; sha256
  size-match skip for idempotent re-run; failures CSV.

Presigned-URL "Download All" (#328 follow-up shipped in this commit)
- routes/gallery.js — when STORAGE_BACKEND=s3 + event.allow_presigned_download
  + downloads enabled + watermark NOT enabled, /download-all returns a
  302 redirect to a 5-minute presigned S3 URL. Per-event opt-in surface
  ships in the next commit's UI.

Tests
- backend/__tests__/integration/storageBackend.test.js — parametrized
  contract suite running against BOTH LocalFs AND MinIO (18 tests, both
  backends — 36 cases total).
- backend/__tests__/integration/imageProcessor.storage.test.js — same
  parametrized pattern for the image processor (10 tests × 2 backends).
- backend/__tests__/integration/backup-s3.test.js — bootstrap fix:
  drop the redundant initDb() (001_init handles it) and remove
  schema-drift in configureS3Backup (app_settings has no created_at
  anymore and the unique constraint is on setting_key alone, not
  composite). 0/12 → 7/12 (5 remaining are unrelated assertion drift).
- backend/src/services/photoResolver.js — mixed-source events (reference
  mode with managed-uploaded photos) now fall back to managed when
  external_relpath is missing instead of throwing.
- tests/e2e/s3-storage-roundtrip.spec.ts — Playwright spec that
  auto-skips against local backend; full upload → serve → delete
  round-trip when run against an S3-mode backend.

Server wiring (server.js)
- initStorage() called after database init, before rate limiters.
- This commit's diff also includes the webhook delivery worker startup
  and the S3 auto-importer startup. Those features ship in the next two
  commits — co-located here for one bisectable diff per file.

Docs + ops
- README §"Storage Backends" — capability matrix, switching playbook,
  IAM policy snippet, MinIO/R2/B2 examples.
- README §"Webhooks" — also added here (full diff bundled).
- .env.example — STORAGE_BACKEND + STORAGE_S3_* + STORAGE_AUTO_IMPORT
  documented; WEBHOOK_* added in the same diff.
- .gitignore — re-anchor the existing `storage/` rule to `/storage/`
  so backend/src/services/storage/ (the new abstraction code) is
  trackable. The runtime ./storage/ data dir stays ignored.

Out of scope for v1 (per the issue): presigned URLs for individual
photo display (always streamed for protection middleware), CDN
integration, hybrid hot/cold tiers, S3 → local migration, multi-bucket
per-event.
 2026-04-28 10:06:36 +02:00
Paul Nothaft e4b0f961b7 fix: prevent backend crash on archive when admin_email is null (#318) 
Archiving an event with no admin_email queued an email_queue row with
recipient_email=null, violating the NOT NULL constraint. The error was
thrown inside the output.on('close') callback (detached from the caller),
becoming an unhandled rejection that crashed Node and dropped admin
sessions on bulk archive.

- Skip queueEmail when event.admin_email is null/empty (admin_email has
  been nullable since migration 073).
- Wrap the close handler in try/catch so any post-archive failure logs
  instead of crashing the process.
 2026-04-26 22:15:00 +02:00
paulandClaude dc1419c051 feat: implement gallery feedback system with version tracking for backups 
Gallery Feedback Features:
- Add feedback system allowing ratings, likes, comments, and favorites on photos
- Implement admin controls for enabling/disabling feedback per event
- Add content moderation with word filters and spam detection
- Implement rate limiting to prevent abuse (10 requests/15min per type)
- Create comprehensive admin interface for feedback management
- Add analytics dashboard for feedback insights
- Export feedback data when archiving events

Frontend Components:
- PhotoRating: 5-star rating system with optimistic updates
- PhotoLikes: Like/unlike with animation
- PhotoComments: Threaded comments with moderation
- PhotoFavorites: Bookmark functionality
- FeedbackSettings: Admin configuration panel
- EventFeedbackPage: Complete management interface

Backend Implementation:
- Database migration 033: 4 new tables for feedback system
- RESTful API with proper authorization
- Guest identification via SHA256(IP+UserAgent)
- Automatic backup integration
- Email notification support

Backup Version Tracking:
- Migration 034: Add version columns to backup tables
- Track app version, Node.js version, and DB schema version
- Create restore_history table for tracking restore attempts
- Add version compatibility checking for safe restores
- Configurable version matching requirements

Security & Performance:
- Input validation and sanitization
- Rate limiting per feedback type
- Content moderation system
- Optimistic UI updates
- Efficient database queries with proper indexes

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
 2025-07-24 16:57:08 +02:00
paulandClaude 1773ed5f95 Initial commit - Project start (July 17, 2025)
continuous-integration/drone/push Build is passing
Details
Mirror to GitHub / mirror (push) Successful in 26s
Details
Test and Lint / backend-test (push) Successful in 1m11s
Details
Test and Lint / frontend-test (push) Successful in 2m28s
Details
Version and Release / version-bump (push) Successful in 32s
Details
Version and Release / trigger-drone (push) Has been skipped
Details 
Original: feat: enhance security logging and ensure rate limit blocks are properly tracked

- Add comprehensive logging for rate limit blocks with full request details
  - IP address (with proper proxy detection), user agent, headers, timestamps
  - Rate limit info (current count, limit, remaining, reset time)
  - Separate tracking for auth vs general endpoints

- Enhance authentication failure logging
  - JWT validation failures with detailed error info
  - Admin auth attempts without token
  - Failed token validation with user context
  - All events include IP, path, method, user agent

- Improve Winston logger configuration for production
  - Add automatic log rotation (10MB errors, 50MB combined)
  - Create separate security.log for auth/rate limit events
  - Ensure logs directory exists automatically
  - Add structured JSON format for log aggregation
  - Support container logging with LOG_TO_CONSOLE env var

- Create comprehensive documentation
  - Security logging guide with examples
  - Monitoring recommendations
  - Configuration reference

- Add test script to verify logging functionality

All rate limit settings remain configurable via admin panel:
- Window duration, max requests, auth limits
- Skip authenticated requests option
- Public endpoints only option

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
 2025-07-24 16:57:07 +02:00