11 Commits

Author	SHA1	Message	Date
Paul Nothaft	8cc7d7d14a	feat(external-media): watch reference folders and import new files automatically (#1345) ... * feat(external-media): watch reference folders and import new files automatically Managed uploads dropped into storage/events/active are picked up by the chokidar watcher; external media had no equivalent, so a NAS folder that keeps growing needed an admin to open the event and press Import every time. Relates to issue 1187. - The import pass moves out of the route into services/externalImportService.js. The watcher and the Import button now run the identical function; the route only validates and maps errors to status codes. - Mutual exclusion is the per-event claim from maintenanceJobState (`external_import:<id>`, seeded on demand by the new ensure()) instead of the in-process Set. The Set stopped a double-click in one process; the claim also stops the watcher on a second replica, or an admin clicking while the watcher is mid-run elsewhere. The run heartbeats so a claim from a dead process is taken over. - services/externalMediaWatcher.js: per-event opt-in via the new events.external_watch column (migration 208), chokidar with awaitWriteFinish so a copy in flight is not imported half-written, debounced full pass per change, a timer sweep every 15 minutes as the fallback for NFS/SMB mounts that deliver no inotify events, optional stat-polling via EXTERNAL_MEDIA_WATCH_POLLING. The set of watched events is re-read every minute, so the toggle works from any replica. A watcher that just started runs one pass immediately. - Deletions are ignored on purpose: a file vanishing from a NAS is at least as likely to be a reorganisation or a dropped mount as an intentional removal, and acting on it would delete a guest-visible photo. Rows whose file is gone stay, as they do today. - Not gated on STORAGE_BACKEND: EXTERNAL_MEDIA_ROOT is always local. - Quiet system passes stay out of the activity log; runs that imported something are logged with actor external-media-watcher. - Frontend: "Watch folder for new files" checkbox under the external folder picker, status line in view mode, EN/DE strings. * fix(external-media): close the review gaps in the folder watcher Codex review of the watcher, round 1. All six findings were real: - Enabling the watcher, or pointing an enabled one at another folder, now requires photos.upload — the permission the manual Import already requires. events.edit alone was a way around it. Only the transition is checked, so a role without photos.upload can still edit an already-watched event. The checkbox is disabled for such roles. - Automatic passes defer files that are still changing: anything modified inside the stability window, or whose size moves across one wait of that window, is left for the next pass. chokidar's awaitWriteFinish only settles the file that fired the event, and the sweep sees no events at all, so a sibling still being copied could be inserted half-written and then skipped forever. - Photos an admin deleted are not brought back by the sweep. The delete routes record the file in external_import_exclusions (migration 209); automatic passes skip the list, the manual Import ignores it and clears it for what it imports. - The six EXTERNAL_MEDIA_WATCH* variables are forwarded in all three compose files; they were documented but the backend services use explicit environment lists, so the kill switch did nothing. - A pass re-checks is_active / is_archived at run time, not only in the minutely reconcile. - The lease is renewed on a timer for the whole run, walk included, and ownership is checked before the event row is touched. * fix(external-media): make automatic passes follow the row, not rewrite it Codex review round 2, four findings, all applied: - The event update route drops non-canonical spellings of external_watch and external_path before the permission guard. SQLite resolves column names case-insensitively, so `External_Watch` reached the column while the guard only looked at the lowercase key. - Exclusions are checked per file at insert time, not against a snapshot taken before the settle wait. A photo deleted during the wait was present in the snapshot and got re-inserted by the loop. - An automatic pass no longer writes source_mode / external_path. It re-reads the row after the walk and the settle wait and stops if the folder changed or the event went managed; the manual Import is the only writer. The options are now `automatic` + `settleMs`. - A pass that deferred files re-arms the debounced import, so a file copied just before the watcher started is not stranded when the sweep is disabled. * fix(external-media): keep exclusions for replaced photos, stop a pass whose event stopped qualifying Codex review round 3, both findings applied: - recordExclusions keys on external_relpath alone. A replaced external photo becomes managed but keeps its relpath on purpose, and deleting that replacement must not republish the NAS original. - An automatic pass checks the full watcher predicate (reference mode, same folder, watch on, active, not archived) before it inserts and on every heartbeat tick during the loop, and stops as soon as the event no longer qualifies. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-07 22:29:55 +02:00
Paul Nothaft	b53e5d97b4	feat: add opt-in product usage and feedback integration (#1110)	2026-09-05 12:59:06 +02:00
Paul Nothaft	8337a716b1	fix(file-watcher): bound concurrent photo processing (#846) ... * fix(file-watcher): bound concurrent photo processing chokidar fires 'add' once per file — with no ignoreInitial option the boot scan fires it for every existing file, and a bulk drop into the watch folder fires it for every new one at once. Each handler runs DB lookups plus (for new files) a full sharp pipeline; sharp.concurrency(2) only caps libvips threads WITHIN one operation, not the number of parallel pipelines, so unbounded handlers can OOM small hosts. Gate both 'add' and 'unlink' through a shared p-limit (FILE_WATCHER_CONCURRENCY, default 2, floor 1) — mass deletes otherwise burst DB work and ZIP-cache invalidation the same way. p-limit is pinned to ^3.1.0, the last CommonJS release. Adapted from the filpgame fork (426ca491) — thanks @filpgame; extended to cover 'unlink', documented in .env.example, plus a lock-in test for the existing Sharp cache/concurrency caps this bound relies on. * chore(compose): pass FILE_WATCHER_CONCURRENCY into the backend container (codex review of #846) The backend service uses an explicit environment list (no env_file), so the documented override never reached the container in the default compose deployments. Added to both compose files + root .env.example.	2026-07-19 22:00:46 +02:00
Paul Nothaft	72e2ef6721	feat(auth): admin TOTP MFA — enrollment, login challenge, recovery, CLI reset ... Backend for #738. Real TOTP 2FA for admin accounts, all roles incl. super_admin (closes #735). - mfaService: otplib TOTP; AES-256-GCM encryption of the secret at rest (key derived from MFA_ENCRYPTION_KEY or JWT_SECRET); bcrypt-hashed, single-use recovery codes; otpauth URI + QR. - Migration 151: adds two_factor_recovery_codes + two_factor_enrolled_at (secret/enabled columns already existed from legacy 016). - Enrollment endpoints (behind adminAuth, per-user): GET /mfa/status, POST /mfa/{setup,enable,disable,recovery-codes}. Disable/regenerate require a current code so a hijacked session can't strip 2FA. - Login challenge: /admin/login returns {mfaRequired, mfaToken} (no session) when 2FA is on; /admin/login/mfa exchanges a TOTP or recovery code for the session. Lockout counter is NOT reset until the second factor passes, so MFA brute-force is rate-limited too. - CLI break-glass: scripts/reset-admin-mfa.js --email <e> | --all --yes, audit-logged, matches reset-admin-password.js convention. - Docs + optional MFA_ENCRYPTION_KEY env. Verified end-to-end on a live backend: enroll (super_admin), challenge, TOTP + single-use recovery login, disable, and CLI reset.	2026-07-03 11:33:38 +02:00
Paul Nothaft	5c7de96b7f	fix(auth): default COOKIE_SECURE to 'auto' in production + first-install UX (#427) ... Two intertwined bugs reported in #427 by @iSchumi6210: 1. Login silently fails over HTTP. Backend defaulted COOKIE_SECURE to true when NODE_ENV=production. Over plain HTTP the browser drops the Secure cookie → next /auth/session request returns 401 → redirect back to /admin/login → no error shown. picpeak-setup.sh writes NODE_ENV=production but never writes COOKIE_SECURE, so every first-time install without a reverse proxy hits this. 2. Admin password is generated but admins can't find it. The 001_init.js migration writes the generated password to data/ADMIN_CREDENTIALS.txt inside the backend container, but picpeak-setup.sh only copies it out when --reset-admin-password is passed. Default-path users never see it and resort to manual bcrypt updates in psql. Changes: - tokenUtils.js: production default goes from `true` to `'auto'`. On real HTTPS req.secure is true → Secure flag is still emitted (no security regression for reverse-proxy deployments). On plain HTTP req.secure is false → Secure flag omitted → login works. Users who explicitly want the strict HTTPS-only behaviour can still set COOKIE_SECURE=true. - .env.example: rewrite the COOKIE_SECURE block to make the new default obvious and explain when to override (set =true for strict, =false to skip the per-request check, leave unset for the auto behaviour). - picpeak-setup.sh (both Docker and native paths): - Write COOKIE_SECURE=auto explicitly to the generated .env (defense in depth so the right behaviour is preserved even if the backend default flips again later) - After migrations, ALWAYS copy ADMIN_CREDENTIALS.txt out of the backend container/data dir to the host data dir, chmod 600, and print the email + password to the install output. The credentials file remains as a backup record that the operator should delete after noting the password. Verified locally with all 4 permutations of NODE_ENV × COOKIE_SECURE: production, unset → HTTPS: secure=true ✓ HTTP: secure=false ✓ (was both true) production, =true → both: secure=true (strict opt-in preserved) production, =auto → HTTPS: secure=true HTTP: secure=false (already-correct) development, unset → both: secure=false (dev unchanged)	2026-05-09 15:55:09 +02:00
Paul Nothaft	15a8ab41fd	feat: add COOKIE_SECURE=auto for mixed HTTPS/HTTP deployments (#298) ... Adds a third value for the COOKIE_SECURE environment variable that decides the cookie Secure flag per-request based on req.secure. This unblocks a common self-hosted setup where the same PicPeak deployment is reachable over both HTTPS (via reverse proxy) and plain HTTP (e.g. LAN access at http://192.168.x.x:3001). Behavior unset - legacy default: follows NODE_ENV (production=true, dev=false) true - always set Secure (unchanged) false - never set Secure (unchanged) auto - NEW: use req.secure per request. In practice this means Secure on HTTPS requests (when X-Forwarded-Proto: https reaches Express via a trusted proxy) and no Secure flag on plain HTTP requests. The existing trust proxy config (`app.set('trust proxy', 'loopback, linklocal, uniquelocal')` in server.js) means X-Forwarded-Proto is honored when forwarded from local/private-network proxies, which covers Docker network setups and most self-hosted deployments behind NPM, Traefik, or Caddy. auto is strictly opt-in. The default behavior is unchanged, so existing users see no difference. A follow-up release can consider promoting auto to the default after real-world feedback. Also fixed (latent bug, benefits everyone) Cookie clear operations (clearAdminAuthCookie, clearGalleryAuthCookies) previously wrote the same `secure` attribute as the set path. When a cookie was set with Secure=true over HTTPS and the clear request came over HTTP (or vice versa under auto mode), some browsers would reject the Set-Cookie delete header, leaving the cookie in place. Browsers match cookies by (name, domain, path) for deletion and don't care about Secure, so the new buildClearCookieOptions() helper simply omits the secure attribute. Implementation - secureCookie string is replaced by secureCookieMode which can hold true, false, or 'auto'. - New resolveSecureFlag(res) returns the boolean for a specific response, delegating to res.req.secure when in auto mode. - buildCookieBaseOptions and buildCookieOptionsWithExpiry now take res and pass it through. - New buildClearCookieOptions() deliberately omits `secure`. - setAdminAuthCookie / setGalleryAuthCookies / clearAdminAuthCookie / clearGalleryAuthCookies all updated to thread res where needed. Public signatures unchanged — every caller already has res in scope. Testing Verified against a real Express instance inside the backend container with trust proxy configured, covering: - (unset) + NODE_ENV=production -> secure: true (legacy) - (unset) + NODE_ENV=development -> secure: false (legacy) - COOKIE_SECURE=true + req.secure=false -> secure: true (literal wins) - COOKIE_SECURE=false + req.secure=true -> secure: false (literal wins) - COOKIE_SECURE=auto + X-Forwarded-Proto: https -> secure: true - COOKIE_SECURE=auto + plain HTTP -> secure: false - clearCookie always omits the secure attribute Documentation Added a COOKIE_SECURE block to both .env.example files (root for docker-compose, backend/.env.example for native install) explaining the four values, when to use auto, and the two requirements (proxy must forward X-Forwarded-Proto, proxy IP must be in the trust list). Also documented COOKIE_SAMESITE and COOKIE_DOMAIN alongside, which were previously undocumented.	2026-04-11 22:41:15 +02:00
Paul Nothaft	3397807670	docs: emphasize importance of STORAGE_PATH in env example	2026-01-17 15:48:24 +01:00
Paul Nothaft	3e69579f5a	docs: add API_URL environment variable to .env.example files ... Document the API_URL environment variable that is used for constructing URLs for assets (logos, images) in email notifications. Without this setting, the system defaults to http://localhost:3001 which causes broken images in production emails. Added to both root and backend .env.example files with clear documentation about its purpose and importance.	2026-01-16 09:39:32 +01:00
paul	a209796b16	refactor: complete configuration cleanup and consistency fixes ... - Create docker-compose.dev.yml with Mailhog for development email testing - Standardize all configurations to use PORT=3001 for backend - Fix database service naming (postgres → db) across all files - Add missing BACKEND_URL environment variable to all configs - Update .env examples to match actual Docker setup requirements - Remove orphaned postgres-init directory (Umami handles its own DB) - Update README roadmap: mark gallery feedback as implemented, add multi-admin support - Update deployment guide with development setup instructions - Fix frontend Dockerfile.dev for proper hot-reload development - Remove unused files (wedding-photos.db, frontend/README.md) This ensures all configuration files are consistent and aligned with the deployment guide.	2025-07-24 21:13:52 +02:00
paul	4ed35f1b16	chore: consolidate and update deployment documentation ... - Remove completed PRODUCTION_TODO_LIST.md - Consolidate deployment guides: keep comprehensive PRODUCTION_DEPLOYMENT_GUIDE.md, remove redundant PRODUCTION_DEPLOYMENT.md - Update all .env.example files to reflect current system: - Remove deprecated ADMIN_EMAIL/ADMIN_PASSWORD (now auto-generated) - Add proper documentation for all environment variables - Clarify that Umami config is optional (primary via Admin UI) - Add realistic examples for SMTP providers - Update ports to match actual defaults (3001) - Update PRODUCTION_DEPLOYMENT_GUIDE.md: - Document auto-generated admin credentials process - Add Traefik configuration section - Update security checklist with current features - Fix outdated environment variables - Add nginx proxy configuration details 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-07-24 16:57:07 +02:00
paul	1773ed5f95	Initial commit - Project start (July 17, 2025) ... Original: feat: enhance security logging and ensure rate limit blocks are properly tracked - Add comprehensive logging for rate limit blocks with full request details - IP address (with proper proxy detection), user agent, headers, timestamps - Rate limit info (current count, limit, remaining, reset time) - Separate tracking for auth vs general endpoints - Enhance authentication failure logging - JWT validation failures with detailed error info - Admin auth attempts without token - Failed token validation with user context - All events include IP, path, method, user agent - Improve Winston logger configuration for production - Add automatic log rotation (10MB errors, 50MB combined) - Create separate security.log for auth/rate limit events - Ensure logs directory exists automatically - Add structured JSON format for log aggregation - Support container logging with LOG_TO_CONSOLE env var - Create comprehensive documentation - Security logging guide with examples - Monitoring recommendations - Configuration reference - Add test script to verify logging functionality All rate limit settings remain configurable via admin panel: - Window duration, max requests, auth limits - Skip authenticated requests option - Public endpoints only option 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-07-24 16:57:07 +02:00