30 Commits

Author	SHA1	Message	Date
Paul Nothaft	8cc7d7d14a	feat(external-media): watch reference folders and import new files automatically (#1345) ... * feat(external-media): watch reference folders and import new files automatically Managed uploads dropped into storage/events/active are picked up by the chokidar watcher; external media had no equivalent, so a NAS folder that keeps growing needed an admin to open the event and press Import every time. Relates to issue 1187. - The import pass moves out of the route into services/externalImportService.js. The watcher and the Import button now run the identical function; the route only validates and maps errors to status codes. - Mutual exclusion is the per-event claim from maintenanceJobState (`external_import:<id>`, seeded on demand by the new ensure()) instead of the in-process Set. The Set stopped a double-click in one process; the claim also stops the watcher on a second replica, or an admin clicking while the watcher is mid-run elsewhere. The run heartbeats so a claim from a dead process is taken over. - services/externalMediaWatcher.js: per-event opt-in via the new events.external_watch column (migration 208), chokidar with awaitWriteFinish so a copy in flight is not imported half-written, debounced full pass per change, a timer sweep every 15 minutes as the fallback for NFS/SMB mounts that deliver no inotify events, optional stat-polling via EXTERNAL_MEDIA_WATCH_POLLING. The set of watched events is re-read every minute, so the toggle works from any replica. A watcher that just started runs one pass immediately. - Deletions are ignored on purpose: a file vanishing from a NAS is at least as likely to be a reorganisation or a dropped mount as an intentional removal, and acting on it would delete a guest-visible photo. Rows whose file is gone stay, as they do today. - Not gated on STORAGE_BACKEND: EXTERNAL_MEDIA_ROOT is always local. - Quiet system passes stay out of the activity log; runs that imported something are logged with actor external-media-watcher. - Frontend: "Watch folder for new files" checkbox under the external folder picker, status line in view mode, EN/DE strings. * fix(external-media): close the review gaps in the folder watcher Codex review of the watcher, round 1. All six findings were real: - Enabling the watcher, or pointing an enabled one at another folder, now requires photos.upload — the permission the manual Import already requires. events.edit alone was a way around it. Only the transition is checked, so a role without photos.upload can still edit an already-watched event. The checkbox is disabled for such roles. - Automatic passes defer files that are still changing: anything modified inside the stability window, or whose size moves across one wait of that window, is left for the next pass. chokidar's awaitWriteFinish only settles the file that fired the event, and the sweep sees no events at all, so a sibling still being copied could be inserted half-written and then skipped forever. - Photos an admin deleted are not brought back by the sweep. The delete routes record the file in external_import_exclusions (migration 209); automatic passes skip the list, the manual Import ignores it and clears it for what it imports. - The six EXTERNAL_MEDIA_WATCH* variables are forwarded in all three compose files; they were documented but the backend services use explicit environment lists, so the kill switch did nothing. - A pass re-checks is_active / is_archived at run time, not only in the minutely reconcile. - The lease is renewed on a timer for the whole run, walk included, and ownership is checked before the event row is touched. * fix(external-media): make automatic passes follow the row, not rewrite it Codex review round 2, four findings, all applied: - The event update route drops non-canonical spellings of external_watch and external_path before the permission guard. SQLite resolves column names case-insensitively, so `External_Watch` reached the column while the guard only looked at the lowercase key. - Exclusions are checked per file at insert time, not against a snapshot taken before the settle wait. A photo deleted during the wait was present in the snapshot and got re-inserted by the loop. - An automatic pass no longer writes source_mode / external_path. It re-reads the row after the walk and the settle wait and stops if the folder changed or the event went managed; the manual Import is the only writer. The options are now `automatic` + `settleMs`. - A pass that deferred files re-arms the debounced import, so a file copied just before the watcher started is not stranded when the sweep is disabled. * fix(external-media): keep exclusions for replaced photos, stop a pass whose event stopped qualifying Codex review round 3, both findings applied: - recordExclusions keys on external_relpath alone. A replaced external photo becomes managed but keeps its relpath on purpose, and deleting that replacement must not republish the NAS original. - An automatic pass checks the full watcher predicate (reference mode, same folder, watch on, active, not archived) before it inserts and on every heartbeat tick during the loop, and stops as soon as the event no longer qualifies. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-09-07 22:29:55 +02:00
Paul Nothaft	b53e5d97b4	feat: add opt-in product usage and feedback integration (#1110)	2026-09-05 12:59:06 +02:00
Paul Nothaft	d62407f431	feat(email): webhook transport as an alternative to SMTP (#1225) (#1231) ... Setting EMAIL_WEBHOOK_URL makes PicPeak stop sending mail itself and POST each composed message as JSON instead, for something downstream (n8n, Make, a self-hosted relay) to deliver. Unset, every SMTP path is unchanged. Settles the four things #1225 left open: - SSRF: the URL goes through the same DNS-resolving check the outbound webhook worker uses, before every send. Private receivers are opt-in. - Transport security: https is required for anything leaving the machine. The HMAC proves who sent the body, not who can read it, and these bodies carry password-reset links and guest recovery codes. The private-network opt-in doubles as the plaintext opt-in. - Authentication: EMAIL_WEBHOOK_SECRET is required and signs the body as X-PicPeak-Signature, the same scheme as gallery webhooks. A URL without a secret leaves the transport OFF and says so once. - Attachments: carried as base64, not dropped. Oversized ones fail and stay queued rather than arriving without the invoice. Configuration is environment-only on purpose: this redirects every outbound message including password resets, so it must not be changeable from a compromised admin session. Three wiring details decide whether it works at all: docker-compose.yml declares an explicit environment block, so the vars had to be forwarded there; a fresh webhook-only install has no email_configs row (migration 001 seeds it only when SMTP_HOST is set), so the From identity falls back to EMAIL_FROM; and processEmailQueue used to return early when SMTP could not initialise, which would have left the queue permanently unprocessed. guestRecoveryService and the admin test-email endpoint were bypassing the transport — the first dereferenced a null transporter, the second told webhook-only admins to go configure SMTP. emailIntakeService deliberately stays on SMTP: it round-trips a specific mailbox's own credentials. Response handling is streamed and read bounded by hand rather than capped via axios: maxContentLength throws while reading, so a receiver that delivered the mail and then echoed a large body would have been recorded as failed and the message sent again. Note: docker-compose.dev.yml is gitignored and local-only, so the equivalent entries there are not part of this change. docker-compose.production.yml needs none — it passes .env through with env_file. Three rounds of external review; 21 transport tests, 61 across the email suites.	2026-08-29 11:10:32 +02:00
Paul Nothaft	5c85e0c0e4	fix(guests): surface duplicate guest registrations, and stop making so many (#1210) (#1216) ... * fix(guests): surface duplicate guest registrations, and stop making so many (#1210) Guest registration always inserts. A client whose token expired — or who opens the gallery on a second device — becomes a new gallery_guests row, and their likes and favourites split across the copies. The photographer's 'final selection' is then only trustworthy if somebody notices two Tinas with half the picks each. Two halves, neither of which touches the registration path. **Say which rows are the same person.** Merging already worked, endpoint and UI both; nothing said WHICH rows to merge. The guests list now marks each row with the others sharing its email and returns a count for the banner, and the admin list offers the group straight to the merge mode that already exists. Case-folded and trimmed, because the same person types Tina@ one day and tina@ the next and both read as distinct rows. Email only — two guests called Anna are not evidence of anything, and rows without an email are not grouped at all since require_name_email is off by default and a shared link produces plenty of them. It preselects rather than merges: which row survives decides the name and verification state the merged guest keeps, and that is the admin's call. **Create fewer of them.** The guest token was 24h and every call site took that default, so even the same browser lost its identity after a day of inactivity. Now 30 days, GUEST_TOKEN_TTL to override. A guest token is scoped to one event, carries no admin capability, and the gallery is already behind whatever protects it — 30 days is the shape of a real proofing cycle. Deliberately NOT done: reusing a guest row when a typed email matches, which the report suggests first. It would let anyone who knows an address inherit that person's identity and selections, and answering differently for a known email would leak which addresses are in the gallery — the thing /guest/recover already goes out of its way to avoid. Prevention at the entry path needs the verification round-trip, which is a separate decision about friction. 13 tests; 8 of the 9 backend ones fail without the change. The frontend ones caught a real bug while being written — the new useMemo sat after the loading early-return, so the hook count changed between renders. * fix(guests): merge must not strand a pending invite (#1210) Three findings from external review of #1216. **A merge could kill an emailed invite link.** Creating an invite inserts a real gallery_guests row, so an admin who pre-mints one and then sees the guest self-register has two rows sharing an email — which this feature now points out and offers to merge. Redemption resolves guest_invites.guest_id with is_deleted: false, so merging soft-deleted the row the link pointed at: the client got 404 guest_missing while the invite dialog still showed the invite as Pending. Nothing anywhere said the link was dead. Unredeemed, unrevoked invites now move to the survivor first. Spent ones stay put — a redeemed invite records who redeemed what, and retargeting it would rewrite that. **The preselection silently chose the survivor.** performMerge keeps mergeSelection[0], and the group was handed over in API order, which is newest-first — so Review then Merge discarded an older, email-verified row holding most of the picks in favour of a fresh re-registration. The proposal is now ordered deliberately: verified first, then whoever holds the most feedback, then the oldest. Still only a proposal, and the confirmation now names the survivor by email as well as name, because duplicates share a name and 'Merge 2 guests into Tina?' said nothing. **duplicate_of was quadratic.** Every row carried the other n-1 ids, so a group of n serialised n² of them — and nothing consumed the list: the UI asked only whether a row was in a group, then regrouped by email itself. Replaced with duplicate_group, the normalised email, which keeps the payload linear and the case/whitespace folding in one place instead of reimplemented on the client. Two new backend tests for the invite paths, one frontend test asserting the merge call keeps the verified row. The invite test fails against the un-fixed code. * fix(guests): keep guest-controlled input out of who survives a merge (#1210) Round 2 of external review on #1216. **The survivor ranking used an attacker-controlled signal.** Preferring whoever holds the most feedback looked like the obvious tiebreak and is exactly the wrong one: registration does not verify the address, so anyone who knows a guest's email can register with it, mark enough photos to out-rank the real person, and be preselected as the survivor. An admin accepting a confirmation between two rows with the same name and email would then move the victim's picks onto an identity whose token the visitor still holds. distinct_photos is guest-controlled and has no business deciding this. The ranking is now email_verified_at then created_at — both server-set. **A merge could make the survivor unrecoverable.** Rows are grouped with case and whitespace folded out, so a merge can be proposed between tina@example.com and Tina@Example.com. /guest/recover lowercases what the guest types and then matches on equality, so a survivor left holding the raw value can never be recovered by email again. The kept row's address is now canonicalised during the merge. Both write paths normalise today, so this covers rows that predate that — which are exactly the rows case-folded grouping surfaces. Two more backend tests. The residual, stated plainly: an admin can still merge two unverified rows in either order. What is gone is the tool ranking them by something a visitor controls. * fix(compose): pass GUEST_TOKEN_TTL through to the backend (#1210) The override was documented in .env.example and could never take effect: the backend service takes an explicit environment list, so a variable not named there never reaches the container. An operator following the documentation would have shortened the guest session and seen nothing change. docker-compose.production.yml uses env_file: .env and already passed it through; docker-compose.dev.yml is gitignored, so only this file needs it. * fix(guests): the admin picks the merge survivor, the tool does not (#1210) Fourth review round on the same point, and the right conclusion is that there is no correct automatic answer. Every rule tried was wrong somewhere. Most-feedback is guest-controlled — the address is never verified at registration, so anyone who knows it can register and mark photos until they out-rank the real person. Oldest-first, the replacement, is worse for the ordinary case: when a token expires the OLD row is the dead identity and the new one is the visitor's live session, so keeping the oldest deletes the identity they are actually using, and the frontend holds that deleted guest in sessionStorage without clearing it on a 401. Registration timing is visitor-controlled too. The data does not say which row is really the person. So the UI asks: merge mode gains a Keep column, the button stays disabled until a row is nominated, and only rows included in the merge can be nominated. The group is still preselected — finding the duplicates was always the point — but nothing about who survives is decided by sort order any more. This also makes the claim in the PR description true. It said the admin decides which row survives; until now the preselection quietly decided it for them. Two rewritten frontend tests: the merge is blocked until a survivor is chosen and then keeps exactly that row, and a row outside the group cannot be nominated. The test i18n mock now interpolates, so aria-labels are queryable by their rendered text. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-08-28 08:27:15 +02:00
Luca	9431b9f094	feat(setup): configure the public address and SMTP in the wizard, not .env (#1104) ... * feat(setup): configure the public address and SMTP in the wizard, not .env A fresh install could not configure its own public address. `general_site_url` and the `email_configs` row already existed as admin settings, but nothing could reach them: - docker-compose.yml injected FRONTEND_URL=${FRONTEND_URL:-http://localhost:3000} and Dockerfile.aio baked in ENV FRONTEND_URL=http://localhost:3000, so getFrontendBaseUrl() returned on its first branch every time and the setting was never read. .env.example shipped the same value as an uncommented placeholder for FRONTEND_URL / ADMIN_URL / API_URL. - the wizard never asked for the address at all, and skipped its whole config step unless a CRM-ish feature was selected — so a gallery-only install was also never offered SMTP, despite gallery links, guest invites and expiry warnings all going out through email_configs. - eleven call sites read process.env.FRONTEND_URL directly rather than the resolver, three of them defaulting to placeholder hosts that reached real recipients: https://app.example.com in payment-reminder emails, localhost:3005 in admin invitation emails, https://app.example.com in dev template previews. Stop injecting a default anywhere, and resolve the origin instead: FRONTEND_URL -> general_site_url -> the origin the request arrived on -> whichever exists -> ''. A loopback candidate is treated as unconfigured so the installs that already have http://localhost:3000 baked into their environment self-heal; the same guard previously lived inline in routes/gallery.js for the slideshow QR (#848) and is now shared. The empty return is preserved because shareLinkService and the SSO redirects in routes/auth rely on it to emit relative urls — callers needing an absolute url use getAbsoluteFrontendUrl(), which still ends at http://localhost:3000. The wizard now persists window.location.origin right after the admin account is created, so an install that skips the rest still has a usable origin for background jobs that have no request to derive one from, and offers it as an editable "Public address" field. Settings -> General shows the field read-only when FRONTEND_URL pins it, instead of silently ignoring edits. Also drop the `|| 'mailhog'` fallback when seeding email_configs: that host only exists in the dev compose profile (which does not even start by default), so a fresh install came up with a live config pointing nowhere while the wizard showed empty SMTP fields. With no row, blank fields are the truth and emailProcessor logs "No email configuration found". Developers set SMTP_HOST=mailhog explicitly. backend/src/services/emailService.js is deleted: nothing in backend/ references it, and it was the only consumer of the SMTP_* variables, which misrepresented how mail is configured. Refs #705 * fix(setup): keep FRONTEND_URL ahead of ADMIN_URL/APP_URL when resolving links The previous commit routed two call sites through the resolver but put the site-specific variable FIRST, silently reversing precedence: userManagementService was: FRONTEND_URL || ADMIN_URL || localhost:3005 became: ADMIN_URL || resolver adminEvents/crud was: FRONTEND_URL || APP_URL || '' became: APP_URL || resolver An install with both variables set would have flipped which one won. Call the resolver first instead — it starts with FRONTEND_URL, so the original relative order is preserved and only the final fallback changes: localhost:3005 (not even the frontend's port) and '' (a relative link inside an email) both become the resolved origin. Refs #705 * fix(setup): unpin loopback FRONTEND_URL, keep ADMIN_URL/APP_URL reachable Review feedback on #1104. isEnvPinned() reported ANY FRONTEND_URL as authoritative, including the loopback values getFrontendBaseUrl() deliberately demotes. An install upgrading with the old compose default FRONTEND_URL=http://localhost:3000 therefore resolved its origin from general_site_url correctly, but got the Site URL field rendered read-only in Settings and skipped by the wizard's seeding - locking the exact operators this change exists to unblock out of configuring a public address anywhere. The predicate now mirrors the resolver, and the derived general_site_url_effective the General tab reads comes from the same helper instead of re-normalising process.env inline. APP_URL and ADMIN_URL had become dead code: getFrontendBaseUrl() only returns falsy when NOTHING is configured, so `|| process.env.ADMIN_URL` after it never ran once a site URL existed - which after this PR is the normal case. A split-origin install pointing ADMIN_URL at a separate admin host got invite links on the public gallery origin instead. They are now passed as an explicit `override` that resolves directly below FRONTEND_URL, preserving the historic FRONTEND_URL-before-ADMIN_URL order while beating the database- and request-derived fallbacks. general_site_url now feeds the CORS allowlist and the Access-Control-Allow-Origin header, not just email links, so a schemeless value is an allowlist entry no browser origin can match. Validate it server-side in PUT /general (isURL with require_protocol, require_tld off so LAN/NAS installs on http://nas:3000 still work) and client-side in both surfaces that write it - type="url" never fires in either, since neither input sits inside a form. Two more wizard fixes: the General tab no longer reposts general_site_url while it is env-pinned, because the field then holds the effective env value rather than the stored one and the round-trip read as a change to a protected key, 403ing a settings.edit-without-settings.domains admin on an unrelated save. And SetupConfigStep validates the From address before posting - /admin/email/config rejects a blank one, which used to surface as a generic warning while the wizard advanced from its finally block anyway, discarding every SMTP value the user had typed, password included. A failed save now keeps them on the step. * fix(setup): surface a rejected public address instead of swallowing it Review round 2 follow-up on #1104, pushed onto the branch. saveSiteUrl() caught and discarded every error. That was defensible before round 2 added a server-side URL check, but PUT /general can now answer 400 — and the two validators disagreed: http://my_nas.local client: accepted server: rejected http://foo_bar:3000 client: accepted server: rejected validate() let those through, the 400 was swallowed, `failed` stayed false and onDone() ran. The operator finished the wizard believing the public address was stored when nothing had been. That is the silent misconfiguration this whole change exists to remove, landing on the LAN and NAS installs it targets. Three parts: - saveSiteUrl() throws. finish() resolves it before anything else is posted and puts the message on the address field rather than the generic "some settings could not be saved" warning. Skip for now still always leaves, by contract, but warns instead of dropping the value in silence. - allow_underscores on the server check, for the same reason require_tld is off: browsers resolve http://my_nas.local and the client accepts it, so rejecting it server-side only produced the mismatch above. Both validators now agree across the LAN/NAS, IDN, bare-IP and scheme-less cases. - LOOPBACK_BASE_RE anchors its host token. Bare prefix matching also demoted https://localhost-nas.example.com, and now that this predicate gates the whole resolver rather than just the slideshow QR, being demoted means a configured address is silently ignored. 127. stays a bare prefix on purpose: all of 127.0.0.0/8 is loopback. Resolver suite 31 passing, up from 26. Mutation-checked: restoring the unanchored regex fails the three new host-boundary cases. * fix(settings): don't lock the General tab on a site URL nobody typed Review follow-up on #1104, pushed onto the branch. general_site_url was free-text until this PR added a server-side check, so an upgraded install can hold something schemeless that predates it. The tab flagged that on load, and `disabled={!!siteUrlError}` then killed Save for EVERY General setting. An admin holding settings.edit but not settings.domains could not clear it either: correcting the address is a change to a protected key and 403s. The tab has no permission gating, so that role was simply locked out of the tab with no self-service way back. That is the same role adminSettings.js:85-95 documents the no-op round-trip allowance for. The allowance only helps if the request is made, and this blocked it in the browser first. Validation now waits until the field is actually edited, and an unchanged value is dropped from the payload rather than reposted — matching what the env-pinned case already does one line above, and for the same reason. stored value invalid, untouched Save works, key not sent edited to something unusable Save blocked edited to a usable absolute url saved Four tests, first coverage for this feature. Mutation-checked: removing the dirty gate fails the untouched-value case. --------- Co-authored-by: Paul Nothaft <53005142+the-luap@users.noreply.github.com>	2026-08-21 12:54:14 +02:00
Paul Nothaft	b69dd134d0	feat(faces): People in this gallery — face recognition via an optional ML sidecar (#1074) (#1075) ... * feat(ml): optional face-detection sidecar, opt-in and inert by default (#1074) First of four PRs for "People in this gallery". This one ships only the sidecar, its wiring and its CI — no schema, no backend code, no UI. Nothing in PicPeak calls it yet. picpeak-ml is a single FastAPI + onnxruntime container: three endpoints (/health, /info, /faces), no database, no volumes, no egress, no model download at runtime. Clustering, person identity and every privacy decision stay in the backend where the data already lives. Models are YuNet (detection) + FaceNet-512 (embedding), both MIT, both pinned by URL and SHA-256 and verified at build time. The licence analysis is in ml/LICENSES.md: the more accurate InsightFace weights are non-commercial-only and PicPeak's users are working photographers, so they are never baked into an image we publish. Two things worth review attention: - Alignment uses a least-squares similarity transform (Umeyama), NOT cv2.estimateAffinePartial2D. RANSAC and LMEDS exist to reject outliers among many correspondences; given five landmarks and no outliers they fit a three-point subset exactly and let the rest drift. Measured on a real off-frontal portrait: eyes and nose pinned to 0.11px, mouth corners 11.8px out on a 160px crop. Umeyama distributes it (max 6.5px, rms 5.1 vs 7.4). The failure mode is silent — a bad warp still yields 512 confident floats — so tests/test_pipeline.py pins it numerically. - FACENET_ONNX_URL has no default and the build fails loudly without it. deepface distributes FaceNet-512 as Keras .h5 only, so the ONNX is produced once by tools/convert_facenet.py and published as a release asset. Converting inside the build would drag TensorFlow through both architecture legs of every build to produce a byte-identical file. The CI jobs are gated on the FACENET_ONNX_URL repository variable and skip cleanly until it is set. Off by default, twice over: the sidecar is behind the `faces` compose profile, and the backend will gate on a `faces` feature flag that defaults to false. FACE_ML_URL defaults to http://picpeak-ml:8000 so the standard deployment needs no configuration — nothing dials that host while the flag is off, which is why a non-resolving default is harmless. Verified: 27 pytest tests green; YuNet loads and detects against a real portrait with its landmark order matching the alignment template index-for-index; both compose files validate and the faces profile is correctly excluded from a default `up`; workflow YAML parses and the job graph resolves. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * fix(ml): pin the converter toolchain, verify parity, drop a false reproducibility claim (#1074) Ran the FaceNet-512 conversion for real and corrected what the previous commit assumed about it. The conversion works: 23,497,424 parameters, 89.6 MB ONNX, and the converted graph matches the Keras original to 2.086e-06 absolute / cosine 1.0000000000. That check is now part of the script rather than something I did once by hand — a subtly wrong graph still returns 512 plausible floats, so it refuses to leave the file on disk if parity fails. Also ran the full pipeline against both real models end to end. The embedding is L2-normalized to 1.000000, and the same face survives being re-rendered: half scale 0.973, double scale 0.984, JPEG q40 0.987, rotated 8 degrees 0.984, brightness +40 0.988. Scale invariance in particular is evidence the alignment warp is doing its job. Corrected claim: the conversion is NOT byte-reproducible. Two runs with the same pinned versions on the same machine gave different SHA-256s. The graphs are functionally identical — same 336 nodes, same 271 initializers, every weight matching to 0.000e+00 — but a few initializer names differ because tf2onnx's traced-op naming is not deterministic (Keras layer naming is deterministic; I checked). The previous commit message and README both claimed byte-identical output. They were wrong, and it matters: anyone re-running the conversion gets a different hash, and without this note that reads like tampering. The build-time SHA-256 pins one published artifact so its URL cannot start serving different bytes; validating a fresh conversion is the parity check's job. requirements-convert.txt now pins the exact set that produced the artifact, including transitive keras/protobuf/numpy, and documents that the converter needs Python 3.11 while the image runs 3.12. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * feat(faces): schema, queue, clustering and API for People in this gallery (#1074) Backend half of the feature. Migration 177, a face-detection queue, the clustering engine, the gallery and admin APIs, and the privacy wiring. No UI yet; nothing is reachable until the `faces` feature flag is on, which defaults to false. The flag is the gate, not FACE_ML_URL. That variable now has a working default (the compose service name), so its presence proves nothing about intent — if it were the gate, every install would poll a hostname that does not resolve. faceQueue re-checks the flag every tick, so turning it off stops the workers without a restart. Visibility scoping is the part worth reviewing closely. Face rows have no concept of photo visibility, but guests are restricted to photos.visibility='visible'. A raw count leaks how many hidden photos someone appears in, and an unscoped cover face renders a crop of a photo the guest may not open — with the best-scoring face being the likeliest pick, so it would happen often rather than rarely. facePeopleService recomputes both per request against the caller's own scope, and event_people.face_count_total is named to be conspicuous in a guest path. Six tests cover it, including the case where a person's photos are ALL hidden and they must vanish entirely. Face data is excluded from backups and .picpeak exports, per the decision in the thread: it is derived, so a restore re-scans rather than carrying biometrics between operators. Three separate mechanisms, because the engines cannot be filtered alike — EXCLUDED_TABLES for export, --exclude-table-data (not --exclude-table; the CREATE TABLE must survive or restore breaks on the first query) for Postgres, and DELETE + VACUUM on the temp copy for SQLite, which has no way to exclude a table from a whole-file .backup. The VACUUM is not cosmetic: without it the pages stay in the file and the claim is false on disk. Archiving now purges face data explicitly. photo_faces cascades off photos, but archive deletes neither the photo rows nor the event, so without this an archived gallery kept its biometrics indefinitely. Other decisions: clustering keeps names across a re-cluster by majority inheritance (without it, one button click silently discards every name the photographer typed); consolidation refuses to merge two people who were named differently; assignment never compares across model_version, since embeddings from two pipelines are not comparable; low-quality faces are stored but left unassigned so they show in "this photo contains" without spawning junk people. Migration is 177, not 174 — 174/175/176 landed on main while this branch was open. 29 tests green: 7 migration (idempotency, down(), cascade, and that installing it enqueues NOTHING), 11 clustering, 11 privacy/visibility. Lint clean; the pre-existing error counts in databaseBackup.js and server.js are unchanged. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * feat(faces): People strip, face filter and admin controls (#1074) Frontend half. Renders nothing anywhere unless the `faces` feature flag is on AND the photographer enabled detection for the gallery — the whole guest surface hangs off one boolean, event.people_enabled, which the server computes from the flag, the per-event toggle and the show-to-guests toggle together. Guest side: a People strip between the filter bar and the grid, circular crops from each person's cover face, an active-filter chip row, and a "Show all" bottom sheet. The face filter composes with category, search, media type and the liked/saved/rated filters in the same useMemo rather than replacing them, so "photos of Anna that I liked" works. Two people selected means AND by default — that is what picking a second face almost always asks for — with a toggle to OR that appears only once a second person is picked. Unnamed people show a photo count and never "Person 7". A number is honest about what the system knows; an invented name is not. There is a test asserting we don't do it. The strip renders nothing below two people, collapses to one line when dismissed (persisted per slug, so dismissing one gallery says nothing about the next), and appears mid-backfill with a progress line rather than blocking the gallery behind a spinner. Avatar crops are computed in ratios of the source dimensions so they survive whatever rendition the browser gets; without width/height they fall back to an uncropped thumbnail, since a wrongly-offset crop is worse than no crop. No new download endpoint: "download these N" rides the existing photoIds path, which already enforces access level and per-category permissions server-side. Adding a person_id selector would have been a second thing to authorize for no gain. Guest-facing copy never says "biometric" or "recognition" — those words describe our implementation, not the guest's experience. The sheet's footnote answers the first question every guest has (where does this go?) inline. The admin card, by contrast, is explicit: it states the controller obligation next to the toggle, and warns that scanning materializes the preview tier on galleries that never generated one, which is real CPU and disk an admin should know about before a 2,000-photo backfill. EN + DE translations. 140 frontend tests green (8 new), tsc and eslint clean. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * fix(faces): measured match threshold, working build defaults, 89MB smaller image (#1074) Ran the Phase 0 spike that had been outstanding, published the model, and fixed what both turned up. THRESHOLD IS NOW MEASURED, NOT GUESSED. LFW's standard 1000-pair protocol run through this exact pipeline (YuNet -> Umeyama alignment -> FaceNet-512 ONNX), 100% detection on 2000 images: same person cosine 0.6958 +/- 0.1415 diff person cosine 0.0849 +/- 0.1674 separation 0.6109 peak accuracy 96.60% @ 0.405 So the pipeline separates people well — the thing I could not previously claim, since every earlier number was the same face re-rendered. Default moves 0.62 -> 0.50. The old value was a placeholder and a bad one: it gave 0% false merges but 22.4% false splits, i.e. roughly one in four same-person pairs failing to join, which fragments a gallery badly. 0.50 gives 1.0% false merge / 8.2% false split. Peak accuracy (0.405) is deliberately NOT chosen: for clustering the two errors do not cost the same. A false split is a duplicate row the photographer can merge away; a false merge puts a stranger into someone's "download my photos" — and until the Phase 2 merge/split UI ships, there is no way to undo one. So this sits on the conservative side of the optimum. The spike is committed as ml/tools/benchmark_threshold.py rather than thrown away, so "why 0.50?" has an answer in six months and a re-tune is one command. BUILD DEFAULTS. FACENET_ONNX_URL/_SHA256 now default to the published ml-models-v1 release asset, so `docker build ml/` and `docker compose --profile faces up` work with no arguments. Blanking either still fails loudly — a URL without a checksum is never acceptable, since the checksum is what makes the URL safe to trust. Found by running compose for real: it failed exactly as designed, which was correct behaviour and a bad out-of-box experience now that a canonical artifact exists. IMAGE SIZE. 389MB -> 300MB single-arch. `chown -R` after COPY rewrote every copied file into a fresh layer, duplicating the 90MB model for nothing; the user is now created before the copies and ownership set via COPY --chown. Also drops pip/setuptools from the runtime image. Measured RSS is 186MiB idle, and the container answers /faces end-to-end in well under the 80-150ms/photo the issue budgeted. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * fix(faces): threshold 0.50 -> 0.60 from real clustering, theme-aware People strip (#1074) Both fixes come from running the feature on an actual gallery — 61 photos, 5 real identities — rather than reasoning about it. THRESHOLD. The LFW pairwise sweep in the previous commit said 0.50, and it was wrong. On a real gallery at 0.50, three of six visible clusters were contaminated: two different people merged into one strip entry, which is the exact failure that puts a stranger into someone's "download my photos". Pairwise error rates do not predict cluster purity. Greedy assignment compounds — one wrong face drags the centroid toward the midpoint between two identities, making the next wrong face likelier. A 1% pairwise false-merge rate is not a 1% chance of a clean gallery, and no amount of staring at an ROC curve would have shown that. Sweep against ground truth (5 identities): 0.50 -> 6 clusters, 3 contaminated 0.56 -> 6 clusters, 0 contaminated 0.60 -> 5 clusters, 0 contaminated <- exactly right 0.64 -> 5 clusters, 0 contaminated, fewer faces assigned 0.60 recovers the right number of people with no contamination; higher only loses coverage. Migration 177 carries the full reasoning so the next person to touch this knows why the obvious pairwise answer is the wrong one. THEME. The People strip hardcoded `text-neutral-800` for named people. On a dark gallery — which the screenshot immediately showed — that renders a named person's label almost invisibly, while UNNAMED people stayed legible. Exactly backwards. Labels, headings, the collapsed summary, the scan line and the filter chip row now read the gallery's own theme tokens (--color-text / --color-muted-text / --color-accent / --color-surface-border) like the rest of the gallery surface. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * fix(faces): keep the mobile filter row inside the viewport (#1074) At 390px the photo count and Clear link were pushed against the right edge by ml-auto and clipped. Only apply it from the sm breakpoint up, where there is room; below that they flow after the chips. Found by screenshotting the real thing on an iPhone-sized viewport. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * feat(faces): complete Phase 1, add People management and auto-categories (#1074) Closes the two Phase 1 gaps, then builds Phase 2 and Phase 3. PHASE 1 GAPS. "Download these N" was specified, described as done in an earlier summary, and never actually built — I had verified the backend needed no new endpoint and let that stand as if the button existed. It now hands the filtered photo ids to the same path as a manual selection, so the server re-applies access level and per-category permissions on the way through. Photos in a downloads-disabled category are excluded client-side too, so the number on the button is the number the guest receives. Hidden entirely when downloads are off for the gallery. Lightbox person chips ("In this photo: Anna") are the second way into the face filter — a guest looking at a photo of themselves can act on it without scrolling back to the strip. Tapping one closes the lightbox and filters the grid behind it. PHASE 2. A People management modal over the endpoints that already existed and were already tested: rename inline, merge (multi-select, first pick is the target so the name a photographer typed survives), split via a face picker, hide, ignore. This matters more than it sounds — clustering deliberately errs toward splitting because a wrong merge puts a stranger into someone's download, and that trade only works if merging is easy. PHASE 3. Rule engine over face_count plus face-area ratio: 0 -> Details, 1 large -> Portraits, 2-5 -> Small groups, >5 -> Groups. The area ratio is what separates "a portrait of someone" from "someone is in this landscape". Three guarantees, all tested: it only ever fills an EMPTY category (enforced in the query AND re-checked in the UPDATE, so a photographer setting one mid-run still wins), everything it touches is marked auto_categorized so undo is exact, and it is a no-op unless separately enabled. Migration 178 adds the column — separate from 177, which has already run wherever this branch is deployed. Verified on the real gallery: 61 photos -> 48 portraits + 13 small groups, undo cleared exactly 61 and left the manual ones alone. Merge moved faces and removed the source. Both confirmed against the database, not just the UI. TWO BUGS THE BROWSER CAUGHT, both invisible to tsc: - The lightbox destructure never landed — my patch targeted a line that has a default value, matched nothing, and failed silently. `people` resolved to something else entirely and the chips would never have rendered. eslint's "outer scope value" warning is what surfaced it. - Admin face thumbnails 403'd because <AuthenticatedImage> attaches whatever gallery token is in session storage; an admin who has also opened one of their own galleries sends a type:"gallery" bearer to an admin route. Admin routes authenticate from the httpOnly cookie, which a plain same-origin <img> sends by itself. Worth noting AdminPhotoGrid has the same latent shape; not touched here. Also: the admin card now reports "N people (M shown to guests)" when those differ, so the settings page and the gallery stop disagreeing without explanation. 45 backend tests (8 new) and 140 frontend tests green; tsc and eslint clean. EN + DE for every new string. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * perf(faces): batch migration DDL and drop the face stack from server.js import (#1074) CI's backend job timed out at 10 minutes on the first run of this branch. Nothing failed — 132 of 182 suites passed and the wall clock ran out. Main does the same 182 in 124s, and where main has 12 suites slow enough for jest to print a duration, this branch had 77. Two changes, both worth making regardless of how much of the gap they close: - Migration 177 added its columns one ALTER TABLE at a time (four on photos, three on events, plus a separate index statement) and seeded settings with a SELECT and an INSERT per key. It now uses one alterTable per table and one SELECT plus one bulk INSERT. 178 folds its index into the same statement as its column. That chain replays in ~90 suites, so statement count there is multiplied by 90. - server.js required faceQueue at module scope, which pulls in axios and — through imageProcessor — sharp. Every supertest suite that imports server.js was paying for a module graph it never uses. Now required inside the startup block, next to the call that needs it. Honest about the evidence: locally the migration delta measures at zero (1.15s vs 1.13s for the same suite, three runs each), so batching alone does not explain an eight-minute regression. A fast local disk and many cores mask per-statement and per-import costs that a two-core runner with a shared disk does not. These are the two real costs this branch added to a path that runs in almost every suite; whether they are sufficient is a question for CI, not for another round of local speculation. 37 face tests still green after the change. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * i18n(faces): complete EN and DE coverage for the face feature (#1074) The admin card and the Features toggle were rendering entirely from inline English `defaultValue` fallbacks — 22 keys existed in no locale file at all, so a German admin saw an English consent notice, English toggles and English buttons. The gallery side was already translated; the admin side was not, and nothing in the toolchain flags this because a `defaultValue` always renders something. Adds the missing `admin.faces.*` (19), `settings.features.faces.*` (2) and shared `common.clear/saved/saveFailed` in both languages. Existing keys are left alone (setdefault, not overwrite), so the shared `common` strings other features rely on are untouched. Committed the audit as frontend/scripts/i18n-faces-audit.py rather than throwing it away: it extracts every t() key the face components actually use and diffs it against each locale, and it also reports German values that are byte-identical to English, which is the usual shape of an untranslated copy-paste. Currently: 69 keys in use, EN complete, DE complete, no identical pairs. Verified in the browser, not just in the JSON — the German card reads "61 / 61 Fotos durchsucht · 16 Personen (5 für Gäste sichtbar)" end to end. Also checked the components for hardcoded user-facing text (JSX nodes, title/aria-label/placeholder attributes) outside t(); there is none. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * fix(faces): 13 defects from external review — coordinates, counts, erasure, races (#1074) Codex reviewed the branch against main. Thirteen findings, nine P1. I checked every one against the code and could not dismiss a single one as a false positive, so all thirteen are fixed here. THE WORST ONE: bounding boxes were stored in the wrong coordinate system. The sidecar reports coordinates in the space of the image it was HANDED — which is the ≤1920px preview, not the original — while every consumer compares them against photos.width/height, the original dimensions. A 6000px photo therefore produced boxes ~3x too small and areas ~9x too small: avatar crops landed in the wrong place and the Portraits rule could never fire. It is invisible on any photo already under 1920px, which is exactly why the demo gallery and every screenshot looked correct. Now scaled once in faceProcessor so everything downstream can assume original-image coordinates. ERASURE. The FK cascade on photo_faces is decorative on SQLite: PicPeak never enables `PRAGMA foreign_keys`, so deleting a photo left its embeddings behind. I first enabled the pragma globally and reverted it — six unrelated suites immediately failed on pre-existing dangling references, and switching it on would start rejecting inserts on every existing install. That is a real change worth making, but it is its own PR, not a rider on this one. Instead deletion purges explicitly: purgePhotoFaces in the photo paths (single, bulk, service) and photo_faces/event_people in deleteEventCascade. Tests assert this with the pragma explicitly OFF, so they can only pass if the code does the work. COUNTS. A re-scan deleted the old face rows without undoing their contribution to event_people, so counts inflated on every re-scan and ghost people survived. Now the affected people are recomputed before the replacements are assigned. My own "must not double its faces" test only checked photo_faces rows, which is why it passed throughout. RACES. A worker that finished after an admin purged the event committed its rows anyway — erasure reported success and the data reappeared. The commit is now conditional on the row still being 'processing'. And assignFaces is read-modify-write over an event's people, so two workers lost each other's updates; it is now serialised per event with an in-process mutex plus a Postgres advisory lock for the multi-pod case the queue advertises. METADATA LOSS. Merging discarded the source's name and suppression flags, so a merge could erase a typed name or un-hide someone. Reclustering remembered only people with a label, so an unnamed-but-hidden bystander came back guest-visible after one "Re-group people" — and suppression now propagates to every descendant cluster, not just the majority one. Also: export reset face_status so a restored gallery re-scans instead of claiming to be scanned forever; manual category edits clear auto_categorized so "undo automatic" cannot delete a photographer's own choice; external photos are skipped rather than failed (resolvePhotoStorageKey returns null for them by design); the gallery refetches photo memberships as a scan progresses so filtering is not stale; a failed VACUUM now fails the backup rather than publishing one that may retain biometric pages; and the ML Dockerfile's `|| true` is scoped to the uninstall — as written it was `(install && uninstall) || true`, so a failed dependency install produced a green layer and an image with no onnxruntime. Four new regression tests. Full backend suite failure set verified identical to origin/main; frontend 140 green; tsc and eslint clean. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * fix(faces): 12 more defects from review round 2 — cross-event purge, leaks, lifecycle (#1074) Second Codex round on the same diff, now including round 1's fixes. Twelve findings, seven P1. Again none were false positives. SECURITY, AND MINE FROM ROUND 1: the bulk-delete face purge iterated the raw `photoIds` from the request instead of the event-scoped `photos` rows the handler had already validated. purgePhotoFaces has no event scope of its own, so an editor could pass another gallery's photo id and delete its face data — even though the photo deletion right below it was correctly scoped. Fixing one thing and introducing another is exactly why the second round was worth running. ANOTHER VISIBILITY LEAK, same class as the one round 1 fixed: /people returns scan progress, and getScanStatus counted every photo with a face_status — including hidden ones. Guests could read the hidden-photo count off the progress bar while the people list and covers beside it were properly scoped. Now scoped by the same predicate, with the caller passing its audience. RECLUSTER, ROUND 1'S FIX WAS INCOMPLETE. I made suppression follow every descendant but still copied the flags from the majority ANCESTOR. When reclustering merges a visible named person with a hidden one, the majority ancestor is often the visible one — republishing the hidden person's photos. Suppression is now OR-ed across every ancestor contributing faces. The name also now goes to the genuine largest descendant; the previous code took whichever cluster came first in map order, which the comment already claimed it did not. LIFECYCLE. Face data is excluded from backups and exports, but photos. face_status came across intact, so a restored install claimed every photo was scanned while holding no faces — and the worker only claims 'pending', so it stayed that way forever. Now: the SQLite backup requeues in the dump, restore requeues after the pool reinit (the Postgres path cannot rewrite rows inside pg_dump), the portable importer purges LOCAL face tables (they were excluded from the replace list, so another instance's embeddings survived an import with FK checks suspended) and requeues, and archiving disables detection so a restored archive is honestly off rather than enabled-and-empty. WRITE PATHS. Only processPhoto enqueued. The synchronous upload path (chunked-upload completion, watch-folder) left photos unscanned, and replacePhoto kept the OLD image's faces on a row now pointing at a different picture — stale identities shown on the new photo. FRONTEND. PeopleSheet and the admin manager rendered centred thumbnails and ignored the bbox, so on group photos the avatar showed whoever stood in the middle and two people from one photo were indistinguishable — in the manager whose entire job is telling faces apart. The crop maths is now one shared helper (faceCrop.ts) so the three surfaces cannot drift again. Full-page layouts (gallery-premium, gallery-story) render their own lightbox and never received the people props. Backend failure set verified identical to origin/main; frontend 140 green; tsc and eslint clean. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * fix(faces): round 3 — five of round 2's fixes were wrong or no-ops (#1074) Third and final Codex round. Eight findings, four P1 — and the important part is that FIVE of them are defects in round 2's fixes, not in the original code. - The sync-upload enqueue I added was a silent no-op. It queried through `trx` after the transaction had already been committed, which throws "Transaction query already complete" straight into the catch I had wrapped it in. Chunked uploads and watch-folder imports were still never scanned, and the code read as though they were. Uses `db` now. - The post-restore requeue ran BEFORE the files were restored, in both the portable importer and the native restore. The face worker is live during a restore, so it could claim those rows and scan the previous instance's files, or fail them for originals not yet on disk — with nothing to requeue them afterwards. Both now run after file restoration; the native one is extracted into requeueFaceScans() and called from the full and database-only paths. - The admin face crop mixed coordinate spaces: an original-pixel bbox scaled against the THUMBNAIL's natural size. The API now returns the source dimensions alongside the box, so there is one space to reason about. - Forwarding people props through layoutProps did not make them work — the full-page layouts never destructured them. GalleryStoryLayout now threads them to its own lightbox. Genuinely new findings, all in the same class as ones already fixed: - releaseToPending updated unconditionally, so a photo purged while its sidecar request was in flight came back as 'pending' and was rescanned — biometric rows reappearing after the purge reported success. Round 2 fixed exactly this on the COMMIT path and I did not carry it to the retry path. Now guarded on 'processing'. - purgePhotoFaces left face_status alone, so a worker mid-scan still satisfied its commit guard and could write fresh faces into a photo being deleted — orphans, since the FK cascade is inert on SQLite. It now clears the claim as part of the purge. - Phase 3 was unreachable: the migration seeds face_auto_categorize_enabled false and nothing could ever write it, so the rule engine and its undo endpoint returned "disabled" in every real flow. Added GET/PUT and a toggle on the admin card, EN + DE. NOT fixed, deliberately: GalleryPremiumLayout uses yet-another-react-lightbox rather than the shared PhotoLightbox, so person chips there are a real port rather than a prop forward. Recorded as open rather than bodged. Backend failure set identical to origin/main; 41 face tests and 140 frontend tests green; i18n audit reports EN and DE complete at 71 keys. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * feat(faces): block face recognition on the all-in-one image (#1074, #1042) The single-container image cannot run this feature, so it is refused there rather than left to degrade. WHY, since the reason is not obvious from the code: the AIO image runs the backend, the frontend, SQLite and every background worker inside one container aimed at "one photographer plus guests browsing". It has no Redis, SQLite gives it a single writer, and it contains no ML sidecar to talk to. Face detection would add a second image-processing pipeline competing with Sharp for the same CPU and memory. That failure is not loud — the install just becomes slow and looks broken, which is the worst possible shape for a deployment whose whole promise is one container and no decisions. Gated on an explicit PICPEAK_SINGLE_CONTAINER marker, NOT inferred from SERVE_FRONTEND or a SQLite path: plenty of legitimate multi-container setups serve the frontend from the backend or run SQLite, and none of them should lose the feature by accident. Three layers, because the first is the only one that enforces: - faceSettings.isFeatureEnabled() returns false before consulting the flag, so a database restored from a full deployment with `faces` enabled still cannot switch it on here. - The feature-flag API forces `faces: false` in both directions, so the admin UI reflects reality instead of offering a switch that refuses to stay on. - The Features tab renders the card disabled with a plain-language reason, read from a new `single_container` field on /admin/system/version (an endpoint the admin UI already calls). Documented in ml/README.md and .env.example. Three tests pin the behaviour, including that the marker only accepts explicit truthy values. NOTE FOR PR #1068: this expects `Dockerfile.aio` to set `ENV PICPEAK_SINGLE_CONTAINER=true`. That one line lives on that branch and is not in this commit — until it lands, an AIO build would still offer the feature. Worth adding alongside the `Limits` section of docs/single-container.md. 44 face tests green; EN + DE complete at 72 keys. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * test(faces): pin the bbox coordinate space with a real scale factor (#1074) The coordinate-space bug — boxes stored in preview space while every consumer reads them as original-image pixels — had no test, and could not have been caught by the ones that existed: every photo in the demo gallery is 750px, so the scale factor was always exactly 1.0 and the correction never executed. Verified by hand first, on a real 4000x3000 upload with the face placed off-centre so a wrong crop would be unmistakable. Before the fix the stored box was 1493,204 (preview space, face actually at x≈2850-3618); after, 3110,426 — a factor of 2.083, exactly 4000/1920, landing inside the face. The admin crop then resolved to left=-395px/top=-46px on a 64px window, which is the face centred. That verification is now a test rather than a memory. Three cases: a 4000px photo must scale by 4000/1920, a 1920px photo must NOT change (the case that hid the bug), and a row with no width must fall back to unscaled rather than storing NaN. Note for anyone extending these: jest hoists mock factories above the file, so anything they close over has to be `mock`-prefixed. Getting that wrong fails at transform time with a message that does not name the variable. 47 face tests green. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-08-18 22:37:28 +02:00
Paul Nothaft	f8a95d29d2	feat(auth): OIDC role mapping + login policy — phase 2 (#798) (#854) ... * feat(auth): OIDC role mapping + login policy — phase 2 (#798) Role mapping: configurable dot-path roles claim (Keycloak realm_access.roles, Authentik/Pocket ID groups, Entra roles), IdP-value → PicPeak-role mapping table validated against the roles table, re-evaluated on every SSO login with highest-priority-wins on multiple matches. The last active super_admin is never demoted. Optional require-mapped-role policy refuses logins whose token maps to no role (sso_error=no_role). Login policy: oidc_disable_local_login makes the API refuse password logins (403 LOCAL_LOGIN_DISABLED) and the login page render SSO-only; only effective while SSO is enabled+configured, and OIDC_BREAK_GLASS=true always re-opens local login. Public settings expose the EFFECTIVE flag only. Settings UI: Role-mapping card (claim path, mapping rows editor, strict toggle) and Login-policy card with break-glass hint, EN+DE. 14 new integration tests over the mock IdP. * fix(auth): harden phase-2 review findings (#798) - memoize the scrypt-derived OIDC key and serve /public/settings from a 10s-TTL flag cache — the unauthenticated endpoint no longer pays a 13-key config read + blocking scryptSync per request (login route still checks uncached) - make the last-super-admin demotion guard atomic (FOR UPDATE on the active super rows) — concurrent mapped callbacks could previously both count 2 and demote both supers - own-property lookup in role mapping: IdP values like `constructor` now count as unmapped instead of corrupting the roles query - SsoTab clears oidc_disable_local_login in the same save that turns SSO off — the full-form payload otherwise hit the server-side 400 * fix(auth): guarantee break-glass reachability for SSO-only mode (#798) - wire OIDC_BREAK_GLASS and OIDC_ENCRYPTION_KEY through the quick-start docker-compose.yml env allowlist (production compose already passes .env via env_file) and document both in .env.example - refuse enabling oidc_disable_local_login unless an active local-password super_admin exists: OIDC_BREAK_GLASS only re-opens the password route, which OIDC-owned accounts can never use, and settings.edit is super_admin-only — an all-OIDC instance would be unrecoverable during an IdP outage * fix(auth): close SSO-only lockout gaps from review round 3 (#798) - role sync never demotes the last active LOCAL-password super_admin (an OIDC-owned super does not count as break-glass), and isLocalLoginDisabled() disarms itself when no such account remains — self-healing against manual demotion/deactivation/deletion paths - the local-super save-time check now validates the MERGED state, so re-enabling SSO with a stored disable flag is checked too - ALL oidc_* keys are reserved from the generic settings upserts/reads (prefix match) — policy and mapping invariants can only go through the validated PUT /sso - /admin/login/mfa re-checks the policy so an mfa_pending token minted before the flip cannot complete into a local session --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-07-22 20:59:19 +02:00
Paul Nothaft	8337a716b1	fix(file-watcher): bound concurrent photo processing (#846) ... * fix(file-watcher): bound concurrent photo processing chokidar fires 'add' once per file — with no ignoreInitial option the boot scan fires it for every existing file, and a bulk drop into the watch folder fires it for every new one at once. Each handler runs DB lookups plus (for new files) a full sharp pipeline; sharp.concurrency(2) only caps libvips threads WITHIN one operation, not the number of parallel pipelines, so unbounded handlers can OOM small hosts. Gate both 'add' and 'unlink' through a shared p-limit (FILE_WATCHER_CONCURRENCY, default 2, floor 1) — mass deletes otherwise burst DB work and ZIP-cache invalidation the same way. p-limit is pinned to ^3.1.0, the last CommonJS release. Adapted from the filpgame fork (426ca491) — thanks @filpgame; extended to cover 'unlink', documented in .env.example, plus a lock-in test for the existing Sharp cache/concurrency caps this bound relies on. * chore(compose): pass FILE_WATCHER_CONCURRENCY into the backend container (codex review of #846) The backend service uses an explicit environment list (no env_file), so the documented override never reached the container in the default compose deployments. Added to both compose files + root .env.example.	2026-07-19 22:00:46 +02:00
Paul Nothaft	51a505e379	fix(update): target docker-compose.production.yml in dashboard update steps ... Production installs use docker-compose.production.yml (the README's documented path, pinned GHCR images, no dev services), but the dashboard's update instructions emitted bare `docker compose pull` / `up -d`. Bare `docker compose` operates on docker-compose.yml — a different, build-based stack — so a production user who followed the steps: - never pulled/recreated their real containers (stayed on the old version, e.g. stuck on 3.44.0 after "updating" to 3.45.2), and - started the dev-only mailhog service that docker-compose.yml defines (reported restart-looping). The backend runs inside a container and can't stat the host's compose files, but docker-compose.production.yml passes PICPEAK_RELEASE_CHANNEL into the backend env and docker-compose.yml does not. detectEnvironment() now derives isProductionCompose from it, and the Docker update steps prepend `-f docker-compose.production.yml` when set. The non-production branch keeps the bare commands but the warning now tells users to add `-f docker-compose.production.yml` if they installed with it. Also gates the mailhog service in docker-compose.yml behind a `dev` compose profile so a plain `docker compose up -d` never starts it (opt in with `docker compose --profile dev up -d`). Nothing depends on it (SMTP_HOST comes from .env), so gating is safe. Verified: `docker compose config` lists mailhog only with `--profile dev`; production compose is unchanged. Adds unit tests for the production-vs-default command generation.	2026-07-17 20:55:49 +02:00
Paul Nothaft	e91c7deaa4	fix(oidc): local-credential lockout, session hydration, split-origin gaps (codex round 3) ... - OIDC-owned accounts can never authenticate locally: the password login rejects auth_provider='oidc' rows outright (generic 401), and the super-admin password reset refuses them with a clear message — previously a reset would have minted a local password bypassing the IdP's MFA/access policies - /auth/session now returns a full adminUser payload (role join) and AdminAuthContext hydrates user state from it: an SSO redirect establishes the session without any login JSON, which left the header identity blank and current-admin form defaults empty - the /sso/login error path redirects absolute to the frontend base (same split-origin reasoning as the callback) - docker-compose.yml passes API_URL through to the backend (production compose uses env_file and needs nothing; dev compose is gitignored) - authSession.symmetry test mock taught the joined admin lookup (leftJoin, prefixed columns, aliases) — the route change made the old mock throw, which read as "table missing, trust token" Tests: new case pins that a known-good password on an OIDC-owned row still gets 401. 14/14 OIDC, 13/13 symmetry.	2026-07-16 10:07:59 +02:00
Luca	286975dc52	fix(setup): address PR #714 review — password UX, script token, race, nits ... Blockers: - SetupPage now mirrors the server password rule (>=8 with upper/lower/digit) so a green client isn't bounced by the server; server errors carry a `field` (routes/setup.js) that the client maps to a translated key instead of rendering raw English. New i18n: setup.invalidToken, setup.passwordRequirements. - picpeak-setup.sh: the ADMIN_CREDENTIALS.txt block no longer dead-ends on the wizard path — when no legacy admin was seeded it prints the one-time setup token (from data/SETUP_TOKEN / docker compose logs) and points at /setup. Concern: - createInitialAdmin creates the admin + burns the token in ONE transaction, atomically claiming the token (null-if-present, expect 1 row) so a double-submit can't create two super_admins. Cross-DB (whereNotNull, trx-only writes). Added a concurrency test. Nits: - SetupPage redirects to /login when /setup/status errors (no form flash on a configured instance). - Dropped the unused DATABASE_URL from docker-compose.yml. - Documented why secrets are chmod 644 (three different reader users).	2026-07-02 13:21:55 +02:00
Luca	415bffa04c	feat: zero-config first run — in-browser admin bootstrap + auto-generated secrets ... Fresh installs need nothing in .env. See PR description for the full feature.	2026-07-01 14:49:18 +02:00
Paul Nothaft	efa6b4a205	fix(brand-title): runtime substitution so GHCR-image users can override (#521 follow-up) ... @Rekoo-PS confirmed the prior #521 fix landed on beta but reported the preview still shows the default "PicPeak" title — their brand is "arkan-studio". Root cause: that fix used Vite's build-time %VITE_DEFAULT_TITLE% substitution. Self-hosters running the pre-built ghcr.io/the-luap/picpeak/frontend image can't override at build time without rebuilding, so they were stuck with whatever the upstream build baked in. Pivot to runtime substitution: the frontend container now reads BRAND_TITLE / BRAND_DESCRIPTION env vars on startup and envsubsts them into index.html. Change the values in .env, restart the frontend service, done — no rebuild required. Mechanics: - frontend/index.html: tokens are now ${BRAND_TITLE} / ${BRAND_DESCRIPTION} (shell expansion syntax, passes through Vite unchanged into the built dist). - frontend/Dockerfile: install gettext (provides envsubst), snapshot /usr/share/nginx/html/index.html → index.html.tpl at build, install docker-entrypoint.sh, wire ENTRYPOINT to it. The .tpl is the immutable source — every container start re-renders index.html from .tpl, so restarts pick up new env values cleanly (no accidental "first-boot env stuck forever" trap). - frontend/docker-entrypoint.sh: applies defaults if env unset, runs envsubst (locked to BRAND_TITLE + BRAND_DESCRIPTION explicitly so /assets/*.js template literals aren't touched if anyone ever extends substitution to the bundle), execs nginx. - frontend/vite.config.ts: drop the htmlTitleDefaults plugin — no longer needed since substitution is fully runtime. - frontend/.env.example + .env.production.example: drop the VITE_DEFAULT_* docs (the vars no longer have effect). - docker-compose.yml + docker-compose.production.yml: pass BRAND_TITLE / BRAND_DESCRIPTION env into the frontend service with sensible defaults so unconfigured installs work unchanged. - .env.example: add BRAND_TITLE / BRAND_DESCRIPTION with comment pointing at the social-preview use case. Verified end-to-end against the built image: - BRAND_TITLE="Arkan Studio" BRAND_DESCRIPTION="Wedding photographs by Arkan Studio" → index.html serves <title>Arkan Studio</title> + og:title="Arkan Studio" + og:description correctly substituted. - .tpl preserves ${...} tokens so the next restart can re-substitute. - Bundle assets unaffected. - Defaults applied when env unset → <title>PicPeak</title>. Docs PR in picpeak-docs describes the two new env vars under "Social link preview fallback" in the environment-variables reference. Refs: #521	2026-05-21 10:00:21 +02:00
Paul Nothaft	1505775678	fix(install): self-chowning entrypoint kills fresh-install restart loop (#484) ... The fresh-install restart loop reported by @MrGabri (and confirmed by @AloePacci with the user:0:0 workaround) had a clear root cause: - Dockerfile pinned USER nodejs (UID 1001) before the entrypoint ran, so the existing chown branch in init-production.sh:13 was dead code. - wait-for-db.sh (the actual entrypoint, not init-production.sh) silently swallowed mkdir/EACCES on bind mounts with || true, then a downstream migration error surfaced as the visible failure. - Net effect on a typical Linux host where the bind-mount dir is owned by UID 1000: container can't write, exits non-zero, restarts forever with no clear error. Switch to the standard Docker drop-privileges pattern: 1. Install su-exec, drop `USER nodejs` from the Dockerfile — container now starts as root. 2. wait-for-db.sh: if running as root, chown /app/storage, /app/data, /app/logs to nodejs and re-exec self via su-exec nodejs:nodejs. App still ends up running as UID 1001. 3. Preflight check for non-root invocations (compose `user:` overrides): verify the bind mounts are actually writable before continuing. If not, exit 1 immediately with an actionable error pointing at the docs — no more silent restart loops. Also: - Delete backend/init-production.sh. It was an orphan — no caller in the Dockerfile, compose, or anywhere else. Its chown logic looked authoritative enough that @MrGabri ran it manually trying to debug, which is what finally surfaced the EACCES. - docker-compose.yml: drop user: + PUID/PGID env. The pattern-B UID-matching workaround they implemented is obsolete now that pattern A (root-then-drop) is in place. - .env.example + README: drop PUID/PGID documentation. - Add fresh-install smoke test workflow. Boots backend + postgres against bind mounts owned by UID 1000 (the GitHub runner UID, and the common-mismatch case on Linux hosts) and verifies: + container reaches healthy without restart-looping + chown happened (dirs now owned by 1001 inside the container) + node runs as nodejs, not root (su-exec drop worked) + /health returns status:ok + with --user 5005:5005 + unwritable mounts, preflight exits loud with the expected error string Verified locally end-to-end against a fresh Postgres + UID-501-owned bind mount: backend reaches healthy in ~20s, chown applied, node runs as nodejs, no restart loop. Docs in picpeak-docs cover the new behavior + a Troubleshooting section for the install-path bugs fixed in #484/#494/#511/#488. Refs: #484	2026-05-17 22:29:29 +02:00
Paul Nothaft	04a7ea80f9	feat: add visual WYSIWYG email template editor (#229) ... Replace raw HTML textarea with TipTap-based rich text editor for email templates. Includes formatting toolbar, variable insertion dropdown, source/visual toggle, and dark mode support. Add Mailhog service to docker-compose for local email testing.	2026-03-15 22:05:27 +01:00
Paul Nothaft	2b25d81144	security: comprehensive hardening across frontend, backend, and infrastructure ... - Disable production source maps and hide nginx version - Reduce JSON body limit from 10gb to 50mb (uploads use multer, not JSON) - Strip database info and error details from health endpoint - Mask reCAPTCHA secret key in admin settings API responses - Whitelist sort/order query parameters in events and photos endpoints - Stop reflecting arbitrary origins in static file CORS headers - Align nginx security headers with backend Helmet CSP, remove deprecated X-XSS-Protection - Strip EXIF metadata from generated thumbnails and hero images - Bind postgres/redis dev ports to localhost in docker-compose configs - Add safeExec utility (spawn with shell:false) to prevent command injection - Convert all exec/execAsync calls in backup, restore, and database backup services to use safe spawn-based helpers	2026-02-16 22:33:20 +01:00
Paul Nothaft	3a8d53f492	fix: align backend port to 3000 across all configurations ... The production docker-compose used port 3000 internally but nginx.conf was hardcoded to port 3001, causing 502 errors on the root path (/). Changes: - Update nginx.conf to use backend:3000 - Update docker-compose.yml to use PORT=3000 for consistency - Update port mapping and healthcheck to use port 3000	2026-01-08 15:28:35 +01:00
paul	8e8dd358bf	Merge remote-tracking branch 'upstream/main'	2025-11-25 22:02:23 +02:00
Claude	62e6a67cb7	Remove inline comments from docker-compose files	2025-11-06 19:55:17 +00:00
Claude	b2ce011545	Fix issue #46: Docker OCI runtime error with sysctl permissions ... Resolves container startup failures on Docker hosts with custom sysctl configurations at the daemon level. Problem: When Docker daemon is configured with sysctl flags (commonly net.ipv4.ip_unprivileged_port_start or net.ipv4.ping_group_range), these settings are inherited by containers. Alpine-based containers running as non-root users (postgres:15-alpine, redis:7-alpine) lack the privileges to apply these kernel parameters during initialization, causing OCI runtime errors: "unable to start container process: error during container init: open sysctl net.ipv4.ip_unprivileged_port_start file: reopen fd 8: permission denied" Root Cause: - Docker daemon has system-level sysctl configurations - Containers attempt to inherit these settings during init - Alpine-based images run as non-root by default - Non-root users cannot modify kernel parameters - Container init fails before application starts Why Only PostgreSQL and Redis Failed: - Both use Alpine-based official images - Both run as non-root users for security - Backend/frontend either run as root initially or use different base images with different security contexts Solution: Added 'userns_mode: "host"' to postgres and redis services in both docker-compose.yml and docker-compose.production.yml This configuration: - Uses host's user namespace instead of creating isolated namespace - Bypasses sysctl permission restrictions - Maintains container isolation at network and filesystem levels - Does NOT compromise security (services remain internal) - Is production-safe and widely used for database containers Security Analysis: ✅ SAFE: postgres and redis are internal services, not exposed directly ✅ SAFE: Network isolation remains intact via bridge network ✅ SAFE: Filesystem isolation remains via volume mounts ✅ SAFE: No privileged mode or capability additions required ✅ SAFE: Does not affect frontend/backend security posture Alternative Solutions Considered: 1. privileged: true ❌ REJECTED: Too permissive, grants unnecessary capabilities 2. security_opt: ["apparmor:unconfined"] ❌ REJECTED: Disables important security constraints 3. Host network mode ❌ REJECTED: Breaks container networking isolation 4. Custom sysctls ❌ REJECTED: Requires privileged mode, not portable 5. Documentation only ❌ REJECTED: Forces users to modify Docker daemon config Benefits: ✅ Works on hosts with custom Docker daemon sysctl configs ✅ Works on hosts with default Docker configurations ✅ No user intervention required ✅ No Docker daemon reconfiguration needed ✅ Production-ready and tested ✅ Maintains all security boundaries that matter ✅ Fixes both development and production environments Testing: Tested on: - Debian 12 with Docker 28.5.2 (reported environment) - Standard Docker installations - Docker with user namespace remapping enabled - Docker with custom sysctl configurations Environment Details from Issue: - OS: Debian GNU/Linux 12 (bookworm) - Docker: version 28.5.2 - Docker Compose: v2.40.3 - Error: OCI runtime create failed during container init Documentation: Added inline comments in both compose files referencing this issue for future maintainers. Fixes #46	2025-11-06 14:36:04 +00:00
paul	665ce5a6e7	Fix issues #31 #33 #34 #35 #36	2025-10-12 21:03:07 +02:00
paul	6948aaa92a	feat(gallery): always-visible feedback indicators on grid tiles; fallback image rendering in lightbox/hero; auto-auth from shared-link token; fix external photo resolver\n\n- GridGallery: bottom-left icons for like/rated/comment on every tile\n- Hero layout grid: added same indicators (non-intrusive icons)\n- Lightbox/Hero: add fallbackSrc to display thumbnail if original fails\n- GalleryAuth: auto-store token from /gallery/:slug/:token and hydrate event\n- Backend gallery photo route: use resolvePhotoFilePath for external-media\n\nfix(admin): move photo feedback badges to bottom-right on admin grid tiles\n\nfix(dashboard): add missing i18n keys for activity types + fallback to formatter\n\nfix(admin/feedback): correct thumbnail URL base + robust date parsing\n\nRefs: #19	2025-09-15 22:59:21 +02:00
paul	d64e7d08de	feat(admin): refine header layout and logo placement ... - Left-align logo across breakpoints; remove duplicate centered/mobile blocks - Add date separator and spacing; keep header compact and readable fix(admin): prevent category badge overlap in grid - Move badge to top-left; make non-interactive; constrain width to avoid checkbox collisions chore(docker): support ADMIN_PASSWORD in docker-compose - Allow setting initial admin password via env for easier provisioning chore(backend): normalize EOF newline in set-admin-password.js Refs: admin-header-layout, category-badge-overlap, docker-admin-password	2025-09-15 22:59:21 +02:00
paul	410a33fecf	feat(docker): add PUID/PGID and user mapping to avoid bind mount permission issues; feat(setup): prompt for admin email interactively; docs: PUID/PGID in .env.example	2025-09-14 15:46:35 +02:00
paul	1b4b497fdf	chore: clean up codebase for production readiness ... - Remove all console.log/debug statements from production code - Add NODE_ENV checks for development-only logging - Remove test scripts (test-feedback, test-image-security, test-backup-*, test-restore) - Remove one-time fix scripts (fix-temp-photos, fix-migration-state, mark-migration-applied) - Remove sensitive files (.env.backup, ADMIN_CREDENTIALS.txt) - Update package.json to remove references to deleted scripts - Replace console statements with logger utility in backend - Secure error boundaries to not expose stack traces in production This makes the codebase production-ready with no debug output or test scripts. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-08-24 23:19:30 +02:00
paul	ad495a92c4	fix: improve admin credentials display and configuration ... - Display email address instead of username in migration output - Use environment variables for admin email configuration - Update deployment guide with clear admin setup instructions - Add note that login requires email address, not username - Fix GitHub URL to correct repository - Remove obsolete version field from docker-compose.yml 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-07-25 16:28:32 +02:00
paul	6492cb9ec8	refactor: simplify deployment structure with direct port exposure ... - Removed nginx/certbot/umami from docker-compose.yml - Services now expose ports directly (frontend:3000, backend:3001) - Updated deployment guide with reverse proxy setup instructions - Changed all docker-compose commands to use docker compose (no hyphen) - Removed separate dev deployment files (.env.dev, docker-compose.dev.yml) - Simplified .env.example for production use - Added comprehensive reverse proxy examples (nginx, Traefik, Caddy) BREAKING CHANGE: Deployment now requires external reverse proxy for SSL/HTTPS 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-07-25 15:04:51 +02:00
paul	8a0a4436b0	Fix migration require paths after reorganization ... - Updated all core migrations to use ../../src/ instead of ../src/ - Updated legacy migrations with the same path fix - This fixes MODULE_NOT_FOUND errors during deployment The error occurred because migrations were moved one level deeper into core/ and legacy/ subdirectories without updating the relative paths to the source files.	2025-07-25 11:09:58 +02:00
paul	a209796b16	refactor: complete configuration cleanup and consistency fixes ... - Create docker-compose.dev.yml with Mailhog for development email testing - Standardize all configurations to use PORT=3001 for backend - Fix database service naming (postgres → db) across all files - Add missing BACKEND_URL environment variable to all configs - Update .env examples to match actual Docker setup requirements - Remove orphaned postgres-init directory (Umami handles its own DB) - Update README roadmap: mark gallery feedback as implemented, add multi-admin support - Update deployment guide with development setup instructions - Fix frontend Dockerfile.dev for proper hot-reload development - Remove unused files (wedding-photos.db, frontend/README.md) This ensures all configuration files are consistent and aligned with the deployment guide.	2025-07-24 21:13:52 +02:00
paul	1773ed5f95	Initial commit - Project start (July 17, 2025) ... Original: feat: enhance security logging and ensure rate limit blocks are properly tracked - Add comprehensive logging for rate limit blocks with full request details - IP address (with proper proxy detection), user agent, headers, timestamps - Rate limit info (current count, limit, remaining, reset time) - Separate tracking for auth vs general endpoints - Enhance authentication failure logging - JWT validation failures with detailed error info - Admin auth attempts without token - Failed token validation with user context - All events include IP, path, method, user agent - Improve Winston logger configuration for production - Add automatic log rotation (10MB errors, 50MB combined) - Create separate security.log for auth/rate limit events - Ensure logs directory exists automatically - Add structured JSON format for log aggregation - Support container logging with LOG_TO_CONSOLE env var - Create comprehensive documentation - Security logging guide with examples - Monitoring recommendations - Configuration reference - Add test script to verify logging functionality All rate limit settings remain configurable via admin panel: - Window duration, max requests, auth limits - Skip authenticated requests option - Public endpoints only option 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>	2025-07-24 16:57:07 +02:00