* feat(ml): optional face-detection sidecar, opt-in and inert by default (#1074) First of four PRs for "People in this gallery". This one ships only the sidecar, its wiring and its CI — no schema, no backend code, no UI. Nothing in PicPeak calls it yet. picpeak-ml is a single FastAPI + onnxruntime container: three endpoints (/health, /info, /faces), no database, no volumes, no egress, no model download at runtime. Clustering, person identity and every privacy decision stay in the backend where the data already lives. Models are YuNet (detection) + FaceNet-512 (embedding), both MIT, both pinned by URL and SHA-256 and verified at build time. The licence analysis is in ml/LICENSES.md: the more accurate InsightFace weights are non-commercial-only and PicPeak's users are working photographers, so they are never baked into an image we publish. Two things worth review attention: - Alignment uses a least-squares similarity transform (Umeyama), NOT cv2.estimateAffinePartial2D. RANSAC and LMEDS exist to reject outliers among many correspondences; given five landmarks and no outliers they fit a three-point subset exactly and let the rest drift. Measured on a real off-frontal portrait: eyes and nose pinned to 0.11px, mouth corners 11.8px out on a 160px crop. Umeyama distributes it (max 6.5px, rms 5.1 vs 7.4). The failure mode is silent — a bad warp still yields 512 confident floats — so tests/test_pipeline.py pins it numerically. - FACENET_ONNX_URL has no default and the build fails loudly without it. deepface distributes FaceNet-512 as Keras .h5 only, so the ONNX is produced once by tools/convert_facenet.py and published as a release asset. Converting inside the build would drag TensorFlow through both architecture legs of every build to produce a byte-identical file. The CI jobs are gated on the FACENET_ONNX_URL repository variable and skip cleanly until it is set. Off by default, twice over: the sidecar is behind the `faces` compose profile, and the backend will gate on a `faces` feature flag that defaults to false. FACE_ML_URL defaults to http://picpeak-ml:8000 so the standard deployment needs no configuration — nothing dials that host while the flag is off, which is why a non-resolving default is harmless. Verified: 27 pytest tests green; YuNet loads and detects against a real portrait with its landmark order matching the alignment template index-for-index; both compose files validate and the faces profile is correctly excluded from a default `up`; workflow YAML parses and the job graph resolves. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * fix(ml): pin the converter toolchain, verify parity, drop a false reproducibility claim (#1074) Ran the FaceNet-512 conversion for real and corrected what the previous commit assumed about it. The conversion works: 23,497,424 parameters, 89.6 MB ONNX, and the converted graph matches the Keras original to 2.086e-06 absolute / cosine 1.0000000000. That check is now part of the script rather than something I did once by hand — a subtly wrong graph still returns 512 plausible floats, so it refuses to leave the file on disk if parity fails. Also ran the full pipeline against both real models end to end. The embedding is L2-normalized to 1.000000, and the same face survives being re-rendered: half scale 0.973, double scale 0.984, JPEG q40 0.987, rotated 8 degrees 0.984, brightness +40 0.988. Scale invariance in particular is evidence the alignment warp is doing its job. Corrected claim: the conversion is NOT byte-reproducible. Two runs with the same pinned versions on the same machine gave different SHA-256s. The graphs are functionally identical — same 336 nodes, same 271 initializers, every weight matching to 0.000e+00 — but a few initializer names differ because tf2onnx's traced-op naming is not deterministic (Keras layer naming is deterministic; I checked). The previous commit message and README both claimed byte-identical output. They were wrong, and it matters: anyone re-running the conversion gets a different hash, and without this note that reads like tampering. The build-time SHA-256 pins one published artifact so its URL cannot start serving different bytes; validating a fresh conversion is the parity check's job. requirements-convert.txt now pins the exact set that produced the artifact, including transitive keras/protobuf/numpy, and documents that the converter needs Python 3.11 while the image runs 3.12. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * feat(faces): schema, queue, clustering and API for People in this gallery (#1074) Backend half of the feature. Migration 177, a face-detection queue, the clustering engine, the gallery and admin APIs, and the privacy wiring. No UI yet; nothing is reachable until the `faces` feature flag is on, which defaults to false. The flag is the gate, not FACE_ML_URL. That variable now has a working default (the compose service name), so its presence proves nothing about intent — if it were the gate, every install would poll a hostname that does not resolve. faceQueue re-checks the flag every tick, so turning it off stops the workers without a restart. Visibility scoping is the part worth reviewing closely. Face rows have no concept of photo visibility, but guests are restricted to photos.visibility='visible'. A raw count leaks how many hidden photos someone appears in, and an unscoped cover face renders a crop of a photo the guest may not open — with the best-scoring face being the likeliest pick, so it would happen often rather than rarely. facePeopleService recomputes both per request against the caller's own scope, and event_people.face_count_total is named to be conspicuous in a guest path. Six tests cover it, including the case where a person's photos are ALL hidden and they must vanish entirely. Face data is excluded from backups and .picpeak exports, per the decision in the thread: it is derived, so a restore re-scans rather than carrying biometrics between operators. Three separate mechanisms, because the engines cannot be filtered alike — EXCLUDED_TABLES for export, --exclude-table-data (not --exclude-table; the CREATE TABLE must survive or restore breaks on the first query) for Postgres, and DELETE + VACUUM on the temp copy for SQLite, which has no way to exclude a table from a whole-file .backup. The VACUUM is not cosmetic: without it the pages stay in the file and the claim is false on disk. Archiving now purges face data explicitly. photo_faces cascades off photos, but archive deletes neither the photo rows nor the event, so without this an archived gallery kept its biometrics indefinitely. Other decisions: clustering keeps names across a re-cluster by majority inheritance (without it, one button click silently discards every name the photographer typed); consolidation refuses to merge two people who were named differently; assignment never compares across model_version, since embeddings from two pipelines are not comparable; low-quality faces are stored but left unassigned so they show in "this photo contains" without spawning junk people. Migration is 177, not 174 — 174/175/176 landed on main while this branch was open. 29 tests green: 7 migration (idempotency, down(), cascade, and that installing it enqueues NOTHING), 11 clustering, 11 privacy/visibility. Lint clean; the pre-existing error counts in databaseBackup.js and server.js are unchanged. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * feat(faces): People strip, face filter and admin controls (#1074) Frontend half. Renders nothing anywhere unless the `faces` feature flag is on AND the photographer enabled detection for the gallery — the whole guest surface hangs off one boolean, event.people_enabled, which the server computes from the flag, the per-event toggle and the show-to-guests toggle together. Guest side: a People strip between the filter bar and the grid, circular crops from each person's cover face, an active-filter chip row, and a "Show all" bottom sheet. The face filter composes with category, search, media type and the liked/saved/rated filters in the same useMemo rather than replacing them, so "photos of Anna that I liked" works. Two people selected means AND by default — that is what picking a second face almost always asks for — with a toggle to OR that appears only once a second person is picked. Unnamed people show a photo count and never "Person 7". A number is honest about what the system knows; an invented name is not. There is a test asserting we don't do it. The strip renders nothing below two people, collapses to one line when dismissed (persisted per slug, so dismissing one gallery says nothing about the next), and appears mid-backfill with a progress line rather than blocking the gallery behind a spinner. Avatar crops are computed in ratios of the source dimensions so they survive whatever rendition the browser gets; without width/height they fall back to an uncropped thumbnail, since a wrongly-offset crop is worse than no crop. No new download endpoint: "download these N" rides the existing photoIds path, which already enforces access level and per-category permissions server-side. Adding a person_id selector would have been a second thing to authorize for no gain. Guest-facing copy never says "biometric" or "recognition" — those words describe our implementation, not the guest's experience. The sheet's footnote answers the first question every guest has (where does this go?) inline. The admin card, by contrast, is explicit: it states the controller obligation next to the toggle, and warns that scanning materializes the preview tier on galleries that never generated one, which is real CPU and disk an admin should know about before a 2,000-photo backfill. EN + DE translations. 140 frontend tests green (8 new), tsc and eslint clean. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * fix(faces): measured match threshold, working build defaults, 89MB smaller image (#1074) Ran the Phase 0 spike that had been outstanding, published the model, and fixed what both turned up. THRESHOLD IS NOW MEASURED, NOT GUESSED. LFW's standard 1000-pair protocol run through this exact pipeline (YuNet -> Umeyama alignment -> FaceNet-512 ONNX), 100% detection on 2000 images: same person cosine 0.6958 +/- 0.1415 diff person cosine 0.0849 +/- 0.1674 separation 0.6109 peak accuracy 96.60% @ 0.405 So the pipeline separates people well — the thing I could not previously claim, since every earlier number was the same face re-rendered. Default moves 0.62 -> 0.50. The old value was a placeholder and a bad one: it gave 0% false merges but 22.4% false splits, i.e. roughly one in four same-person pairs failing to join, which fragments a gallery badly. 0.50 gives 1.0% false merge / 8.2% false split. Peak accuracy (0.405) is deliberately NOT chosen: for clustering the two errors do not cost the same. A false split is a duplicate row the photographer can merge away; a false merge puts a stranger into someone's "download my photos" — and until the Phase 2 merge/split UI ships, there is no way to undo one. So this sits on the conservative side of the optimum. The spike is committed as ml/tools/benchmark_threshold.py rather than thrown away, so "why 0.50?" has an answer in six months and a re-tune is one command. BUILD DEFAULTS. FACENET_ONNX_URL/_SHA256 now default to the published ml-models-v1 release asset, so `docker build ml/` and `docker compose --profile faces up` work with no arguments. Blanking either still fails loudly — a URL without a checksum is never acceptable, since the checksum is what makes the URL safe to trust. Found by running compose for real: it failed exactly as designed, which was correct behaviour and a bad out-of-box experience now that a canonical artifact exists. IMAGE SIZE. 389MB -> 300MB single-arch. `chown -R` after COPY rewrote every copied file into a fresh layer, duplicating the 90MB model for nothing; the user is now created before the copies and ownership set via COPY --chown. Also drops pip/setuptools from the runtime image. Measured RSS is 186MiB idle, and the container answers /faces end-to-end in well under the 80-150ms/photo the issue budgeted. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * fix(faces): threshold 0.50 -> 0.60 from real clustering, theme-aware People strip (#1074) Both fixes come from running the feature on an actual gallery — 61 photos, 5 real identities — rather than reasoning about it. THRESHOLD. The LFW pairwise sweep in the previous commit said 0.50, and it was wrong. On a real gallery at 0.50, three of six visible clusters were contaminated: two different people merged into one strip entry, which is the exact failure that puts a stranger into someone's "download my photos". Pairwise error rates do not predict cluster purity. Greedy assignment compounds — one wrong face drags the centroid toward the midpoint between two identities, making the next wrong face likelier. A 1% pairwise false-merge rate is not a 1% chance of a clean gallery, and no amount of staring at an ROC curve would have shown that. Sweep against ground truth (5 identities): 0.50 -> 6 clusters, 3 contaminated 0.56 -> 6 clusters, 0 contaminated 0.60 -> 5 clusters, 0 contaminated <- exactly right 0.64 -> 5 clusters, 0 contaminated, fewer faces assigned 0.60 recovers the right number of people with no contamination; higher only loses coverage. Migration 177 carries the full reasoning so the next person to touch this knows why the obvious pairwise answer is the wrong one. THEME. The People strip hardcoded `text-neutral-800` for named people. On a dark gallery — which the screenshot immediately showed — that renders a named person's label almost invisibly, while UNNAMED people stayed legible. Exactly backwards. Labels, headings, the collapsed summary, the scan line and the filter chip row now read the gallery's own theme tokens (--color-text / --color-muted-text / --color-accent / --color-surface-border) like the rest of the gallery surface. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * fix(faces): keep the mobile filter row inside the viewport (#1074) At 390px the photo count and Clear link were pushed against the right edge by ml-auto and clipped. Only apply it from the sm breakpoint up, where there is room; below that they flow after the chips. Found by screenshotting the real thing on an iPhone-sized viewport. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * feat(faces): complete Phase 1, add People management and auto-categories (#1074) Closes the two Phase 1 gaps, then builds Phase 2 and Phase 3. PHASE 1 GAPS. "Download these N" was specified, described as done in an earlier summary, and never actually built — I had verified the backend needed no new endpoint and let that stand as if the button existed. It now hands the filtered photo ids to the same path as a manual selection, so the server re-applies access level and per-category permissions on the way through. Photos in a downloads-disabled category are excluded client-side too, so the number on the button is the number the guest receives. Hidden entirely when downloads are off for the gallery. Lightbox person chips ("In this photo: Anna") are the second way into the face filter — a guest looking at a photo of themselves can act on it without scrolling back to the strip. Tapping one closes the lightbox and filters the grid behind it. PHASE 2. A People management modal over the endpoints that already existed and were already tested: rename inline, merge (multi-select, first pick is the target so the name a photographer typed survives), split via a face picker, hide, ignore. This matters more than it sounds — clustering deliberately errs toward splitting because a wrong merge puts a stranger into someone's download, and that trade only works if merging is easy. PHASE 3. Rule engine over face_count plus face-area ratio: 0 -> Details, 1 large -> Portraits, 2-5 -> Small groups, >5 -> Groups. The area ratio is what separates "a portrait of someone" from "someone is in this landscape". Three guarantees, all tested: it only ever fills an EMPTY category (enforced in the query AND re-checked in the UPDATE, so a photographer setting one mid-run still wins), everything it touches is marked auto_categorized so undo is exact, and it is a no-op unless separately enabled. Migration 178 adds the column — separate from 177, which has already run wherever this branch is deployed. Verified on the real gallery: 61 photos -> 48 portraits + 13 small groups, undo cleared exactly 61 and left the manual ones alone. Merge moved faces and removed the source. Both confirmed against the database, not just the UI. TWO BUGS THE BROWSER CAUGHT, both invisible to tsc: - The lightbox destructure never landed — my patch targeted a line that has a default value, matched nothing, and failed silently. `people` resolved to something else entirely and the chips would never have rendered. eslint's "outer scope value" warning is what surfaced it. - Admin face thumbnails 403'd because <AuthenticatedImage> attaches whatever gallery token is in session storage; an admin who has also opened one of their own galleries sends a type:"gallery" bearer to an admin route. Admin routes authenticate from the httpOnly cookie, which a plain same-origin <img> sends by itself. Worth noting AdminPhotoGrid has the same latent shape; not touched here. Also: the admin card now reports "N people (M shown to guests)" when those differ, so the settings page and the gallery stop disagreeing without explanation. 45 backend tests (8 new) and 140 frontend tests green; tsc and eslint clean. EN + DE for every new string. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * perf(faces): batch migration DDL and drop the face stack from server.js import (#1074) CI's backend job timed out at 10 minutes on the first run of this branch. Nothing failed — 132 of 182 suites passed and the wall clock ran out. Main does the same 182 in 124s, and where main has 12 suites slow enough for jest to print a duration, this branch had 77. Two changes, both worth making regardless of how much of the gap they close: - Migration 177 added its columns one ALTER TABLE at a time (four on photos, three on events, plus a separate index statement) and seeded settings with a SELECT and an INSERT per key. It now uses one alterTable per table and one SELECT plus one bulk INSERT. 178 folds its index into the same statement as its column. That chain replays in ~90 suites, so statement count there is multiplied by 90. - server.js required faceQueue at module scope, which pulls in axios and — through imageProcessor — sharp. Every supertest suite that imports server.js was paying for a module graph it never uses. Now required inside the startup block, next to the call that needs it. Honest about the evidence: locally the migration delta measures at zero (1.15s vs 1.13s for the same suite, three runs each), so batching alone does not explain an eight-minute regression. A fast local disk and many cores mask per-statement and per-import costs that a two-core runner with a shared disk does not. These are the two real costs this branch added to a path that runs in almost every suite; whether they are sufficient is a question for CI, not for another round of local speculation. 37 face tests still green after the change. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * i18n(faces): complete EN and DE coverage for the face feature (#1074) The admin card and the Features toggle were rendering entirely from inline English `defaultValue` fallbacks — 22 keys existed in no locale file at all, so a German admin saw an English consent notice, English toggles and English buttons. The gallery side was already translated; the admin side was not, and nothing in the toolchain flags this because a `defaultValue` always renders something. Adds the missing `admin.faces.*` (19), `settings.features.faces.*` (2) and shared `common.clear/saved/saveFailed` in both languages. Existing keys are left alone (setdefault, not overwrite), so the shared `common` strings other features rely on are untouched. Committed the audit as frontend/scripts/i18n-faces-audit.py rather than throwing it away: it extracts every t() key the face components actually use and diffs it against each locale, and it also reports German values that are byte-identical to English, which is the usual shape of an untranslated copy-paste. Currently: 69 keys in use, EN complete, DE complete, no identical pairs. Verified in the browser, not just in the JSON — the German card reads "61 / 61 Fotos durchsucht · 16 Personen (5 für Gäste sichtbar)" end to end. Also checked the components for hardcoded user-facing text (JSX nodes, title/aria-label/placeholder attributes) outside t(); there is none. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * fix(faces): 13 defects from external review — coordinates, counts, erasure, races (#1074) Codex reviewed the branch against main. Thirteen findings, nine P1. I checked every one against the code and could not dismiss a single one as a false positive, so all thirteen are fixed here. THE WORST ONE: bounding boxes were stored in the wrong coordinate system. The sidecar reports coordinates in the space of the image it was HANDED — which is the ≤1920px preview, not the original — while every consumer compares them against photos.width/height, the original dimensions. A 6000px photo therefore produced boxes ~3x too small and areas ~9x too small: avatar crops landed in the wrong place and the Portraits rule could never fire. It is invisible on any photo already under 1920px, which is exactly why the demo gallery and every screenshot looked correct. Now scaled once in faceProcessor so everything downstream can assume original-image coordinates. ERASURE. The FK cascade on photo_faces is decorative on SQLite: PicPeak never enables `PRAGMA foreign_keys`, so deleting a photo left its embeddings behind. I first enabled the pragma globally and reverted it — six unrelated suites immediately failed on pre-existing dangling references, and switching it on would start rejecting inserts on every existing install. That is a real change worth making, but it is its own PR, not a rider on this one. Instead deletion purges explicitly: purgePhotoFaces in the photo paths (single, bulk, service) and photo_faces/event_people in deleteEventCascade. Tests assert this with the pragma explicitly OFF, so they can only pass if the code does the work. COUNTS. A re-scan deleted the old face rows without undoing their contribution to event_people, so counts inflated on every re-scan and ghost people survived. Now the affected people are recomputed before the replacements are assigned. My own "must not double its faces" test only checked photo_faces rows, which is why it passed throughout. RACES. A worker that finished after an admin purged the event committed its rows anyway — erasure reported success and the data reappeared. The commit is now conditional on the row still being 'processing'. And assignFaces is read-modify-write over an event's people, so two workers lost each other's updates; it is now serialised per event with an in-process mutex plus a Postgres advisory lock for the multi-pod case the queue advertises. METADATA LOSS. Merging discarded the source's name and suppression flags, so a merge could erase a typed name or un-hide someone. Reclustering remembered only people with a label, so an unnamed-but-hidden bystander came back guest-visible after one "Re-group people" — and suppression now propagates to every descendant cluster, not just the majority one. Also: export reset face_status so a restored gallery re-scans instead of claiming to be scanned forever; manual category edits clear auto_categorized so "undo automatic" cannot delete a photographer's own choice; external photos are skipped rather than failed (resolvePhotoStorageKey returns null for them by design); the gallery refetches photo memberships as a scan progresses so filtering is not stale; a failed VACUUM now fails the backup rather than publishing one that may retain biometric pages; and the ML Dockerfile's `|| true` is scoped to the uninstall — as written it was `(install && uninstall) || true`, so a failed dependency install produced a green layer and an image with no onnxruntime. Four new regression tests. Full backend suite failure set verified identical to origin/main; frontend 140 green; tsc and eslint clean. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * fix(faces): 12 more defects from review round 2 — cross-event purge, leaks, lifecycle (#1074) Second Codex round on the same diff, now including round 1's fixes. Twelve findings, seven P1. Again none were false positives. SECURITY, AND MINE FROM ROUND 1: the bulk-delete face purge iterated the raw `photoIds` from the request instead of the event-scoped `photos` rows the handler had already validated. purgePhotoFaces has no event scope of its own, so an editor could pass another gallery's photo id and delete its face data — even though the photo deletion right below it was correctly scoped. Fixing one thing and introducing another is exactly why the second round was worth running. ANOTHER VISIBILITY LEAK, same class as the one round 1 fixed: /people returns scan progress, and getScanStatus counted every photo with a face_status — including hidden ones. Guests could read the hidden-photo count off the progress bar while the people list and covers beside it were properly scoped. Now scoped by the same predicate, with the caller passing its audience. RECLUSTER, ROUND 1'S FIX WAS INCOMPLETE. I made suppression follow every descendant but still copied the flags from the majority ANCESTOR. When reclustering merges a visible named person with a hidden one, the majority ancestor is often the visible one — republishing the hidden person's photos. Suppression is now OR-ed across every ancestor contributing faces. The name also now goes to the genuine largest descendant; the previous code took whichever cluster came first in map order, which the comment already claimed it did not. LIFECYCLE. Face data is excluded from backups and exports, but photos. face_status came across intact, so a restored install claimed every photo was scanned while holding no faces — and the worker only claims 'pending', so it stayed that way forever. Now: the SQLite backup requeues in the dump, restore requeues after the pool reinit (the Postgres path cannot rewrite rows inside pg_dump), the portable importer purges LOCAL face tables (they were excluded from the replace list, so another instance's embeddings survived an import with FK checks suspended) and requeues, and archiving disables detection so a restored archive is honestly off rather than enabled-and-empty. WRITE PATHS. Only processPhoto enqueued. The synchronous upload path (chunked-upload completion, watch-folder) left photos unscanned, and replacePhoto kept the OLD image's faces on a row now pointing at a different picture — stale identities shown on the new photo. FRONTEND. PeopleSheet and the admin manager rendered centred thumbnails and ignored the bbox, so on group photos the avatar showed whoever stood in the middle and two people from one photo were indistinguishable — in the manager whose entire job is telling faces apart. The crop maths is now one shared helper (faceCrop.ts) so the three surfaces cannot drift again. Full-page layouts (gallery-premium, gallery-story) render their own lightbox and never received the people props. Backend failure set verified identical to origin/main; frontend 140 green; tsc and eslint clean. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * fix(faces): round 3 — five of round 2's fixes were wrong or no-ops (#1074) Third and final Codex round. Eight findings, four P1 — and the important part is that FIVE of them are defects in round 2's fixes, not in the original code. - The sync-upload enqueue I added was a silent no-op. It queried through `trx` after the transaction had already been committed, which throws "Transaction query already complete" straight into the catch I had wrapped it in. Chunked uploads and watch-folder imports were still never scanned, and the code read as though they were. Uses `db` now. - The post-restore requeue ran BEFORE the files were restored, in both the portable importer and the native restore. The face worker is live during a restore, so it could claim those rows and scan the previous instance's files, or fail them for originals not yet on disk — with nothing to requeue them afterwards. Both now run after file restoration; the native one is extracted into requeueFaceScans() and called from the full and database-only paths. - The admin face crop mixed coordinate spaces: an original-pixel bbox scaled against the THUMBNAIL's natural size. The API now returns the source dimensions alongside the box, so there is one space to reason about. - Forwarding people props through layoutProps did not make them work — the full-page layouts never destructured them. GalleryStoryLayout now threads them to its own lightbox. Genuinely new findings, all in the same class as ones already fixed: - releaseToPending updated unconditionally, so a photo purged while its sidecar request was in flight came back as 'pending' and was rescanned — biometric rows reappearing after the purge reported success. Round 2 fixed exactly this on the COMMIT path and I did not carry it to the retry path. Now guarded on 'processing'. - purgePhotoFaces left face_status alone, so a worker mid-scan still satisfied its commit guard and could write fresh faces into a photo being deleted — orphans, since the FK cascade is inert on SQLite. It now clears the claim as part of the purge. - Phase 3 was unreachable: the migration seeds face_auto_categorize_enabled false and nothing could ever write it, so the rule engine and its undo endpoint returned "disabled" in every real flow. Added GET/PUT and a toggle on the admin card, EN + DE. NOT fixed, deliberately: GalleryPremiumLayout uses yet-another-react-lightbox rather than the shared PhotoLightbox, so person chips there are a real port rather than a prop forward. Recorded as open rather than bodged. Backend failure set identical to origin/main; 41 face tests and 140 frontend tests green; i18n audit reports EN and DE complete at 71 keys. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * feat(faces): block face recognition on the all-in-one image (#1074, #1042) The single-container image cannot run this feature, so it is refused there rather than left to degrade. WHY, since the reason is not obvious from the code: the AIO image runs the backend, the frontend, SQLite and every background worker inside one container aimed at "one photographer plus guests browsing". It has no Redis, SQLite gives it a single writer, and it contains no ML sidecar to talk to. Face detection would add a second image-processing pipeline competing with Sharp for the same CPU and memory. That failure is not loud — the install just becomes slow and looks broken, which is the worst possible shape for a deployment whose whole promise is one container and no decisions. Gated on an explicit PICPEAK_SINGLE_CONTAINER marker, NOT inferred from SERVE_FRONTEND or a SQLite path: plenty of legitimate multi-container setups serve the frontend from the backend or run SQLite, and none of them should lose the feature by accident. Three layers, because the first is the only one that enforces: - faceSettings.isFeatureEnabled() returns false before consulting the flag, so a database restored from a full deployment with `faces` enabled still cannot switch it on here. - The feature-flag API forces `faces: false` in both directions, so the admin UI reflects reality instead of offering a switch that refuses to stay on. - The Features tab renders the card disabled with a plain-language reason, read from a new `single_container` field on /admin/system/version (an endpoint the admin UI already calls). Documented in ml/README.md and .env.example. Three tests pin the behaviour, including that the marker only accepts explicit truthy values. NOTE FOR PR #1068: this expects `Dockerfile.aio` to set `ENV PICPEAK_SINGLE_CONTAINER=true`. That one line lives on that branch and is not in this commit — until it lands, an AIO build would still offer the feature. Worth adding alongside the `Limits` section of docs/single-container.md. 44 face tests green; EN + DE complete at 72 keys. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * test(faces): pin the bbox coordinate space with a real scale factor (#1074) The coordinate-space bug — boxes stored in preview space while every consumer reads them as original-image pixels — had no test, and could not have been caught by the ones that existed: every photo in the demo gallery is 750px, so the scale factor was always exactly 1.0 and the correction never executed. Verified by hand first, on a real 4000x3000 upload with the face placed off-centre so a wrong crop would be unmistakable. Before the fix the stored box was 1493,204 (preview space, face actually at x≈2850-3618); after, 3110,426 — a factor of 2.083, exactly 4000/1920, landing inside the face. The admin crop then resolved to left=-395px/top=-46px on a 64px window, which is the face centred. That verification is now a test rather than a memory. Three cases: a 4000px photo must scale by 4000/1920, a 1920px photo must NOT change (the case that hid the bug), and a row with no width must fall back to unscaled rather than storing NaN. Note for anyone extending these: jest hoists mock factories above the file, so anything they close over has to be `mock`-prefixed. Getting that wrong fails at transform time with a message that does not name the variable. 47 face tests green. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc --------- Co-authored-by: Paul Nothaft <[email protected]>
This commit is contained in:
Paul Nothaft
co-authored by
Paul Nothaft
Paul Nothaft
parent
6ebdd13dde
commit
b69dd134d0
@@ -225,6 +225,35 @@ async function archiveEvent(event) {
|
||||
}
|
||||
}
|
||||
|
||||
// Purge face data (#1074). photo_faces cascades off photos, but archiving
|
||||
// does NOT delete the photo rows — and event_people hangs off the event,
|
||||
// which also survives. So neither would go without an explicit purge, and
|
||||
// an archived gallery would keep its biometric data indefinitely.
|
||||
//
|
||||
// Face data is derived: if the event is ever restored, re-enabling
|
||||
// detection re-scans. Nothing irreplaceable is lost except assigned
|
||||
// names, which is the same trade already accepted for backups/exports.
|
||||
try {
|
||||
const { purgeEvent } = require('./faceProcessor');
|
||||
await purgeEvent(event.id);
|
||||
|
||||
// Turn detection OFF as well. purgeEvent clears the rows but leaves the
|
||||
// toggle on, so restoring the archive would bring back a gallery that
|
||||
// claims face detection is enabled while having no people and no queued
|
||||
// work — indistinguishable from a broken scan. Off is the honest state:
|
||||
// the photographer re-enables it and gets a fresh backfill, which is
|
||||
// exactly the flow the toggle already implements.
|
||||
await db('events').where({ id: event.id })
|
||||
.update({ face_recognition_enabled: false, faces_last_scan_at: null });
|
||||
} catch (err) {
|
||||
// Never fail an archive over this — but say so loudly, because it
|
||||
// means biometric data outlived the gallery.
|
||||
logger.error(
|
||||
`Archive: failed to purge face data for event ${event.slug} — ` +
|
||||
`face rows may remain. ${err.message}`
|
||||
);
|
||||
}
|
||||
|
||||
// Queue completion email — admin_email is nullable on events (migration 073);
|
||||
// skip queueing rather than violating email_queue.recipient_email NOT NULL.
|
||||
//
|
||||
|
||||
@@ -16,6 +16,20 @@ const packageJson = require('../../package.json');
|
||||
const CHUNK_SIZE = 1024 * 1024; // 1MB chunks for streaming
|
||||
const PROGRESS_INTERVAL = 100; // Report progress every 100 rows
|
||||
|
||||
// Face recognition tables (#1074). Their SCHEMA is backed up, their CONTENTS
|
||||
// are not: embeddings are biometric data (GDPR Art. 9) and fully derived from
|
||||
// the photos, so a restore re-scans instead of carrying biometrics forward.
|
||||
//
|
||||
// The schema must survive, which is why Postgres uses --exclude-table-DATA
|
||||
// rather than --exclude-table: knex_migrations records 177 as applied, so a
|
||||
// restore whose dump lacked the CREATE TABLE would fail on the first query
|
||||
// instead of merely coming back empty.
|
||||
//
|
||||
// SQLite cannot filter at all — `sqlite3 .backup` is a whole-file binary copy
|
||||
// — so the rows are deleted from the temp copy before it is finalised. See
|
||||
// createSQLiteBackup below.
|
||||
const FACE_TABLES = ['photo_faces', 'event_people'];
|
||||
|
||||
/**
|
||||
* Database Backup Service
|
||||
* Supports both SQLite and PostgreSQL with proper escaping,
|
||||
@@ -149,18 +163,20 @@ class DatabaseBackupService {
|
||||
AND name != 'knex_migrations_lock'
|
||||
ORDER BY name
|
||||
`);
|
||||
return result.map(row => row.name);
|
||||
return result.map(row => row.name).filter((t) => !FACE_TABLES.includes(t));
|
||||
} else {
|
||||
// PostgreSQL
|
||||
const result = await db.raw(`
|
||||
SELECT table_name
|
||||
FROM information_schema.tables
|
||||
WHERE table_schema = 'public'
|
||||
SELECT table_name
|
||||
FROM information_schema.tables
|
||||
WHERE table_schema = 'public'
|
||||
AND table_type = 'BASE TABLE'
|
||||
AND table_name NOT IN ('knex_migrations', 'knex_migrations_lock')
|
||||
ORDER BY table_name
|
||||
`);
|
||||
return result.rows.map(row => row.table_name);
|
||||
return result.rows
|
||||
.map(row => row.table_name)
|
||||
.filter((t) => !FACE_TABLES.includes(t));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -175,6 +191,48 @@ class DatabaseBackupService {
|
||||
// Use SQLite's backup API for consistency
|
||||
await spawnAsync('sqlite3', [dbPath, `.backup '${tempPath}'`]);
|
||||
|
||||
// Strip face data from the COPY (#1074). `.backup` is a whole-file
|
||||
// binary copy with no way to exclude a table, so the rows come out and
|
||||
// are deleted here — the live database is never touched.
|
||||
//
|
||||
// VACUUM is not cosmetic: without it the deleted pages remain in the
|
||||
// file and "not backed up" would be false on disk, which is the exact
|
||||
// claim this code exists to make true.
|
||||
for (const table of FACE_TABLES) {
|
||||
await spawnAsync('sqlite3', [
|
||||
tempPath,
|
||||
`DELETE FROM ${table} WHERE 1=1;`,
|
||||
]).catch(() => {
|
||||
// Table absent on installs that predate migration 177 — fine.
|
||||
});
|
||||
}
|
||||
|
||||
// Reset the DERIVED state on photos as well. Without this the restored
|
||||
// database says every photo is scanned ('done') while the face tables
|
||||
// are empty, and the worker only ever claims 'pending' — so the gallery
|
||||
// reports a finished scan and shows nobody, permanently, until an admin
|
||||
// works out that a manual re-scan is needed. Requeue instead.
|
||||
await spawnAsync('sqlite3', [
|
||||
tempPath,
|
||||
"UPDATE photos SET face_status = CASE WHEN face_status IS NULL THEN NULL ELSE 'pending' END, "
|
||||
+ 'face_count = NULL, face_started_at = NULL, face_error = NULL;',
|
||||
]).catch(() => {});
|
||||
// FATAL, not a warning. Deleting rows leaves their pages in the file
|
||||
// until VACUUM rewrites it, so a backup that skipped the VACUUM can
|
||||
// still contain recoverable face embeddings. Publishing it would break
|
||||
// the explicit promise that biometric data is never backed up — better
|
||||
// to fail the backup and say so than to hand over an artifact that
|
||||
// quietly violates it.
|
||||
try {
|
||||
await spawnAsync('sqlite3', [tempPath, 'VACUUM;']);
|
||||
} catch (err) {
|
||||
await fs.unlink(tempPath).catch(() => {});
|
||||
throw new Error(
|
||||
'SQLite backup aborted: could not VACUUM after removing face data, so the '
|
||||
+ `backup could still contain recoverable biometric pages (${err.message})`
|
||||
);
|
||||
}
|
||||
|
||||
// Verify the backup
|
||||
const verifyResult = await spawnAsync('sqlite3', [tempPath, 'PRAGMA integrity_check']);
|
||||
if (!verifyResult.stdout.includes('ok')) {
|
||||
@@ -232,6 +290,17 @@ class DatabaseBackupService {
|
||||
// callers (scheduled cron, dedicated admin DB-backup page) hit
|
||||
// the same failure but on installs that had never exercised them.
|
||||
|
||||
// Face data (#1074): schema yes, rows no. --exclude-table-DATA, not
|
||||
// --exclude-table — see the FACE_TABLES comment at the top of this file
|
||||
// for why dropping the CREATE TABLE would break restore outright.
|
||||
for (const table of FACE_TABLES) {
|
||||
pgDumpOptions.push(`--exclude-table-data=public.${table}`);
|
||||
}
|
||||
// NOTE: the Postgres path cannot rewrite rows inside pg_dump the way the
|
||||
// SQLite path can, so photos.face_status is restored as-is here. The
|
||||
// restore path compensates — see resetDerivedFaceState in restoreService.
|
||||
// Both engines must end up requeued, not "done with no people".
|
||||
|
||||
// Add compression if not doing it separately
|
||||
if (options.compress && !options.separateCompression) {
|
||||
pgDumpOptions.push('--compress=6');
|
||||
|
||||
@@ -0,0 +1,178 @@
|
||||
/**
|
||||
* Rule-based auto-categorisation from face counts (#1074 phase 3).
|
||||
*
|
||||
* Once faces are detected, `photos.face_count` and the landmark yaw are free
|
||||
* inputs for the sorting @bluecow94 asked for in discussion #1069. This is a
|
||||
* small rule list, not a classifier — the point is that it is predictable and
|
||||
* explainable, which a photographer can work with.
|
||||
*
|
||||
* THREE NON-NEGOTIABLE RULES, in order of importance:
|
||||
*
|
||||
* 1. It only ever fills an EMPTY category. It never overwrites a category
|
||||
* a photographer set, because a human assignment is a decision and this
|
||||
* is a heuristic. There is no configuration that changes this.
|
||||
*
|
||||
* 2. Everything it does is marked `auto_categorized = true`, so "undo all
|
||||
* automatic categories" is one query rather than an archaeology
|
||||
* exercise.
|
||||
*
|
||||
* 3. It is a separate step from detection and separately enableable. Some
|
||||
* photographers want the sorting and not the people strip.
|
||||
*/
|
||||
|
||||
const { db } = require('../database/db');
|
||||
const logger = require('../utils/logger');
|
||||
const { getThresholds } = require('./faceSettings');
|
||||
|
||||
/**
|
||||
* Category slugs this engine manages, in evaluation order. First match wins.
|
||||
*
|
||||
* `minFaceAreaRatio` on Portraits is what separates "a portrait of someone"
|
||||
* from "someone happens to be in this landscape" — one face in the frame
|
||||
* means nothing without knowing how much of the frame it occupies.
|
||||
*/
|
||||
const RULES = [
|
||||
{
|
||||
slug: 'details',
|
||||
name: 'Details',
|
||||
match: (photo) => photo.face_count === 0,
|
||||
},
|
||||
{
|
||||
slug: 'portraits',
|
||||
name: 'Portraits',
|
||||
match: (photo, ctx) => photo.face_count === 1 && ctx.largestFaceAreaRatio >= 0.08,
|
||||
},
|
||||
{
|
||||
slug: 'small-groups',
|
||||
name: 'Small groups',
|
||||
match: (photo) => photo.face_count >= 2 && photo.face_count <= 5,
|
||||
},
|
||||
{
|
||||
slug: 'groups',
|
||||
name: 'Groups',
|
||||
match: (photo) => photo.face_count > 5,
|
||||
},
|
||||
];
|
||||
|
||||
/**
|
||||
* Find or create the event-scoped category for a rule.
|
||||
*
|
||||
* Event-scoped rather than global: these are generated categories, and
|
||||
* polluting every gallery's category list with them would be worse than the
|
||||
* duplication. An existing GLOBAL category with the same slug is reused when
|
||||
* present, so an operator who already has "Portraits" keeps one.
|
||||
*/
|
||||
async function resolveCategory(eventId, rule, cache, trx = db) {
|
||||
if (cache.has(rule.slug)) return cache.get(rule.slug);
|
||||
|
||||
let category = await trx('photo_categories')
|
||||
.where('slug', rule.slug)
|
||||
.where(function () {
|
||||
this.where('event_id', eventId).orWhere('is_global', true);
|
||||
})
|
||||
.first();
|
||||
|
||||
if (!category) {
|
||||
const [inserted] = await trx('photo_categories').insert({
|
||||
name: rule.name,
|
||||
slug: rule.slug,
|
||||
is_global: false,
|
||||
event_id: eventId,
|
||||
created_at: new Date().toISOString(),
|
||||
}).returning('id');
|
||||
const id = typeof inserted === 'object' ? inserted.id : inserted;
|
||||
category = { id, slug: rule.slug };
|
||||
}
|
||||
|
||||
cache.set(rule.slug, category);
|
||||
return category;
|
||||
}
|
||||
|
||||
/**
|
||||
* Apply the rules to one event.
|
||||
*
|
||||
* Only touches photos that have been face-scanned (`face_status = 'done'`)
|
||||
* and have no category. Returns a per-slug tally.
|
||||
*/
|
||||
async function categorizeEvent(eventId) {
|
||||
const thresholds = await getThresholds();
|
||||
if (!thresholds.face_auto_categorize_enabled) {
|
||||
return { skipped: true, reason: 'disabled' };
|
||||
}
|
||||
|
||||
const photos = await db('photos')
|
||||
.where({ event_id: eventId, face_status: 'done' })
|
||||
// Rule 1, enforced in the query rather than trusted to a later branch.
|
||||
.whereNull('category_id')
|
||||
.select('id', 'face_count', 'width', 'height');
|
||||
|
||||
if (!photos.length) return { assigned: 0, byCategory: {} };
|
||||
|
||||
// One query for every face box in the event, rather than one per photo.
|
||||
const faceRows = await db('photo_faces')
|
||||
.where({ event_id: eventId })
|
||||
.whereIn('photo_id', photos.map((p) => p.id))
|
||||
.select('photo_id', 'bbox_w', 'bbox_h');
|
||||
|
||||
const largestByPhoto = new Map();
|
||||
for (const row of faceRows) {
|
||||
const area = (row.bbox_w || 0) * (row.bbox_h || 0);
|
||||
if (area > (largestByPhoto.get(row.photo_id) || 0)) {
|
||||
largestByPhoto.set(row.photo_id, area);
|
||||
}
|
||||
}
|
||||
|
||||
const cache = new Map();
|
||||
const byCategory = {};
|
||||
let assigned = 0;
|
||||
|
||||
for (const photo of photos) {
|
||||
const frameArea = (photo.width || 0) * (photo.height || 0);
|
||||
const ctx = {
|
||||
largestFaceAreaRatio: frameArea > 0
|
||||
? (largestByPhoto.get(photo.id) || 0) / frameArea
|
||||
: 0,
|
||||
};
|
||||
|
||||
const rule = RULES.find((r) => r.match(photo, ctx));
|
||||
if (!rule) continue;
|
||||
|
||||
const category = await resolveCategory(eventId, rule, cache);
|
||||
|
||||
// Guard the UPDATE on category_id still being NULL. Between the SELECT
|
||||
// above and here a photographer may have set one by hand, and their
|
||||
// choice wins — rule 1 is not a best-effort.
|
||||
const updated = await db('photos')
|
||||
.where({ id: photo.id })
|
||||
.whereNull('category_id')
|
||||
.update({ category_id: category.id, auto_categorized: true });
|
||||
|
||||
if (updated > 0) {
|
||||
assigned += updated;
|
||||
byCategory[rule.slug] = (byCategory[rule.slug] || 0) + updated;
|
||||
}
|
||||
}
|
||||
|
||||
logger.info(
|
||||
`faceAutoCategories: assigned ${assigned} photo(s) in event ${eventId} `
|
||||
+ `(${Object.entries(byCategory).map(([k, v]) => `${k}=${v}`).join(', ') || 'none'})`
|
||||
);
|
||||
return { assigned, byCategory };
|
||||
}
|
||||
|
||||
/**
|
||||
* Undo every automatic assignment for an event.
|
||||
*
|
||||
* Clears only rows this engine set — a photographer's own categories are
|
||||
* untouched, which is the entire reason `auto_categorized` exists.
|
||||
*/
|
||||
async function undoEvent(eventId) {
|
||||
const cleared = await db('photos')
|
||||
.where({ event_id: eventId, auto_categorized: true })
|
||||
.update({ category_id: null, auto_categorized: false });
|
||||
|
||||
logger.info(`faceAutoCategories: cleared ${cleared} automatic category assignment(s) in event ${eventId}`);
|
||||
return { cleared };
|
||||
}
|
||||
|
||||
module.exports = { RULES, categorizeEvent, undoEvent };
|
||||
@@ -0,0 +1,134 @@
|
||||
/**
|
||||
* HTTP client for the picpeak-ml sidecar (#1074).
|
||||
*
|
||||
* The important behaviour here is the error taxonomy, because the queue
|
||||
* treats the two classes completely differently:
|
||||
*
|
||||
* SidecarUnavailableError — the sidecar is down, unreachable, timing out,
|
||||
* or returned 5xx. The photo goes BACK to 'pending' and is retried later.
|
||||
* Turning the container off for a week must not require a manual re-scan.
|
||||
*
|
||||
* Everything else (4xx) — this image is a lost cause. The photo is marked
|
||||
* 'failed' and never retried, because retrying an undecodable file
|
||||
* forever is just a busy loop.
|
||||
*
|
||||
* Log volume matters too: an hour of downtime at a 1s poll is 3,600 identical
|
||||
* warnings. Unavailability is logged at most once per LOG_INTERVAL_MS.
|
||||
*/
|
||||
|
||||
const axios = require('axios');
|
||||
const logger = require('../utils/logger');
|
||||
const { getSidecarUrl, getSidecarToken } = require('./faceSettings');
|
||||
|
||||
const REQUEST_TIMEOUT_MS = parseInt(process.env.FACE_ML_TIMEOUT_MS || '30000', 10);
|
||||
const LOG_INTERVAL_MS = 5 * 60 * 1000;
|
||||
|
||||
let lastUnavailableLogAt = 0;
|
||||
|
||||
class SidecarUnavailableError extends Error {
|
||||
constructor(message) {
|
||||
super(message);
|
||||
this.name = 'SidecarUnavailableError';
|
||||
}
|
||||
}
|
||||
|
||||
class SidecarRejectedError extends Error {
|
||||
constructor(message, status) {
|
||||
super(message);
|
||||
this.name = 'SidecarRejectedError';
|
||||
this.status = status;
|
||||
}
|
||||
}
|
||||
|
||||
function logUnavailable(message) {
|
||||
const now = Date.now();
|
||||
if (now - lastUnavailableLogAt < LOG_INTERVAL_MS) return;
|
||||
lastUnavailableLogAt = now;
|
||||
logger.warn(
|
||||
`faceClient: ML sidecar unavailable (${message}). Photos stay queued and will ` +
|
||||
'be retried; no action needed unless this persists. Further identical warnings ' +
|
||||
'are suppressed for 5 minutes.'
|
||||
);
|
||||
}
|
||||
|
||||
function authHeaders() {
|
||||
const token = getSidecarToken();
|
||||
return token ? { 'X-Face-ML-Token': token } : {};
|
||||
}
|
||||
|
||||
/**
|
||||
* Translate an axios failure into our two-class taxonomy.
|
||||
*/
|
||||
function classify(err) {
|
||||
const status = err.response?.status;
|
||||
|
||||
if (status && status >= 400 && status < 500) {
|
||||
// 401 is a configuration error, not a bad image — but it is also not
|
||||
// something retrying fixes, so it surfaces loudly and stops the photo.
|
||||
if (status === 401) {
|
||||
logger.error(
|
||||
'faceClient: sidecar rejected our token (401). FACE_ML_TOKEN must match ' +
|
||||
'on both the backend and the picpeak-ml container.'
|
||||
);
|
||||
}
|
||||
return new SidecarRejectedError(
|
||||
err.response?.data?.detail || `Sidecar rejected the request (${status})`,
|
||||
status
|
||||
);
|
||||
}
|
||||
|
||||
logUnavailable(err.code || err.message || `HTTP ${status}`);
|
||||
return new SidecarUnavailableError(err.message || 'Sidecar unreachable');
|
||||
}
|
||||
|
||||
/**
|
||||
* Detect faces in an image. `buffer` is the preview rendition's bytes.
|
||||
* Returns the sidecar's `{ model_version, faces: [...] }`.
|
||||
*/
|
||||
async function detectFaces(buffer, filename = 'photo.jpg') {
|
||||
const form = new FormData();
|
||||
// Node 18+ ships FormData/Blob globally, so no multipart dependency is
|
||||
// needed for the one endpoint that uploads anything.
|
||||
form.append('image', new Blob([buffer]), filename);
|
||||
|
||||
try {
|
||||
const response = await axios.post(`${getSidecarUrl()}/faces`, form, {
|
||||
headers: authHeaders(),
|
||||
timeout: REQUEST_TIMEOUT_MS,
|
||||
// A 45MP preview is ~2MB; the cap is generous but not unbounded.
|
||||
maxBodyLength: 64 * 1024 * 1024,
|
||||
maxContentLength: 64 * 1024 * 1024,
|
||||
});
|
||||
return response.data;
|
||||
} catch (err) {
|
||||
throw classify(err);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Sidecar identity + liveness, for the admin connection test.
|
||||
* Returns { ok: true, info } or { ok: false, error } — never throws, because
|
||||
* the caller is a UI button and a stack trace helps nobody there.
|
||||
*/
|
||||
async function checkHealth() {
|
||||
try {
|
||||
const { data } = await axios.get(`${getSidecarUrl()}/info`, {
|
||||
headers: authHeaders(),
|
||||
timeout: 5000,
|
||||
});
|
||||
return { ok: true, info: data };
|
||||
} catch (err) {
|
||||
const status = err.response?.status;
|
||||
if (status === 401) {
|
||||
return { ok: false, error: 'Sidecar rejected the token (check FACE_ML_TOKEN on both containers)' };
|
||||
}
|
||||
return { ok: false, error: err.message || 'Sidecar unreachable' };
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
detectFaces,
|
||||
checkHealth,
|
||||
SidecarUnavailableError,
|
||||
SidecarRejectedError,
|
||||
};
|
||||
@@ -0,0 +1,533 @@
|
||||
/**
|
||||
* Per-event face clustering (#1074).
|
||||
*
|
||||
* Plain JS over a few thousand rows. A gallery is hundreds to low-thousands
|
||||
* of faces, so this needs no vector database and no native extension — the
|
||||
* embeddings live in a BLOB column and the maths is a dot product.
|
||||
*
|
||||
* Algorithm: greedy incremental assignment. A new face is compared against
|
||||
* every existing person centroid in the event; above the match threshold it
|
||||
* joins the nearest one and updates that centroid as a running mean,
|
||||
* otherwise it opens a new person.
|
||||
*
|
||||
* Greedy assignment is order-dependent by construction — the same photos
|
||||
* uploaded in a different order can produce different clusters. That is
|
||||
* accepted here (it is what Immich, PhotoPrism and Ente all do) and mitigated
|
||||
* by `consolidate()`, which merges centroid pairs that have drifted together,
|
||||
* plus the admin merge/split tools. What it buys is that a photo can be
|
||||
* clustered the moment it is scanned, so the strip fills in during a backfill
|
||||
* instead of after it.
|
||||
*
|
||||
* Embeddings arrive L2-normalized from the sidecar, so cosine similarity is a
|
||||
* plain dot product. Centroids are re-normalized after every update to keep
|
||||
* that true.
|
||||
*/
|
||||
|
||||
const { db } = require('../database/db');
|
||||
const logger = require('../utils/logger');
|
||||
const { getThresholds } = require('./faceSettings');
|
||||
|
||||
const FLOAT_BYTES = 4;
|
||||
|
||||
// -- serialization -----------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Float32Array → Buffer for the BLOB column. Little-endian on every platform
|
||||
* we ship (x86_64, aarch64), and the value never leaves the deployment, so
|
||||
* no byte-order header is needed.
|
||||
*/
|
||||
function packEmbedding(vec) {
|
||||
const arr = vec instanceof Float32Array ? vec : Float32Array.from(vec);
|
||||
return Buffer.from(arr.buffer, arr.byteOffset, arr.byteLength);
|
||||
}
|
||||
|
||||
function unpackEmbedding(buf) {
|
||||
if (!buf) return null;
|
||||
// Postgres bytea comes back as Buffer; SQLite may hand back a Uint8Array.
|
||||
const b = Buffer.isBuffer(buf) ? buf : Buffer.from(buf);
|
||||
if (b.length % FLOAT_BYTES !== 0) return null;
|
||||
// Copy rather than aliasing: a Buffer from the driver may be a view into a
|
||||
// larger pooled allocation, and Float32Array over an unaligned offset
|
||||
// throws.
|
||||
const copy = Buffer.from(b);
|
||||
return new Float32Array(copy.buffer, copy.byteOffset, copy.length / FLOAT_BYTES);
|
||||
}
|
||||
|
||||
// -- vector maths ------------------------------------------------------------
|
||||
|
||||
function dot(a, b) {
|
||||
let sum = 0;
|
||||
const n = Math.min(a.length, b.length);
|
||||
for (let i = 0; i < n; i++) sum += a[i] * b[i];
|
||||
return sum;
|
||||
}
|
||||
|
||||
function normalize(vec) {
|
||||
let sumSq = 0;
|
||||
for (let i = 0; i < vec.length; i++) sumSq += vec[i] * vec[i];
|
||||
const norm = Math.sqrt(sumSq);
|
||||
if (norm === 0) return vec;
|
||||
const out = new Float32Array(vec.length);
|
||||
for (let i = 0; i < vec.length; i++) out[i] = vec[i] / norm;
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Running mean of `count` existing vectors with one new vector, re-normalized.
|
||||
*/
|
||||
function updateCentroid(centroid, count, incoming) {
|
||||
const out = new Float32Array(centroid.length);
|
||||
for (let i = 0; i < centroid.length; i++) {
|
||||
out[i] = (centroid[i] * count + incoming[i]) / (count + 1);
|
||||
}
|
||||
return normalize(out);
|
||||
}
|
||||
|
||||
// -- quality -----------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Is this face good enough to define a person?
|
||||
*
|
||||
* Low-quality faces are still STORED and still shown in "this photo contains"
|
||||
* — they just don't get assigned, so they can't spawn junk people or drag a
|
||||
* good centroid off course. A blurry profile at 30px is a real detection and
|
||||
* a terrible identity.
|
||||
*/
|
||||
function meetsQualityFloor(face, thresholds) {
|
||||
if (face.det_score != null && face.det_score < thresholds.face_quality_min_score) return false;
|
||||
const size = Math.min(face.bbox_w ?? 0, face.bbox_h ?? 0);
|
||||
if (size < thresholds.face_quality_min_px) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
// -- clustering --------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Assign a set of freshly-inserted faces to people within one event.
|
||||
*
|
||||
* Loads the event's people once, mutates centroids in memory across the whole
|
||||
* batch, then writes back — so a photo with five faces costs one read and one
|
||||
* write pass rather than five of each.
|
||||
*/
|
||||
/**
|
||||
* Per-event serialization for cluster assignment.
|
||||
*
|
||||
* assignFaces is read-modify-write over an event's people: it loads every
|
||||
* centroid, mutates them in memory across the batch, then writes back. Two
|
||||
* workers on the same event therefore lose updates — both read the same
|
||||
* snapshot, and the second write clobbers the first (or both open a duplicate
|
||||
* person for the same face). The default concurrency is 1, but the queue
|
||||
* advertises multi-pod safety and the tunable exists, so this cannot rely on
|
||||
* there only ever being one writer.
|
||||
*
|
||||
* In-process mutex + (on Postgres) a transaction-scoped advisory lock keyed on
|
||||
* the event. The advisory lock is what covers multiple pods; the mutex avoids
|
||||
* pointless lock round-trips within one process. SQLite is single-writer
|
||||
* anyway, so the mutex alone is sufficient there.
|
||||
*/
|
||||
const eventLocks = new Map();
|
||||
|
||||
async function withEventLock(eventId, trx, fn) {
|
||||
const previous = eventLocks.get(eventId) || Promise.resolve();
|
||||
let release;
|
||||
const current = new Promise((resolve) => { release = resolve; });
|
||||
eventLocks.set(eventId, previous.then(() => current));
|
||||
|
||||
await previous;
|
||||
try {
|
||||
// pg_advisory_xact_lock is released automatically when the transaction
|
||||
// ends, including on rollback — no leak if the caller throws.
|
||||
if (trx && typeof trx.raw === 'function') {
|
||||
const client = db.client.config.client;
|
||||
const isPg = client === 'pg' || (typeof client === 'string' && client.includes('postgres'));
|
||||
if (isPg) {
|
||||
await trx.raw('SELECT pg_advisory_xact_lock(?, ?)', [1074, Number(eventId)]);
|
||||
}
|
||||
}
|
||||
return await fn();
|
||||
} finally {
|
||||
release();
|
||||
if (eventLocks.get(eventId) === current) eventLocks.delete(eventId);
|
||||
}
|
||||
}
|
||||
|
||||
async function assignFaces(eventId, faceRows, options = {}) {
|
||||
const thresholds = options.thresholds || (await getThresholds());
|
||||
const trx = options.trx || db;
|
||||
|
||||
return withEventLock(eventId, options.trx, () => assignFacesLocked(eventId, faceRows, thresholds, trx));
|
||||
}
|
||||
|
||||
async function assignFacesLocked(eventId, faceRows, thresholds, trx) {
|
||||
const people = await trx('event_people')
|
||||
.where({ event_id: eventId })
|
||||
.select('id', 'centroid', 'face_count_total', 'model_version');
|
||||
|
||||
const state = people.map((p) => ({
|
||||
id: p.id,
|
||||
centroid: unpackEmbedding(p.centroid),
|
||||
count: p.face_count_total || 0,
|
||||
dirty: false,
|
||||
modelVersion: p.model_version,
|
||||
})).filter((p) => p.centroid);
|
||||
|
||||
const assignments = [];
|
||||
|
||||
for (const face of faceRows) {
|
||||
const embedding = unpackEmbedding(face.embedding);
|
||||
if (!embedding) continue;
|
||||
|
||||
if (!meetsQualityFloor(face, thresholds)) {
|
||||
assignments.push({ faceId: face.id, personId: null });
|
||||
continue;
|
||||
}
|
||||
|
||||
let best = null;
|
||||
let bestScore = -Infinity;
|
||||
for (const person of state) {
|
||||
// Never compare across embedding spaces — a model change makes old
|
||||
// centroids meaningless rather than merely stale.
|
||||
if (person.modelVersion && face.model_version && person.modelVersion !== face.model_version) {
|
||||
continue;
|
||||
}
|
||||
const score = dot(embedding, person.centroid);
|
||||
if (score > bestScore) {
|
||||
bestScore = score;
|
||||
best = person;
|
||||
}
|
||||
}
|
||||
|
||||
if (best && bestScore >= thresholds.face_match_threshold) {
|
||||
best.centroid = updateCentroid(best.centroid, best.count, embedding);
|
||||
best.count += 1;
|
||||
best.dirty = true;
|
||||
assignments.push({ faceId: face.id, personId: best.id });
|
||||
} else {
|
||||
const [inserted] = await trx('event_people').insert({
|
||||
event_id: eventId,
|
||||
centroid: packEmbedding(embedding),
|
||||
face_count_total: 1,
|
||||
model_version: face.model_version,
|
||||
cover_face_id: face.id,
|
||||
created_at: new Date().toISOString(),
|
||||
updated_at: new Date().toISOString(),
|
||||
}).returning('id');
|
||||
const personId = typeof inserted === 'object' ? inserted.id : inserted;
|
||||
|
||||
state.push({
|
||||
id: personId,
|
||||
centroid: embedding,
|
||||
count: 1,
|
||||
dirty: false,
|
||||
modelVersion: face.model_version,
|
||||
});
|
||||
assignments.push({ faceId: face.id, personId });
|
||||
}
|
||||
}
|
||||
|
||||
for (const { faceId, personId } of assignments) {
|
||||
await trx('photo_faces').where({ id: faceId }).update({ person_id: personId });
|
||||
}
|
||||
|
||||
for (const person of state) {
|
||||
if (!person.dirty) continue;
|
||||
await trx('event_people').where({ id: person.id }).update({
|
||||
centroid: packEmbedding(person.centroid),
|
||||
face_count_total: person.count,
|
||||
updated_at: new Date().toISOString(),
|
||||
});
|
||||
}
|
||||
|
||||
return assignments;
|
||||
}
|
||||
|
||||
/**
|
||||
* Merge people whose centroids have drifted together.
|
||||
*
|
||||
* Greedy assignment can open "Anna in daylight" and "Anna at the party" as
|
||||
* two clusters when the first few photos of each were dissimilar. Once both
|
||||
* have absorbed enough faces their centroids converge, and this pass catches
|
||||
* that. Runs at a slightly stricter threshold than initial assignment:
|
||||
* merging two established clusters is a bigger claim than adding one face to
|
||||
* one of them, and an over-eager merge is much harder for a photographer to
|
||||
* unpick than a missed one.
|
||||
*/
|
||||
async function consolidate(eventId, options = {}) {
|
||||
const thresholds = options.thresholds || (await getThresholds());
|
||||
const mergeThreshold = Math.min(0.95, thresholds.face_match_threshold + 0.08);
|
||||
|
||||
const people = await db('event_people')
|
||||
.where({ event_id: eventId })
|
||||
.select('id', 'centroid', 'face_count_total', 'model_version', 'label');
|
||||
|
||||
const state = people
|
||||
.map((p) => ({ ...p, vec: unpackEmbedding(p.centroid) }))
|
||||
.filter((p) => p.vec);
|
||||
|
||||
const merged = [];
|
||||
const absorbed = new Set();
|
||||
|
||||
for (let i = 0; i < state.length; i++) {
|
||||
if (absorbed.has(state[i].id)) continue;
|
||||
for (let j = i + 1; j < state.length; j++) {
|
||||
if (absorbed.has(state[j].id)) continue;
|
||||
const a = state[i];
|
||||
const b = state[j];
|
||||
if (a.model_version !== b.model_version) continue;
|
||||
|
||||
// Never silently merge two people the photographer has NAMED
|
||||
// differently — that is a human assertion this heuristic does not get
|
||||
// to overrule.
|
||||
if (a.label && b.label && a.label !== b.label) continue;
|
||||
|
||||
if (dot(a.vec, b.vec) >= mergeThreshold) {
|
||||
await mergePeople(eventId, [b.id], a.id);
|
||||
absorbed.add(b.id);
|
||||
merged.push({ from: b.id, into: a.id });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (merged.length) {
|
||||
logger.info(`faceClustering: consolidated ${merged.length} person pair(s) in event ${eventId}`);
|
||||
}
|
||||
return merged;
|
||||
}
|
||||
|
||||
/**
|
||||
* Move every face from `sourceIds` onto `targetId` and delete the sources.
|
||||
* Recomputes the target centroid from its actual members rather than
|
||||
* averaging the two centroids — cheap at this scale, and exact.
|
||||
*/
|
||||
async function mergePeople(eventId, sourceIds, targetId) {
|
||||
const ids = sourceIds.filter((id) => id !== targetId);
|
||||
if (!ids.length) return { moved: 0 };
|
||||
|
||||
return db.transaction(async (trx) => {
|
||||
// Carry metadata forward before the sources are deleted. Without this a
|
||||
// merge silently discards a photographer-entered name, or un-hides a
|
||||
// person they had suppressed — the target keeps its own values where it
|
||||
// has them, and inherits from a source only where it does not.
|
||||
const target = await trx('event_people').where({ id: targetId }).first();
|
||||
const sources = await trx('event_people').whereIn('id', ids).select('label', 'is_hidden', 'is_ignored');
|
||||
|
||||
const inherited = {};
|
||||
if (target && !target.label) {
|
||||
const named = sources.find((p) => p.label);
|
||||
if (named) inherited.label = named.label;
|
||||
}
|
||||
// Suppression is one-way on merge: if ANY party was hidden or ignored,
|
||||
// the survivor stays that way. Re-exposing someone by merging is the
|
||||
// failure that matters; leaving them hidden is trivially reversible.
|
||||
if (sources.some((p) => p.is_hidden) || target?.is_hidden) inherited.is_hidden = true;
|
||||
if (sources.some((p) => p.is_ignored) || target?.is_ignored) inherited.is_ignored = true;
|
||||
|
||||
const moved = await trx('photo_faces')
|
||||
.where({ event_id: eventId })
|
||||
.whereIn('person_id', ids)
|
||||
.update({ person_id: targetId });
|
||||
|
||||
await trx('event_people').where({ event_id: eventId }).whereIn('id', ids).del();
|
||||
|
||||
if (Object.keys(inherited).length) {
|
||||
await trx('event_people').where({ id: targetId })
|
||||
.update({ ...inherited, updated_at: new Date().toISOString() });
|
||||
}
|
||||
|
||||
await recomputeCentroid(targetId, trx);
|
||||
return { moved };
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Pull `faceIds` out of their cluster into a brand-new person.
|
||||
*/
|
||||
async function splitPerson(eventId, personId, faceIds) {
|
||||
if (!faceIds?.length) return null;
|
||||
|
||||
return db.transaction(async (trx) => {
|
||||
const faces = await trx('photo_faces')
|
||||
.where({ event_id: eventId, person_id: personId })
|
||||
.whereIn('id', faceIds)
|
||||
.select('id', 'embedding', 'model_version');
|
||||
|
||||
if (!faces.length) return null;
|
||||
|
||||
const [inserted] = await trx('event_people').insert({
|
||||
event_id: eventId,
|
||||
face_count_total: 0,
|
||||
model_version: faces[0].model_version,
|
||||
created_at: new Date().toISOString(),
|
||||
updated_at: new Date().toISOString(),
|
||||
}).returning('id');
|
||||
const newPersonId = typeof inserted === 'object' ? inserted.id : inserted;
|
||||
|
||||
await trx('photo_faces')
|
||||
.whereIn('id', faces.map((f) => f.id))
|
||||
.update({ person_id: newPersonId });
|
||||
|
||||
await recomputeCentroid(newPersonId, trx);
|
||||
await recomputeCentroid(personId, trx);
|
||||
return newPersonId;
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Recompute one person's centroid and count from its member faces.
|
||||
* Deletes the person if it has no members left.
|
||||
*/
|
||||
async function recomputeCentroid(personId, trx = db) {
|
||||
const faces = await trx('photo_faces')
|
||||
.where({ person_id: personId })
|
||||
.select('id', 'embedding', 'det_score');
|
||||
|
||||
if (!faces.length) {
|
||||
await trx('event_people').where({ id: personId }).del();
|
||||
return;
|
||||
}
|
||||
|
||||
const vectors = faces.map((f) => unpackEmbedding(f.embedding)).filter(Boolean);
|
||||
if (!vectors.length) return;
|
||||
|
||||
const mean = new Float32Array(vectors[0].length);
|
||||
for (const vec of vectors) {
|
||||
for (let i = 0; i < mean.length; i++) mean[i] += vec[i];
|
||||
}
|
||||
for (let i = 0; i < mean.length; i++) mean[i] /= vectors.length;
|
||||
|
||||
// Cover face: the highest-scoring detection in the cluster, so the avatar
|
||||
// is the sharpest available crop of that person rather than whichever face
|
||||
// happened to arrive first.
|
||||
const cover = faces.reduce((a, b) => ((b.det_score ?? 0) > (a.det_score ?? 0) ? b : a));
|
||||
|
||||
await trx('event_people').where({ id: personId }).update({
|
||||
centroid: packEmbedding(normalize(mean)),
|
||||
face_count_total: faces.length,
|
||||
cover_face_id: cover.id,
|
||||
updated_at: new Date().toISOString(),
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Re-derive every cluster in an event from stored embeddings, without
|
||||
* touching the sidecar. Needed after threshold tuning — the expensive part
|
||||
* (inference) is already done and cached in the rows.
|
||||
*
|
||||
* Preserves labels by re-attaching them to whichever new cluster inherited
|
||||
* the most faces from the old one. Without this, re-clustering a gallery
|
||||
* would silently discard every name the photographer typed.
|
||||
*/
|
||||
async function recluster(eventId) {
|
||||
const thresholds = await getThresholds();
|
||||
|
||||
// Remember every person carrying HUMAN state — a name, or a hidden/ignored
|
||||
// decision. Keying this on `label` alone silently dropped the privacy flags
|
||||
// of unnamed people: a bystander the photographer had suppressed came back
|
||||
// guest-visible after one "Re-group people".
|
||||
const previousLabels = await db('event_people')
|
||||
.where({ event_id: eventId })
|
||||
.where(function () {
|
||||
this.whereNotNull('label').orWhere('is_hidden', true).orWhere('is_ignored', true);
|
||||
})
|
||||
.select('id', 'label', 'is_hidden', 'is_ignored');
|
||||
|
||||
const priorMembership = new Map();
|
||||
if (previousLabels.length) {
|
||||
const rows = await db('photo_faces')
|
||||
.where({ event_id: eventId })
|
||||
.whereIn('person_id', previousLabels.map((p) => p.id))
|
||||
.select('id', 'person_id');
|
||||
for (const row of rows) priorMembership.set(row.id, row.person_id);
|
||||
}
|
||||
|
||||
await db.transaction(async (trx) => {
|
||||
await trx('photo_faces').where({ event_id: eventId }).update({ person_id: null });
|
||||
await trx('event_people').where({ event_id: eventId }).del();
|
||||
});
|
||||
|
||||
const faces = await db('photo_faces')
|
||||
.where({ event_id: eventId })
|
||||
.orderBy('id', 'asc')
|
||||
.select('id', 'embedding', 'model_version', 'det_score', 'bbox_w', 'bbox_h');
|
||||
|
||||
const assignments = await assignFaces(eventId, faces, { thresholds });
|
||||
|
||||
// Re-attach labels by majority inheritance.
|
||||
if (previousLabels.length) {
|
||||
const tally = new Map(); // newPersonId -> Map<oldPersonId, count>
|
||||
for (const { faceId, personId } of assignments) {
|
||||
if (personId == null) continue;
|
||||
const oldId = priorMembership.get(faceId);
|
||||
if (oldId == null) continue;
|
||||
if (!tally.has(personId)) tally.set(personId, new Map());
|
||||
const inner = tally.get(personId);
|
||||
inner.set(oldId, (inner.get(oldId) || 0) + 1);
|
||||
}
|
||||
|
||||
// Suppression is OR-ed across EVERY ancestor that contributed faces to a
|
||||
// new cluster — not copied from the majority one. Reclustering can merge a
|
||||
// visible named person with a hidden one; taking the majority ancestor's
|
||||
// flags would then publish the hidden person's photos. Erring toward
|
||||
// staying hidden is trivially reversible; erring toward visible is not.
|
||||
const suppression = new Map(); // newId -> { is_hidden, is_ignored }
|
||||
for (const [newId, inner] of tally) {
|
||||
let hidden = false;
|
||||
let ignored = false;
|
||||
for (const oldId of inner.keys()) {
|
||||
const old = previousLabels.find((p) => p.id === oldId);
|
||||
if (!old) continue;
|
||||
if (old.is_hidden) hidden = true;
|
||||
if (old.is_ignored) ignored = true;
|
||||
}
|
||||
suppression.set(newId, { is_hidden: hidden, is_ignored: ignored });
|
||||
}
|
||||
|
||||
// The NAME goes to exactly one cluster: the descendant that inherited the
|
||||
// MOST of that old person's faces, chosen globally. Iterating `tally` and
|
||||
// taking the first match gave the name to whichever cluster happened to
|
||||
// come first — an early outlier could take it while the real majority
|
||||
// cluster ended up unnamed.
|
||||
const bestDescendant = new Map(); // oldId -> { newId, count }
|
||||
for (const [newId, inner] of tally) {
|
||||
for (const [oldId, count] of inner) {
|
||||
const current = bestDescendant.get(oldId);
|
||||
if (!current || count > current.count) bestDescendant.set(oldId, { newId, count });
|
||||
}
|
||||
}
|
||||
|
||||
const labelFor = new Map(); // newId -> label
|
||||
for (const [oldId, { newId }] of bestDescendant) {
|
||||
const old = previousLabels.find((p) => p.id === oldId);
|
||||
if (old?.label && !labelFor.has(newId)) labelFor.set(newId, old.label);
|
||||
}
|
||||
|
||||
for (const [newId, flags] of suppression) {
|
||||
const update = { ...flags, updated_at: new Date().toISOString() };
|
||||
if (labelFor.has(newId)) update.label = labelFor.get(newId);
|
||||
await db('event_people').where({ id: newId }).update(update);
|
||||
}
|
||||
}
|
||||
|
||||
for (const personId of new Set(assignments.map((a) => a.personId).filter(Boolean))) {
|
||||
await recomputeCentroid(personId);
|
||||
}
|
||||
await consolidate(eventId, { thresholds });
|
||||
|
||||
const count = await db('event_people').where({ event_id: eventId }).count({ c: '*' }).first();
|
||||
logger.info(`faceClustering: reclustered event ${eventId} → ${count?.c ?? 0} people`);
|
||||
return Number(count?.c ?? 0);
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
packEmbedding,
|
||||
unpackEmbedding,
|
||||
dot,
|
||||
normalize,
|
||||
meetsQualityFloor,
|
||||
assignFaces,
|
||||
consolidate,
|
||||
mergePeople,
|
||||
splitPerson,
|
||||
recomputeCentroid,
|
||||
recluster,
|
||||
};
|
||||
@@ -0,0 +1,247 @@
|
||||
/**
|
||||
* People queries for the gallery and admin surfaces (#1074).
|
||||
*
|
||||
* THE RULE THIS MODULE EXISTS TO ENFORCE: a guest's view of "who is in this
|
||||
* gallery" must be computed over the photos that guest can actually see.
|
||||
*
|
||||
* Guests are restricted to `photos.visibility = 'visible'` (gallery.js);
|
||||
* PIN-clients see everything. `event_people.face_count_total` counts ALL
|
||||
* faces, so handing it to a guest leaks the existence and volume of hidden
|
||||
* photos — and a cover face chosen from a hidden photo would render a crop of
|
||||
* an image the guest was never allowed to open. Neither is theoretical: a
|
||||
* photographer hides shots precisely because someone should not see them.
|
||||
*
|
||||
* So the guest-facing count and cover are recomputed per request against the
|
||||
* same predicate the photo query uses. `face_count_total` is never returned
|
||||
* to a guest.
|
||||
*/
|
||||
|
||||
const { db } = require('../database/db');
|
||||
|
||||
/**
|
||||
* Apply the visibility predicate a given audience is allowed to see.
|
||||
* Mirrors gallery.js: guests get 'visible' (or NULL, pre-migration rows),
|
||||
* clients get everything. Also excludes photos still being processed, which
|
||||
* are invisible in the gallery payload too.
|
||||
*/
|
||||
function applyVisibilityScope(query, { isClient }) {
|
||||
query.where(function () {
|
||||
this.where('photos.processing_status', 'complete').orWhereNull('photos.processing_status');
|
||||
});
|
||||
if (!isClient) {
|
||||
query.where(function () {
|
||||
this.where('photos.visibility', 'visible').orWhereNull('photos.visibility');
|
||||
});
|
||||
}
|
||||
return query;
|
||||
}
|
||||
|
||||
/**
|
||||
* People in an event, scoped to what this audience can see.
|
||||
*
|
||||
* Returns [{ id, label, face_count, cover: { photo_id, bbox } }] ordered by
|
||||
* visible photo count. People whose visible count falls below `minClusterSize`
|
||||
* are dropped — a person who appears in three hidden photos and one visible
|
||||
* one is not "in this gallery" from the guest's point of view.
|
||||
*/
|
||||
async function listPeople(eventId, { isClient = false, forAdmin = false, minClusterSize = 3 } = {}) {
|
||||
const people = await db('event_people')
|
||||
.where({ event_id: eventId })
|
||||
.modify((q) => {
|
||||
if (!forAdmin) {
|
||||
// Hidden = photographer-only. Ignored = bystander/false positive.
|
||||
// Neither ever reaches a guest response.
|
||||
q.where('is_hidden', false).where('is_ignored', false);
|
||||
}
|
||||
})
|
||||
.select('id', 'label', 'is_hidden', 'is_ignored', 'face_count_total', 'cover_face_id');
|
||||
|
||||
if (!people.length) return [];
|
||||
|
||||
// One grouped query for the audience-scoped counts, rather than N queries.
|
||||
const counts = await applyVisibilityScope(
|
||||
db('photo_faces')
|
||||
.join('photos', 'photos.id', 'photo_faces.photo_id')
|
||||
.where('photo_faces.event_id', eventId)
|
||||
.whereNotNull('photo_faces.person_id'),
|
||||
{ isClient }
|
||||
)
|
||||
.groupBy('photo_faces.person_id')
|
||||
.select('photo_faces.person_id')
|
||||
.countDistinct({ visible_photos: 'photos.id' });
|
||||
|
||||
const countByPerson = new Map(
|
||||
counts.map((r) => [r.person_id, Number(r.visible_photos) || 0])
|
||||
);
|
||||
|
||||
// Cover faces, also scoped: the stored cover_face_id may point at a face in
|
||||
// a photo this audience cannot see. Pick the best VISIBLE face instead.
|
||||
const covers = await applyVisibilityScope(
|
||||
db('photo_faces')
|
||||
.join('photos', 'photos.id', 'photo_faces.photo_id')
|
||||
.where('photo_faces.event_id', eventId)
|
||||
.whereNotNull('photo_faces.person_id'),
|
||||
{ isClient }
|
||||
)
|
||||
.orderBy('photo_faces.det_score', 'desc')
|
||||
.select(
|
||||
'photo_faces.id',
|
||||
'photo_faces.person_id',
|
||||
'photo_faces.photo_id',
|
||||
'photo_faces.bbox_x',
|
||||
'photo_faces.bbox_y',
|
||||
'photo_faces.bbox_w',
|
||||
'photo_faces.bbox_h',
|
||||
// The bbox is in ORIGINAL image pixels, so any consumer cropping it has
|
||||
// to know what those pixels were measured against. Without these the
|
||||
// admin manager was scaling an original-space box by a THUMBNAIL's
|
||||
// natural size and rendering the wrong region entirely.
|
||||
'photos.width as photo_width',
|
||||
'photos.height as photo_height'
|
||||
);
|
||||
|
||||
const coverByPerson = new Map();
|
||||
for (const row of covers) {
|
||||
// Rows arrive best-score-first, so the first hit per person wins.
|
||||
if (!coverByPerson.has(row.person_id)) coverByPerson.set(row.person_id, row);
|
||||
}
|
||||
|
||||
const out = [];
|
||||
for (const person of people) {
|
||||
const count = countByPerson.get(person.id) || 0;
|
||||
const cover = coverByPerson.get(person.id);
|
||||
|
||||
// A person with no visible photos, or too few to be worth a face in the
|
||||
// strip, simply does not exist for this audience.
|
||||
if (!forAdmin && (count < minClusterSize || !cover)) continue;
|
||||
if (forAdmin && !cover && count === 0) {
|
||||
out.push({
|
||||
id: person.id,
|
||||
label: person.label || null,
|
||||
face_count: 0,
|
||||
total_face_count: person.face_count_total,
|
||||
is_hidden: !!person.is_hidden,
|
||||
is_ignored: !!person.is_ignored,
|
||||
cover: null,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
|
||||
const entry = {
|
||||
id: person.id,
|
||||
label: person.label || null,
|
||||
face_count: count,
|
||||
cover: cover
|
||||
? {
|
||||
face_id: cover.id,
|
||||
photo_id: cover.photo_id,
|
||||
bbox: [cover.bbox_x, cover.bbox_y, cover.bbox_w, cover.bbox_h],
|
||||
photo_width: cover.photo_width ?? null,
|
||||
photo_height: cover.photo_height ?? null,
|
||||
}
|
||||
: null,
|
||||
};
|
||||
|
||||
if (forAdmin) {
|
||||
// face_count_total is admin-only by design — see the module header.
|
||||
entry.total_face_count = person.face_count_total;
|
||||
entry.is_hidden = !!person.is_hidden;
|
||||
entry.is_ignored = !!person.is_ignored;
|
||||
}
|
||||
|
||||
out.push(entry);
|
||||
}
|
||||
|
||||
return out.sort((a, b) => b.face_count - a.face_count);
|
||||
}
|
||||
|
||||
/**
|
||||
* Map of photoId → person ids, for the photos in `photoIds`.
|
||||
*
|
||||
* Only ever called with ids already present in the gallery payload, which is
|
||||
* itself visibility-filtered — so this cannot widen what a caller sees. It
|
||||
* still filters hidden/ignored people, so a hidden person leaves no trace on
|
||||
* a photo the guest CAN see.
|
||||
*/
|
||||
async function getPersonIdsByPhoto(eventId, photoIds, { forAdmin = false } = {}) {
|
||||
if (!photoIds?.length) return new Map();
|
||||
|
||||
const rows = await db('photo_faces')
|
||||
.join('event_people', 'event_people.id', 'photo_faces.person_id')
|
||||
.where('photo_faces.event_id', eventId)
|
||||
.whereIn('photo_faces.photo_id', photoIds)
|
||||
.whereNotNull('photo_faces.person_id')
|
||||
.modify((q) => {
|
||||
if (!forAdmin) {
|
||||
q.where('event_people.is_hidden', false).where('event_people.is_ignored', false);
|
||||
}
|
||||
})
|
||||
.select('photo_faces.photo_id', 'photo_faces.person_id');
|
||||
|
||||
const map = new Map();
|
||||
for (const row of rows) {
|
||||
if (!map.has(row.photo_id)) map.set(row.photo_id, []);
|
||||
const list = map.get(row.photo_id);
|
||||
if (!list.includes(row.person_id)) list.push(row.person_id);
|
||||
}
|
||||
return map;
|
||||
}
|
||||
|
||||
/**
|
||||
* Scan progress for the admin status line and the gallery's "Finding
|
||||
* people… 240/1200" indicator.
|
||||
*/
|
||||
async function getScanStatus(eventId, { isClient = true } = {}) {
|
||||
// `isClient` defaults to TRUE (the admin/photographer view) because every
|
||||
// existing caller is an admin surface. A guest must pass isClient:false:
|
||||
// counting every photo with a face_status would otherwise tell them how
|
||||
// many hidden photos the gallery holds — the same leak the people list and
|
||||
// covers are already scoped against, arriving through the progress bar.
|
||||
const rows = await applyVisibilityScope(
|
||||
db('photos').where({ event_id: eventId }).whereNotNull('face_status'),
|
||||
{ isClient }
|
||||
)
|
||||
.groupBy('face_status')
|
||||
.select('face_status')
|
||||
.count({ c: '*' });
|
||||
|
||||
const byStatus = Object.fromEntries(rows.map((r) => [r.face_status, Number(r.c) || 0]));
|
||||
const done = byStatus.done || 0;
|
||||
const total = Object.values(byStatus).reduce((a, b) => a + b, 0);
|
||||
|
||||
const peopleRow = await db('event_people')
|
||||
.where({ event_id: eventId })
|
||||
.count({ c: '*' })
|
||||
.first();
|
||||
|
||||
// Two counts, deliberately. `people` is every cluster that exists;
|
||||
// `people_visible_to_guests` applies the minimum-cluster-size floor and the
|
||||
// hidden/ignored flags, i.e. what the gallery strip actually shows. An
|
||||
// admin comparing the settings page against their own gallery will
|
||||
// otherwise see two numbers that disagree with no explanation.
|
||||
const { getThresholds } = require('./faceSettings');
|
||||
const thresholds = await getThresholds();
|
||||
const visible = await listPeople(eventId, {
|
||||
isClient: false,
|
||||
forAdmin: false,
|
||||
minClusterSize: thresholds.face_min_cluster_size,
|
||||
});
|
||||
|
||||
return {
|
||||
scanned: done,
|
||||
total,
|
||||
pending: (byStatus.pending || 0) + (byStatus.processing || 0),
|
||||
failed: byStatus.failed || 0,
|
||||
skipped: byStatus.skipped || 0,
|
||||
people: Number(peopleRow?.c ?? 0),
|
||||
people_visible_to_guests: visible.length,
|
||||
in_progress: ((byStatus.pending || 0) + (byStatus.processing || 0)) > 0,
|
||||
};
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
listPeople,
|
||||
getPersonIdsByPhoto,
|
||||
getScanStatus,
|
||||
applyVisibilityScope,
|
||||
};
|
||||
@@ -0,0 +1,325 @@
|
||||
/**
|
||||
* Per-photo face detection (#1074).
|
||||
*
|
||||
* Runs on the PREVIEW rendition, not the original. A 1920px JPEG is ~40x
|
||||
* cheaper to decode than a 45MP original and costs nothing in recall at the
|
||||
* face sizes an event gallery actually contains.
|
||||
*
|
||||
* Note the side effect that has: on an install with `lightbox_preview_enabled`
|
||||
* off, no previews exist, so a face backfill generates the whole preview tier
|
||||
* as it goes — a real Sharp workload and real disk. That is why the admin
|
||||
* toggle warns about it and why the face queue defaults to concurrency 1.
|
||||
*/
|
||||
|
||||
const sharp = require('sharp');
|
||||
const { db } = require('../database/db');
|
||||
const logger = require('../utils/logger');
|
||||
const { getStorage } = require('./storage');
|
||||
const { ensurePreviewImage } = require('./imageProcessor');
|
||||
// SidecarUnavailableError is deliberately NOT caught here — it propagates to
|
||||
// faceQueue, which is the only layer that knows to retry rather than fail.
|
||||
const { detectFaces } = require('./faceClient');
|
||||
const { assignFaces, packEmbedding, recomputeCentroid } = require('./faceClustering');
|
||||
const { getThresholds, isEnabledForEvent } = require('./faceSettings');
|
||||
|
||||
/**
|
||||
* Thrown when a scan finishes but the photo is no longer the row we claimed —
|
||||
* the event was purged, archived or re-queued mid-flight. Not a failure of
|
||||
* the photo, so the queue must not mark it 'failed'.
|
||||
*/
|
||||
class StaleScanError extends Error {
|
||||
constructor(photoId) {
|
||||
super(`Face scan for photo ${photoId} was superseded`);
|
||||
this.name = 'StaleScanError';
|
||||
}
|
||||
}
|
||||
|
||||
async function streamToBuffer(stream) {
|
||||
if (Buffer.isBuffer(stream)) return stream;
|
||||
const chunks = [];
|
||||
for await (const chunk of stream) {
|
||||
chunks.push(Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk));
|
||||
}
|
||||
return Buffer.concat(chunks);
|
||||
}
|
||||
|
||||
/**
|
||||
* Detect and cluster the faces in one photo.
|
||||
*
|
||||
* Throws SidecarUnavailableError when the sidecar is down — the caller must
|
||||
* return the row to 'pending' rather than failing it.
|
||||
*/
|
||||
async function processPhotoFaces(photoId) {
|
||||
const photo = await db('photos').where({ id: photoId }).first();
|
||||
if (!photo) {
|
||||
logger.warn(`faceProcessor: photo ${photoId} disappeared before processing`);
|
||||
return { status: 'skipped' };
|
||||
}
|
||||
|
||||
const event = await db('events').where({ id: photo.event_id }).first();
|
||||
if (!(await isEnabledForEvent(event))) {
|
||||
// The toggle was switched off between enqueue and claim. Not an error.
|
||||
await db('photos').where({ id: photoId }).update({
|
||||
face_status: 'skipped', face_started_at: null, face_error: null,
|
||||
});
|
||||
return { status: 'skipped' };
|
||||
}
|
||||
|
||||
// Videos have no preview tier and no meaningful single frame to scan.
|
||||
const isVideo = photo.media_type === 'video'
|
||||
|| (photo.mime_type && photo.mime_type.startsWith('video/'));
|
||||
if (isVideo) {
|
||||
await db('photos').where({ id: photoId }).update({
|
||||
face_status: 'skipped', face_started_at: null, face_count: 0,
|
||||
});
|
||||
return { status: 'skipped' };
|
||||
}
|
||||
|
||||
// External / reference photos live outside managed storage, and
|
||||
// resolvePhotoStorageKey returns null for them by design (photoResolver.js).
|
||||
// ensurePreviewImage therefore cannot build a preview, so scanning them is
|
||||
// unsupported rather than broken — 'skipped', not 'failed', so an external
|
||||
// gallery does not report every photo as an error.
|
||||
if (photo.source_origin === 'external' || photo.source_origin === 'reference') {
|
||||
await db('photos').where({ id: photoId }).update({
|
||||
face_status: 'skipped', face_started_at: null, face_error: null,
|
||||
});
|
||||
return { status: 'skipped' };
|
||||
}
|
||||
|
||||
const previewKey = await ensurePreviewImage(photo);
|
||||
if (!previewKey) {
|
||||
// No preview and no way to make one — the source is missing or corrupt.
|
||||
// That is a property of this photo, so it fails rather than retries.
|
||||
await db('photos').where({ id: photoId }).update({
|
||||
face_status: 'failed',
|
||||
face_started_at: null,
|
||||
face_error: 'Could not generate a preview rendition to scan',
|
||||
});
|
||||
return { status: 'failed' };
|
||||
}
|
||||
|
||||
const storage = getStorage();
|
||||
const buffer = await streamToBuffer(await storage.get(previewKey));
|
||||
|
||||
// Throws SidecarUnavailableError upward on a down sidecar — deliberately
|
||||
// not caught here, so the queue can distinguish "retry" from "failed".
|
||||
const result = await detectFaces(buffer, photo.filename || 'photo.jpg');
|
||||
const faces = result?.faces || [];
|
||||
const modelVersion = result?.model_version || null;
|
||||
|
||||
// Bounding boxes come back in the pixel space of the image the SIDECAR was
|
||||
// given — which is the preview (≤1920px long edge), not the original. Every
|
||||
// consumer compares them against photos.width/height, which are the
|
||||
// ORIGINAL dimensions: the strip's avatar crop works in ratios of them, and
|
||||
// the auto-category portrait rule divides face area by frame area.
|
||||
//
|
||||
// Left unscaled, a 6000px photo yields boxes ~3x too small (and areas ~9x
|
||||
// too small), so avatars crop to the wrong place and "Portraits" never
|
||||
// fires. It is invisible on any photo already under 1920px, which is why
|
||||
// the demo gallery looked correct.
|
||||
//
|
||||
// Scale here, once, so everything downstream can assume original-image
|
||||
// coordinates.
|
||||
let boxScale = 1;
|
||||
try {
|
||||
const previewMeta = await sharp(buffer).metadata();
|
||||
if (previewMeta?.width && photo.width) {
|
||||
boxScale = photo.width / previewMeta.width;
|
||||
}
|
||||
} catch (err) {
|
||||
logger.warn(`faceProcessor: could not read preview dimensions for photo ${photoId}`, {
|
||||
error: err.message,
|
||||
});
|
||||
}
|
||||
|
||||
const thresholds = await getThresholds();
|
||||
|
||||
try {
|
||||
await db.transaction(async (trx) => {
|
||||
// Replace rather than append: a re-scan of the same photo must not
|
||||
// double its faces. Detaching first keeps the FK to event_people clean.
|
||||
//
|
||||
// Remember which people the OLD faces belonged to. Deleting the rows does
|
||||
// not undo their contribution to event_people.face_count_total or to the
|
||||
// running-mean centroid, so without recomputing those afterwards a
|
||||
// re-scan inflates every count (typically doubling it) and leaves ghost
|
||||
// people behind when a face is no longer detected.
|
||||
const affectedPeople = await trx('photo_faces')
|
||||
.where({ photo_id: photoId })
|
||||
.whereNotNull('person_id')
|
||||
.distinct('person_id')
|
||||
.pluck('person_id');
|
||||
|
||||
await trx('photo_faces').where({ photo_id: photoId }).del();
|
||||
|
||||
const inserted = [];
|
||||
for (const face of faces) {
|
||||
const [bx, by, bw, bh] = (face.bbox || [0, 0, 0, 0]).map((v) => v * boxScale);
|
||||
const row = {
|
||||
photo_id: photoId,
|
||||
event_id: photo.event_id,
|
||||
bbox_x: bx, bbox_y: by, bbox_w: bw, bbox_h: bh,
|
||||
det_score: face.score ?? null,
|
||||
yaw: face.yaw ?? null,
|
||||
pitch: face.pitch ?? null,
|
||||
blur: face.blur ?? null,
|
||||
embedding: face.embedding ? packEmbedding(face.embedding) : null,
|
||||
model_version: modelVersion,
|
||||
// ISO string, not a Date — under Jest a Date handed to the sqlite3
|
||||
// binding stores as the literal "[object Object]" (see CLAUDE.md).
|
||||
created_at: new Date().toISOString(),
|
||||
};
|
||||
const [id] = await trx('photo_faces').insert(row).returning('id');
|
||||
inserted.push({ ...row, id: typeof id === 'object' ? id.id : id });
|
||||
}
|
||||
|
||||
// Rebuild the people the removed faces belonged to BEFORE assigning the
|
||||
// replacements, so assignment compares against honest centroids.
|
||||
for (const personId of affectedPeople) {
|
||||
await recomputeCentroid(personId, trx);
|
||||
}
|
||||
|
||||
if (inserted.length) {
|
||||
await assignFaces(photo.event_id, inserted, { thresholds, trx });
|
||||
}
|
||||
|
||||
// Commit the photo only if it is STILL the row we claimed. An admin who
|
||||
// purges or archives the event while this worker was waiting on the
|
||||
// sidecar has already cleared face_status; without this guard the worker
|
||||
// would write its rows back moments after "all face data deleted"
|
||||
// reported success, so an erasure request would silently not stick.
|
||||
const committed = await trx('photos')
|
||||
.where({ id: photoId, face_status: 'processing' })
|
||||
.update({
|
||||
face_status: 'done',
|
||||
face_count: faces.length,
|
||||
face_started_at: null,
|
||||
face_error: null,
|
||||
});
|
||||
|
||||
if (committed === 0) {
|
||||
logger.info(
|
||||
`faceProcessor: photo ${photoId} was purged or re-queued while scanning — discarding results`
|
||||
);
|
||||
// Undo this transaction's inserts rather than leaving orphans behind.
|
||||
throw new StaleScanError(photoId);
|
||||
}
|
||||
});
|
||||
} catch (err) {
|
||||
// Losing a race with a purge is an expected outcome, not a photo that
|
||||
// failed to scan. The transaction has already rolled back, so nothing was
|
||||
// written; leave face_status exactly as the purge left it.
|
||||
if (err instanceof StaleScanError) return { status: 'skipped' };
|
||||
throw err;
|
||||
}
|
||||
|
||||
// Auto-categories (#1074 phase 3) run as a distinct step AFTER detection,
|
||||
// outside the transaction: they are a convenience, and a rule-engine
|
||||
// failure must never roll back the faces we just paid the sidecar for.
|
||||
// No-ops unless separately enabled.
|
||||
try {
|
||||
const { categorizeEvent } = require('./faceAutoCategories');
|
||||
await categorizeEvent(photo.event_id);
|
||||
} catch (err) {
|
||||
logger.warn(`faceProcessor: auto-categorisation failed for event ${photo.event_id}`, {
|
||||
error: err.message,
|
||||
});
|
||||
}
|
||||
|
||||
return { status: 'done', faceCount: faces.length };
|
||||
}
|
||||
|
||||
/**
|
||||
* Queue every eligible photo in an event for (re-)scanning.
|
||||
*
|
||||
* `onlyUnscanned` is the backfill case — enabling the toggle on a gallery
|
||||
* that already has photos. Without it, this is a full re-scan.
|
||||
*/
|
||||
async function enqueueEvent(eventId, { onlyUnscanned = false } = {}) {
|
||||
const query = db('photos')
|
||||
.where({ event_id: eventId })
|
||||
// Photos still being processed have no preview and no dimensions yet;
|
||||
// photoProcessor enqueues them itself on completion.
|
||||
.where(function () {
|
||||
this.where('processing_status', 'complete').orWhereNull('processing_status');
|
||||
});
|
||||
|
||||
if (onlyUnscanned) {
|
||||
query.where(function () {
|
||||
this.whereNull('face_status').orWhereIn('face_status', ['failed', 'skipped']);
|
||||
});
|
||||
}
|
||||
|
||||
const updated = await query.update({
|
||||
face_status: 'pending',
|
||||
face_started_at: null,
|
||||
face_error: null,
|
||||
});
|
||||
|
||||
logger.info(`faceProcessor: queued ${updated} photo(s) for face detection in event ${eventId}`);
|
||||
return updated;
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove every face row for an event. Used by "delete all face data", by the
|
||||
* per-event toggle going off, and by archival.
|
||||
*/
|
||||
async function purgeEvent(eventId) {
|
||||
return db.transaction(async (trx) => {
|
||||
// Detach before deleting people so the SET NULL FK never fires against
|
||||
// rows that are about to go anyway.
|
||||
await trx('photo_faces').where({ event_id: eventId }).update({ person_id: null });
|
||||
const faces = await trx('photo_faces').where({ event_id: eventId }).del();
|
||||
const people = await trx('event_people').where({ event_id: eventId }).del();
|
||||
await trx('photos').where({ event_id: eventId }).update({
|
||||
face_status: null, face_count: null, face_started_at: null, face_error: null,
|
||||
});
|
||||
logger.info(`faceProcessor: purged ${faces} face(s) and ${people} person(s) from event ${eventId}`);
|
||||
return { faces, people };
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove the face rows belonging to one photo, and rebuild the people that
|
||||
* lose members as a result.
|
||||
*
|
||||
* Called from every photo-deletion path. The schema declares ON DELETE
|
||||
* CASCADE, but SQLite only enforces foreign keys when `PRAGMA foreign_keys`
|
||||
* is on and PicPeak does not enable it globally — so on SQLite the cascade
|
||||
* never fires and biometric embeddings would outlive the photo. Even where it
|
||||
* does fire (Postgres), the cascade cannot fix up event_people counts or
|
||||
* centroids, which is the other half of this.
|
||||
*
|
||||
* Safe to call for photos that were never scanned: it simply deletes nothing.
|
||||
*/
|
||||
async function purgePhotoFaces(photoId, trx = db) {
|
||||
const affected = await trx('photo_faces')
|
||||
.where({ photo_id: photoId })
|
||||
.whereNotNull('person_id')
|
||||
.distinct('person_id')
|
||||
.pluck('person_id');
|
||||
|
||||
// Drop any in-flight claim as well. A worker holding this photo would
|
||||
// otherwise still satisfy its `face_status = 'processing'` commit guard and
|
||||
// write fresh faces straight after the purge — and with FK enforcement off
|
||||
// on SQLite, the subsequent photo delete cannot cascade them away, leaving
|
||||
// orphaned biometric rows.
|
||||
await trx('photos').where({ id: photoId }).update({
|
||||
face_status: null, face_count: null, face_started_at: null, face_error: null,
|
||||
});
|
||||
|
||||
const removed = await trx('photo_faces').where({ photo_id: photoId }).del();
|
||||
if (!removed) return { removed: 0 };
|
||||
|
||||
const { recomputeCentroid: recompute } = require('./faceClustering');
|
||||
for (const personId of affected) {
|
||||
// Deletes the person outright when it has no members left.
|
||||
await recompute(personId, trx);
|
||||
}
|
||||
return { removed, peopleTouched: affected.length };
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
processPhotoFaces, enqueueEvent, purgeEvent, purgePhotoFaces, StaleScanError,
|
||||
};
|
||||
@@ -0,0 +1,210 @@
|
||||
/**
|
||||
* Background face-detection worker pool (#1074).
|
||||
*
|
||||
* Deliberately a near-copy of backgroundProcessor.js rather than a shared
|
||||
* abstraction: the two differ in exactly one behaviour (below), and inventing
|
||||
* a generic queue framework to save forty lines would make both harder to
|
||||
* read. Claim semantics, the janitor and the tunable shape are identical, so
|
||||
* anyone who understands one understands the other.
|
||||
*
|
||||
* THE ONE DIFFERENCE — a sidecar that is unreachable puts the photo back to
|
||||
* 'pending' with backoff, never 'failed'. Turning the ML container off for a
|
||||
* week must not require a manual re-scan afterwards. Only a 4xx (a genuinely
|
||||
* unprocessable image) marks a row failed.
|
||||
*
|
||||
* Nothing here starts unless the `faces` feature flag is on. That matters
|
||||
* more than usual because FACE_ML_URL now has a working default, so "is the
|
||||
* variable set" is no longer a signal — without the flag check, every install
|
||||
* would poll a hostname that does not resolve.
|
||||
*
|
||||
* Tunables (env, all optional):
|
||||
* FACE_PROCESSOR_CONCURRENCY default 1. Face scanning shares a host
|
||||
* with Sharp; 1 is the safe default on
|
||||
* the 2GB VPS class this project supports.
|
||||
* FACE_PROCESSOR_POLL_MS default 2000
|
||||
* FACE_PROCESSOR_STUCK_TIMEOUT_MS default 600000 (10 minutes)
|
||||
* FACE_PROCESSOR_DISABLED default false ('true' to opt out, e.g. CI)
|
||||
*/
|
||||
|
||||
const { db } = require('../database/db');
|
||||
const logger = require('../utils/logger');
|
||||
const { processPhotoFaces } = require('./faceProcessor');
|
||||
const { SidecarUnavailableError } = require('./faceClient');
|
||||
const { isFeatureEnabled } = require('./faceSettings');
|
||||
|
||||
const POLL_INTERVAL_MS = parseInt(process.env.FACE_PROCESSOR_POLL_MS || '2000', 10);
|
||||
const CONCURRENCY = Math.max(1, parseInt(process.env.FACE_PROCESSOR_CONCURRENCY || '1', 10));
|
||||
const STUCK_TIMEOUT_MS = parseInt(process.env.FACE_PROCESSOR_STUCK_TIMEOUT_MS || '600000', 10);
|
||||
const JANITOR_INTERVAL_MS = 60 * 1000;
|
||||
|
||||
// Backoff after the sidecar goes away. Without it a down sidecar turns into a
|
||||
// hot loop: claim, fail, release, claim again, thousands of times a minute.
|
||||
const UNAVAILABLE_BACKOFF_MS = parseInt(process.env.FACE_PROCESSOR_BACKOFF_MS || '30000', 10);
|
||||
|
||||
let running = false;
|
||||
let workerHandles = [];
|
||||
let janitorHandle = null;
|
||||
|
||||
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
||||
|
||||
function isPostgres() {
|
||||
const c = db.client.config.client;
|
||||
return c === 'pg' || (typeof c === 'string' && c.includes('postgres'));
|
||||
}
|
||||
|
||||
/**
|
||||
* Atomically claim the oldest pending photo. Returns the row or null.
|
||||
* Same two-path approach as backgroundProcessor: SKIP LOCKED on Postgres so
|
||||
* multiple pods race cleanly, a status-guarded UPDATE on SQLite.
|
||||
*/
|
||||
async function claimNextPhoto() {
|
||||
if (isPostgres()) {
|
||||
return db.transaction(async (trx) => {
|
||||
const row = await trx('photos')
|
||||
.where('face_status', 'pending')
|
||||
.orderBy('id', 'asc')
|
||||
.forUpdate()
|
||||
.skipLocked()
|
||||
.first();
|
||||
if (!row) return null;
|
||||
await trx('photos').where('id', row.id).update({
|
||||
face_status: 'processing',
|
||||
face_started_at: new Date().toISOString(),
|
||||
});
|
||||
return row;
|
||||
});
|
||||
}
|
||||
|
||||
return db.transaction(async (trx) => {
|
||||
const row = await trx('photos')
|
||||
.where('face_status', 'pending')
|
||||
.orderBy('id', 'asc')
|
||||
.first();
|
||||
if (!row) return null;
|
||||
const updated = await trx('photos')
|
||||
.where({ id: row.id, face_status: 'pending' })
|
||||
.update({
|
||||
face_status: 'processing',
|
||||
face_started_at: new Date().toISOString(),
|
||||
});
|
||||
return updated > 0 ? row : null;
|
||||
});
|
||||
}
|
||||
|
||||
async function releaseToPending(photoId) {
|
||||
// Guarded on 'processing' — the state THIS worker put the row in. If the
|
||||
// event was purged while the sidecar request was in flight, purgeEvent has
|
||||
// already set face_status to NULL, and an unconditional update would put it
|
||||
// back to 'pending' and rescan it: biometric rows reappearing after the
|
||||
// purge endpoint reported success. Same reasoning as the commit guard in
|
||||
// faceProcessor; the retry path needed it too.
|
||||
await db('photos')
|
||||
.where({ id: photoId, face_status: 'processing' })
|
||||
.update({ face_status: 'pending', face_started_at: null });
|
||||
}
|
||||
|
||||
async function workerLoop(workerIdx) {
|
||||
while (running) {
|
||||
// Re-checked every tick, not once at startup: an admin turning the flag
|
||||
// off must stop the workers without a restart.
|
||||
if (!(await isFeatureEnabled())) {
|
||||
await sleep(POLL_INTERVAL_MS * 5);
|
||||
continue;
|
||||
}
|
||||
|
||||
let claimed;
|
||||
try {
|
||||
claimed = await claimNextPhoto();
|
||||
} catch (e) {
|
||||
logger.warn(`faceQueue[${workerIdx}]: claim error`, { error: e.message });
|
||||
await sleep(POLL_INTERVAL_MS);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!claimed) {
|
||||
await sleep(POLL_INTERVAL_MS);
|
||||
continue;
|
||||
}
|
||||
|
||||
try {
|
||||
await processPhotoFaces(claimed.id);
|
||||
} catch (err) {
|
||||
if (err instanceof SidecarUnavailableError) {
|
||||
// Retry, don't fail. faceClient already rate-limits the log line.
|
||||
await releaseToPending(claimed.id).catch(() => {});
|
||||
await sleep(UNAVAILABLE_BACKOFF_MS);
|
||||
continue;
|
||||
}
|
||||
|
||||
logger.error(`faceQueue[${workerIdx}]: photo ${claimed.id} failed`, {
|
||||
error: err.message,
|
||||
stack: err.stack,
|
||||
});
|
||||
try {
|
||||
await db('photos').where({ id: claimed.id }).update({
|
||||
face_status: 'failed',
|
||||
face_started_at: null,
|
||||
face_error: String(err.message || err).slice(0, 1000),
|
||||
});
|
||||
} catch (updateErr) {
|
||||
logger.error(`faceQueue[${workerIdx}]: failed to mark photo ${claimed.id} as failed`, {
|
||||
error: updateErr.message,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function janitorLoop() {
|
||||
while (running) {
|
||||
try {
|
||||
const cutoff = new Date(Date.now() - STUCK_TIMEOUT_MS).toISOString();
|
||||
const reset = await db('photos')
|
||||
.where('face_status', 'processing')
|
||||
.where('face_started_at', '<', cutoff)
|
||||
.update({ face_status: 'pending', face_started_at: null });
|
||||
if (reset > 0) {
|
||||
logger.warn(`faceQueue: janitor reset ${reset} stuck photo(s) from 'processing' to 'pending'`);
|
||||
}
|
||||
} catch (e) {
|
||||
logger.warn('faceQueue: janitor error', { error: e.message });
|
||||
}
|
||||
await sleep(JANITOR_INTERVAL_MS);
|
||||
}
|
||||
}
|
||||
|
||||
function start() {
|
||||
if (running) return;
|
||||
if (process.env.FACE_PROCESSOR_DISABLED === 'true') {
|
||||
logger.info('faceQueue: disabled via FACE_PROCESSOR_DISABLED');
|
||||
return;
|
||||
}
|
||||
|
||||
running = true;
|
||||
workerHandles = [];
|
||||
for (let i = 0; i < CONCURRENCY; i++) {
|
||||
workerHandles.push(
|
||||
workerLoop(i).catch((e) =>
|
||||
logger.error(`faceQueue[${i}]: crashed`, { error: e.message, stack: e.stack })
|
||||
)
|
||||
);
|
||||
}
|
||||
janitorHandle = janitorLoop().catch((e) =>
|
||||
logger.error('faceQueue: janitor crashed', { error: e.message, stack: e.stack })
|
||||
);
|
||||
|
||||
logger.info(
|
||||
`faceQueue: started ${CONCURRENCY} worker(s), poll=${POLL_INTERVAL_MS}ms, stuck=${STUCK_TIMEOUT_MS}ms ` +
|
||||
'(idle until the `faces` feature flag is enabled)'
|
||||
);
|
||||
}
|
||||
|
||||
async function stop() {
|
||||
if (!running) return;
|
||||
running = false;
|
||||
await Promise.all([...workerHandles, janitorHandle].filter(Boolean));
|
||||
workerHandles = [];
|
||||
janitorHandle = null;
|
||||
}
|
||||
|
||||
module.exports = { start, stop, claimNextPhoto };
|
||||
@@ -0,0 +1,170 @@
|
||||
/**
|
||||
* Face-recognition settings and the feature gate (#1074).
|
||||
*
|
||||
* One module owns the answer to "is this feature on?", because the answer has
|
||||
* three independent parts and getting any of them wrong is either a privacy
|
||||
* problem or a log-spam problem:
|
||||
*
|
||||
* 1. The `faces` feature flag (global, default OFF).
|
||||
* 2. `FACE_ML_URL` — which now has a DEFAULT (the compose service name),
|
||||
* so its presence proves nothing. This is exactly why the flag is the
|
||||
* gate: if the URL were the gate, every install would try to reach a
|
||||
* hostname that doesn't resolve.
|
||||
* 3. The per-event `face_recognition_enabled` toggle.
|
||||
*
|
||||
* Nothing may contact the sidecar unless (1) is true. `isFeatureEnabled()` is
|
||||
* the only correct way to ask.
|
||||
*/
|
||||
|
||||
const { db } = require('../database/db');
|
||||
const {
|
||||
isFeatureEnabled: isFlagEnabled,
|
||||
invalidateFeatureFlagCache,
|
||||
} = require('../middleware/requireFeatureFlag');
|
||||
const logger = require('../utils/logger');
|
||||
|
||||
// Canonical flag key. Must match the entry in adminFeatureFlags KNOWN_FLAGS
|
||||
// and the FeatureKey union on the frontend.
|
||||
const FLAG_KEY = 'faces';
|
||||
|
||||
/**
|
||||
* Is this the all-in-one single-container image (#1042 / PR #1068)?
|
||||
*
|
||||
* Face recognition is BLOCKED there, on performance grounds. The AIO image
|
||||
* runs the backend, the frontend, SQLite and every background worker inside
|
||||
* one container sized for "one photographer plus guests browsing" — it has no
|
||||
* Redis and SQLite gives it a single writer. Face detection would add a
|
||||
* second image-processing pipeline competing with Sharp for the same CPU and
|
||||
* RAM, on top of an ML sidecar the image does not contain and cannot start.
|
||||
*
|
||||
* Enabling it there would not fail loudly; it would just make the whole
|
||||
* install slow and appear broken, which is the worst shape for a deployment
|
||||
* aimed at people who want one container and no decisions.
|
||||
*
|
||||
* Detected from an explicit marker rather than inferred from SERVE_FRONTEND
|
||||
* or a SQLite path: plenty of legitimate multi-container setups serve the
|
||||
* frontend from the backend or run SQLite, and none of them should lose the
|
||||
* feature by accident.
|
||||
*/
|
||||
function isSingleContainerImage() {
|
||||
const v = String(process.env.PICPEAK_SINGLE_CONTAINER || '').toLowerCase();
|
||||
return v === 'true' || v === '1' || v === 'yes';
|
||||
}
|
||||
|
||||
const DEFAULTS = {
|
||||
// Measured on a real gallery, not just on LFW pairs — see migration 177
|
||||
// for why those two disagree and why cluster purity wins.
|
||||
face_match_threshold: 0.60,
|
||||
face_min_cluster_size: 3,
|
||||
face_quality_min_score: 0.7,
|
||||
face_quality_min_px: 40,
|
||||
face_auto_categorize_enabled: false,
|
||||
};
|
||||
|
||||
const SETTING_KEYS = Object.keys(DEFAULTS);
|
||||
|
||||
function parseSetting(raw, fallback) {
|
||||
if (raw === undefined || raw === null) return fallback;
|
||||
try {
|
||||
return JSON.parse(raw);
|
||||
} catch {
|
||||
// Pre-JSON rows (and plain strings written by hand) — coerce by the
|
||||
// shape of the default so callers always get the type they expect.
|
||||
if (typeof fallback === 'number') {
|
||||
const n = Number(raw);
|
||||
return Number.isFinite(n) ? n : fallback;
|
||||
}
|
||||
if (typeof fallback === 'boolean') return raw === 'true' || raw === '1';
|
||||
return raw;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Is the `faces` feature flag on? This is THE gate — see the header.
|
||||
*
|
||||
* Delegates to the shared `feature_flags` reader (already cached, already
|
||||
* invalidated by PUT /admin/feature-flags) rather than keeping a second
|
||||
* cache that could disagree with the middleware guarding the same routes.
|
||||
*/
|
||||
async function isFeatureEnabled() {
|
||||
// Hard block on the all-in-one image, ahead of the flag. Even if the row
|
||||
// says true — restored from a backup taken on a full deployment, say — the
|
||||
// feature stays off here.
|
||||
if (isSingleContainerImage()) return false;
|
||||
|
||||
try {
|
||||
return await isFlagEnabled(FLAG_KEY);
|
||||
} catch (e) {
|
||||
// A missing table (pre-migration install) means off, not a crash.
|
||||
logger.debug?.('faceSettings: feature flag read failed', { error: e.message });
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Effective sidecar URL. Callers must check isFeatureEnabled() first — this
|
||||
* always returns a string, because the default is a hostname rather than an
|
||||
* absence.
|
||||
*/
|
||||
function getSidecarUrl() {
|
||||
return (process.env.FACE_ML_URL || 'http://picpeak-ml:8000').replace(/\/+$/, '');
|
||||
}
|
||||
|
||||
function getSidecarToken() {
|
||||
return process.env.FACE_ML_TOKEN || '';
|
||||
}
|
||||
|
||||
/**
|
||||
* Clustering/quality thresholds, with defaults. Deliberately settings rather
|
||||
* than constants: no single threshold survives contact with every library.
|
||||
*/
|
||||
async function getThresholds() {
|
||||
const out = { ...DEFAULTS };
|
||||
try {
|
||||
const rows = await db('app_settings')
|
||||
.whereIn('setting_key', SETTING_KEYS)
|
||||
.select('setting_key', 'setting_value');
|
||||
for (const row of rows) {
|
||||
out[row.setting_key] = parseSetting(row.setting_value, DEFAULTS[row.setting_key]);
|
||||
}
|
||||
} catch (e) {
|
||||
logger.debug?.('faceSettings: threshold read failed, using defaults', { error: e.message });
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Is detection enabled for this specific event? Requires BOTH the global flag
|
||||
* and the per-event toggle — the "two deliberate actions" rule from #1074 §6.
|
||||
*/
|
||||
async function isEnabledForEvent(event) {
|
||||
if (!(await isFeatureEnabled())) return false;
|
||||
if (!event) return false;
|
||||
return event.face_recognition_enabled === true || event.face_recognition_enabled === 1;
|
||||
}
|
||||
|
||||
/**
|
||||
* Do GUESTS of this event see the people strip? Separate from the above so a
|
||||
* photographer can use clustering as a private tool.
|
||||
*
|
||||
* Defaults to true when detection is on, because the column is nullable and
|
||||
* NULL means "not explicitly set" rather than "off" — matching the tri-state
|
||||
* convention used by show_watermark / show_qr.
|
||||
*/
|
||||
function areFacesVisibleToGuests(event) {
|
||||
if (!event) return false;
|
||||
return event.faces_visible_to_guests !== false && event.faces_visible_to_guests !== 0;
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
DEFAULTS,
|
||||
FLAG_KEY,
|
||||
isSingleContainerImage,
|
||||
isFeatureEnabled,
|
||||
invalidateFeatureFlagCache,
|
||||
getSidecarUrl,
|
||||
getSidecarToken,
|
||||
getThresholds,
|
||||
isEnabledForEvent,
|
||||
areFacesVisibleToGuests,
|
||||
};
|
||||
@@ -283,6 +283,25 @@ async function processUploadedPhotos(files, eventId, uploadedBy = 'admin', categ
|
||||
});
|
||||
} catch (e) { /* non-fatal */ }
|
||||
|
||||
// Face detection (#1074). processPhoto() — the ASYNC path — enqueues on
|
||||
// completion, but this synchronous path (chunked-upload completion,
|
||||
// watch-folder import) writes a finished photo row directly and never
|
||||
// reaches it, so those photos stayed unscanned in a face-enabled event
|
||||
// until someone ran a manual re-scan.
|
||||
try {
|
||||
const { isEnabledForEvent } = require('./faceSettings');
|
||||
if (!isVideo && await isEnabledForEvent(event)) {
|
||||
// `db`, NOT `trx`: the transaction is committed above, so a query
|
||||
// through it throws "Transaction query already complete" — which the
|
||||
// catch below swallowed, making this whole enqueue a silent no-op.
|
||||
await db('photos').where({ id: photoId }).update({ face_status: 'pending' });
|
||||
}
|
||||
} catch (err) {
|
||||
logger.warn(`processUploadedPhotos: face enqueue failed for photo ${photoId}`, {
|
||||
error: err.message,
|
||||
});
|
||||
}
|
||||
|
||||
uploadedPhotos.push({
|
||||
id: photoId,
|
||||
filename: newFilename,
|
||||
@@ -541,6 +560,23 @@ async function processPhoto(photoId) {
|
||||
// Mark complete
|
||||
updateData.processing_status = 'complete';
|
||||
updateData.processing_error = null;
|
||||
|
||||
// Face detection (#1074): this is the only correct place to enqueue.
|
||||
// Earlier and there is no preview rendition to scan; later and there is no
|
||||
// hook at all. Same UPDATE rather than a follow-up write, so a crash
|
||||
// between the two can't leave a complete photo permanently unqueued.
|
||||
// Guarded on BOTH the global flag and the per-event toggle, so installs
|
||||
// without the feature never write a face_status at all.
|
||||
try {
|
||||
const { isEnabledForEvent } = require('./faceSettings');
|
||||
if (!isVideo && await isEnabledForEvent(event)) {
|
||||
updateData.face_status = 'pending';
|
||||
}
|
||||
} catch (err) {
|
||||
// Never let the face feature block a photo from completing.
|
||||
logger.warn(`processPhoto: face enqueue check failed for ${photoId}`, { error: err.message });
|
||||
}
|
||||
|
||||
await db('photos').where({ id: photoId }).update(updateData);
|
||||
|
||||
// Side effects (best-effort, never fail the photo if these break)
|
||||
|
||||
@@ -122,6 +122,26 @@ async function replacePhoto(existingPhoto, newFileTempPath, { originalFilename,
|
||||
media_type: mimeType?.startsWith('video/') ? 'video' : 'image',
|
||||
};
|
||||
|
||||
// Face data (#1074): the row keeps its id but now points at a DIFFERENT
|
||||
// image, so every stored face describes the old picture. Left alone the
|
||||
// gallery would keep showing the previous subject's identities on the new
|
||||
// photo. Drop them, then re-queue if the event has detection on.
|
||||
try {
|
||||
const { purgePhotoFaces } = require('./faceProcessor');
|
||||
const { isEnabledForEvent } = require('./faceSettings');
|
||||
await purgePhotoFaces(existingPhoto.id);
|
||||
|
||||
const event = await db('events').where({ id: existingPhoto.event_id }).first();
|
||||
updates.face_status = (await isEnabledForEvent(event)) ? 'pending' : null;
|
||||
updates.face_count = null;
|
||||
updates.face_started_at = null;
|
||||
updates.face_error = null;
|
||||
} catch (err) {
|
||||
logger.warn(`replacePhoto: face reset failed for photo ${existingPhoto.id}`, {
|
||||
error: err.message,
|
||||
});
|
||||
}
|
||||
|
||||
await db('photos').where({ id: existingPhoto.id }).update(updates);
|
||||
|
||||
const updatedPhoto = await db('photos').where({ id: existingPhoto.id }).first();
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
*/
|
||||
|
||||
const path = require('path');
|
||||
const logger = require('../utils/logger');
|
||||
const fs = require('fs').promises;
|
||||
const { db } = require('../database/db');
|
||||
const { formatBoolean } = require('../utils/dbCompat');
|
||||
@@ -120,6 +121,15 @@ const deletePhoto = async (photoId, options = {}) => {
|
||||
}
|
||||
}
|
||||
|
||||
// Face data (#1074) — see purgePhotoFaces on why the FK cascade is not
|
||||
// relied on. Must run BEFORE the photo row goes.
|
||||
try {
|
||||
const { purgePhotoFaces } = require('./faceProcessor');
|
||||
await purgePhotoFaces(photoId);
|
||||
} catch (err) {
|
||||
logger.warn(`deletePhoto: face purge failed for photo ${photoId}`, { error: err.message });
|
||||
}
|
||||
|
||||
// Delete from database
|
||||
await db('photos').where('id', photoId).delete();
|
||||
}
|
||||
|
||||
@@ -29,7 +29,20 @@ const packageJson = require('../../package.json');
|
||||
const PICPEAK_FORMAT_VERSION = 1;
|
||||
|
||||
// Never exported as data — the target owns these (its own migrations set them).
|
||||
const EXCLUDED_TABLES = new Set(['knex_migrations', 'knex_migrations_lock']);
|
||||
//
|
||||
// photo_faces / event_people (#1074) are excluded for a different reason:
|
||||
// face embeddings are biometric data (GDPR Art. 9), and this export gets
|
||||
// handed to clients and moved between operators. The data is fully derived
|
||||
// from the photos, so the target re-scans rather than receiving biometrics it
|
||||
// has no lawful basis for. NOTE the cost of that decision: any names the
|
||||
// photographer assigned to people are lost too, since they live in
|
||||
// event_people. That is accepted — see ml/README.md and the #1074 thread.
|
||||
const EXCLUDED_TABLES = new Set([
|
||||
'knex_migrations',
|
||||
'knex_migrations_lock',
|
||||
'photo_faces',
|
||||
'event_people',
|
||||
]);
|
||||
|
||||
// Storage subdirs holding non-recalculable blobs — always included.
|
||||
const DOC_DIRS = ['business-docs', 'uploads'];
|
||||
@@ -90,7 +103,19 @@ async function writeTableNdjson(table, dataDir) {
|
||||
const hash = crypto.createHash('sha256');
|
||||
const rows = await db(table).select('*');
|
||||
const lines = rows.map((row) => {
|
||||
const line = JSON.stringify(row);
|
||||
// photo_faces / event_people are excluded from the export (#1074), so a
|
||||
// restored install has no face data — but `photos.face_status = 'done'`
|
||||
// would come across intact and the worker only ever claims 'pending'.
|
||||
// The gallery would then report itself fully scanned while showing no
|
||||
// people at all, permanently, with no way to tell why.
|
||||
//
|
||||
// Reset the derived state so the target simply re-scans once the operator
|
||||
// enables the feature there.
|
||||
const line = JSON.stringify(
|
||||
table === 'photos' && (row.face_status !== null && row.face_status !== undefined)
|
||||
? { ...row, face_status: null, face_count: null, face_started_at: null, face_error: null }
|
||||
: row
|
||||
);
|
||||
hash.update(`${line}\n`);
|
||||
return line;
|
||||
});
|
||||
|
||||
@@ -351,6 +351,21 @@ async function replaceAllTables(tables, dataDir, currentAdmin, roleSnapshot, { c
|
||||
for (const table of tables) {
|
||||
await trx(table).del();
|
||||
}
|
||||
|
||||
// Face data (#1074) is excluded from the archive, which also excludes it
|
||||
// from `tables` — so the LOCAL rows would survive a whole-DB replace.
|
||||
// FK enforcement is deliberately suspended during import, so those
|
||||
// orphans can end up attached to reused photo/event ids from the incoming
|
||||
// archive: one instance's biometric data silently adopted by another's
|
||||
// galleries. Purge them explicitly.
|
||||
for (const faceTable of ['photo_faces', 'event_people']) {
|
||||
try {
|
||||
await trx(faceTable).del();
|
||||
} catch (err) {
|
||||
// Absent on targets that predate migration 177 — nothing to purge.
|
||||
}
|
||||
}
|
||||
|
||||
for (const table of tables) {
|
||||
const rows = parseNdjson(path.join(dataDir, `${table}.ndjson`));
|
||||
if (!rows.length) continue;
|
||||
@@ -493,7 +508,30 @@ async function importFromPicpeak({ picpeakPath, currentAdminId }) {
|
||||
await resyncSequences(tables);
|
||||
await setSessionsValidAfter(Math.floor(Date.now() / 1000));
|
||||
|
||||
|
||||
const filesRestored = await restoreFiles(staging);
|
||||
|
||||
// Face data (#1074): queue ONLY once the files are on disk. The archive
|
||||
// carries no face rows and the export blanked photos.face_status, but the
|
||||
// event toggles come across enabled, so the "enable" transition that
|
||||
// normally triggers a backfill never happens here.
|
||||
//
|
||||
// Ordering matters: the worker is live during a restore. Queued before
|
||||
// restoreFiles, it races the copy and either scans the PREVIOUS
|
||||
// instance's files or marks photos failed for originals that are not
|
||||
// there yet — and nothing re-queues them afterwards.
|
||||
try {
|
||||
const requeued = await db('photos')
|
||||
.whereIn('event_id', db('events').select('id').where('face_recognition_enabled', true))
|
||||
.update({
|
||||
face_status: 'pending', face_count: null, face_started_at: null, face_error: null,
|
||||
});
|
||||
if (requeued > 0) {
|
||||
logger.info(`picpeakImport: queued ${requeued} photo(s) for face detection after import`);
|
||||
}
|
||||
} catch (err) {
|
||||
logger.debug?.(`picpeakImport: face requeue skipped: ${err.message}`);
|
||||
}
|
||||
const usesExternalMedia = await detectExternalMedia();
|
||||
|
||||
logger.info(
|
||||
|
||||
@@ -212,9 +212,15 @@ class RestoreService {
|
||||
switch (options.restoreType) {
|
||||
case 'full':
|
||||
restoreResult = await this.performFullRestore(localBackupPath, manifest, options);
|
||||
// After the FILES too — see requeueFaceScans on why the ordering
|
||||
// matters with a live worker.
|
||||
await this.requeueFaceScans();
|
||||
break;
|
||||
case 'database':
|
||||
restoreResult = await this.performDatabaseRestore(localBackupPath, manifest, options);
|
||||
// Database-only restore: the existing files stay in place, so there
|
||||
// is nothing to race.
|
||||
await this.requeueFaceScans();
|
||||
break;
|
||||
case 'files':
|
||||
restoreResult = await this.performFilesRestore(localBackupPath, manifest, options);
|
||||
@@ -836,6 +842,45 @@ class RestoreService {
|
||||
/**
|
||||
* Perform full restore (database + files)
|
||||
*/
|
||||
/**
|
||||
* Requeue face detection after a restore (#1074).
|
||||
*
|
||||
* Face data is deliberately excluded from backups — it is derived, and
|
||||
* biometric data should not travel in an archive. But photos.face_status
|
||||
* DOES restore, so without this the restored install claims every photo is
|
||||
* scanned while photo_faces is empty, and the worker never picks them up
|
||||
* because it only claims 'pending'. The gallery shows a finished scan and
|
||||
* no people, forever, with nothing to indicate why.
|
||||
*
|
||||
* MUST run after the FILES are restored, not merely after the database.
|
||||
* The face worker is live throughout a restore; queued earlier it races the
|
||||
* file copy and either scans the previous instance's originals or marks
|
||||
* photos failed for files that are not there yet — and nothing re-queues
|
||||
* them afterwards.
|
||||
*
|
||||
* Only touches rows that had been scanned; NULL stays NULL, so this never
|
||||
* switches the feature on for anyone.
|
||||
*/
|
||||
async requeueFaceScans() {
|
||||
try {
|
||||
const { db: restoredDb } = require('../database/db');
|
||||
const requeued = await restoredDb('photos')
|
||||
.whereNotNull('face_status')
|
||||
.update({
|
||||
face_status: 'pending',
|
||||
face_count: null,
|
||||
face_started_at: null,
|
||||
face_error: null,
|
||||
});
|
||||
if (requeued > 0) {
|
||||
this.log('info', `Requeued ${requeued} photo(s) for face detection after restore`);
|
||||
}
|
||||
} catch (err) {
|
||||
// Pre-migration-177 backups have no such column; not an error.
|
||||
this.log('info', `Face state reset skipped: ${err.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
async performFullRestore(backupPath, manifest, options) {
|
||||
const result = {
|
||||
databaseRestored: false,
|
||||
@@ -1166,6 +1211,7 @@ END $$;`
|
||||
await reinitPool();
|
||||
this.log('info', 'Knex pool re-initialized');
|
||||
|
||||
|
||||
// NOTE: we deliberately do NOT call `db.migrate.latest()` here.
|
||||
//
|
||||
// The picpeak migrations directory contains `helpers.js` (a
|
||||
|
||||
Reference in New Issue
Block a user