feat(faces): People in this gallery — face recognition via an optional ML sidecar (#1074) (#1075)

* feat(ml): optional face-detection sidecar, opt-in and inert by default (#1074)

First of four PRs for "People in this gallery". This one ships only the
sidecar, its wiring and its CI — no schema, no backend code, no UI. Nothing
in PicPeak calls it yet.

picpeak-ml is a single FastAPI + onnxruntime container: three endpoints
(/health, /info, /faces), no database, no volumes, no egress, no model
download at runtime. Clustering, person identity and every privacy decision
stay in the backend where the data already lives.

Models are YuNet (detection) + FaceNet-512 (embedding), both MIT, both
pinned by URL and SHA-256 and verified at build time. The licence analysis
is in ml/LICENSES.md: the more accurate InsightFace weights are
non-commercial-only and PicPeak's users are working photographers, so they
are never baked into an image we publish.

Two things worth review attention:

- Alignment uses a least-squares similarity transform (Umeyama), NOT
  cv2.estimateAffinePartial2D. RANSAC and LMEDS exist to reject outliers
  among many correspondences; given five landmarks and no outliers they fit
  a three-point subset exactly and let the rest drift. Measured on a real
  off-frontal portrait: eyes and nose pinned to 0.11px, mouth corners
  11.8px out on a 160px crop. Umeyama distributes it (max 6.5px, rms 5.1 vs
  7.4). The failure mode is silent — a bad warp still yields 512 confident
  floats — so tests/test_pipeline.py pins it numerically.

- FACENET_ONNX_URL has no default and the build fails loudly without it.
  deepface distributes FaceNet-512 as Keras .h5 only, so the ONNX is
  produced once by tools/convert_facenet.py and published as a release
  asset. Converting inside the build would drag TensorFlow through both
  architecture legs of every build to produce a byte-identical file. The CI
  jobs are gated on the FACENET_ONNX_URL repository variable and skip
  cleanly until it is set.

Off by default, twice over: the sidecar is behind the `faces` compose
profile, and the backend will gate on a `faces` feature flag that defaults
to false. FACE_ML_URL defaults to http://picpeak-ml:8000 so the standard
deployment needs no configuration — nothing dials that host while the flag
is off, which is why a non-resolving default is harmless.

Verified: 27 pytest tests green; YuNet loads and detects against a real
portrait with its landmark order matching the alignment template
index-for-index; both compose files validate and the faces profile is
correctly excluded from a default `up`; workflow YAML parses and the job
graph resolves.

Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc

* fix(ml): pin the converter toolchain, verify parity, drop a false reproducibility claim (#1074)

Ran the FaceNet-512 conversion for real and corrected what the previous
commit assumed about it.

The conversion works: 23,497,424 parameters, 89.6 MB ONNX, and the converted
graph matches the Keras original to 2.086e-06 absolute / cosine
1.0000000000. That check is now part of the script rather than something I
did once by hand — a subtly wrong graph still returns 512 plausible floats,
so it refuses to leave the file on disk if parity fails.

Also ran the full pipeline against both real models end to end. The
embedding is L2-normalized to 1.000000, and the same face survives being
re-rendered: half scale 0.973, double scale 0.984, JPEG q40 0.987, rotated
8 degrees 0.984, brightness +40 0.988. Scale invariance in particular is
evidence the alignment warp is doing its job.

Corrected claim: the conversion is NOT byte-reproducible. Two runs with the
same pinned versions on the same machine gave different SHA-256s. The graphs
are functionally identical — same 336 nodes, same 271 initializers, every
weight matching to 0.000e+00 — but a few initializer names differ because
tf2onnx's traced-op naming is not deterministic (Keras layer naming is
deterministic; I checked). The previous commit message and README both
claimed byte-identical output. They were wrong, and it matters: anyone
re-running the conversion gets a different hash, and without this note that
reads like tampering. The build-time SHA-256 pins one published artifact so
its URL cannot start serving different bytes; validating a fresh conversion
is the parity check's job.

requirements-convert.txt now pins the exact set that produced the artifact,
including transitive keras/protobuf/numpy, and documents that the converter
needs Python 3.11 while the image runs 3.12.

Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc

* feat(faces): schema, queue, clustering and API for People in this gallery (#1074)

Backend half of the feature. Migration 177, a face-detection queue, the
clustering engine, the gallery and admin APIs, and the privacy wiring. No UI
yet; nothing is reachable until the `faces` feature flag is on, which
defaults to false.

The flag is the gate, not FACE_ML_URL. That variable now has a working
default (the compose service name), so its presence proves nothing about
intent — if it were the gate, every install would poll a hostname that does
not resolve. faceQueue re-checks the flag every tick, so turning it off stops
the workers without a restart.

Visibility scoping is the part worth reviewing closely. Face rows have no
concept of photo visibility, but guests are restricted to
photos.visibility='visible'. A raw count leaks how many hidden photos someone
appears in, and an unscoped cover face renders a crop of a photo the guest
may not open — with the best-scoring face being the likeliest pick, so it
would happen often rather than rarely. facePeopleService recomputes both per
request against the caller's own scope, and event_people.face_count_total is
named to be conspicuous in a guest path. Six tests cover it, including the
case where a person's photos are ALL hidden and they must vanish entirely.

Face data is excluded from backups and .picpeak exports, per the decision in
the thread: it is derived, so a restore re-scans rather than carrying
biometrics between operators. Three separate mechanisms, because the engines
cannot be filtered alike — EXCLUDED_TABLES for export, --exclude-table-data
(not --exclude-table; the CREATE TABLE must survive or restore breaks on the
first query) for Postgres, and DELETE + VACUUM on the temp copy for SQLite,
which has no way to exclude a table from a whole-file .backup. The VACUUM is
not cosmetic: without it the pages stay in the file and the claim is false on
disk.

Archiving now purges face data explicitly. photo_faces cascades off photos,
but archive deletes neither the photo rows nor the event, so without this an
archived gallery kept its biometrics indefinitely.

Other decisions: clustering keeps names across a re-cluster by majority
inheritance (without it, one button click silently discards every name the
photographer typed); consolidation refuses to merge two people who were named
differently; assignment never compares across model_version, since embeddings
from two pipelines are not comparable; low-quality faces are stored but left
unassigned so they show in "this photo contains" without spawning junk people.

Migration is 177, not 174 — 174/175/176 landed on main while this branch was
open.

29 tests green: 7 migration (idempotency, down(), cascade, and that
installing it enqueues NOTHING), 11 clustering, 11 privacy/visibility. Lint
clean; the pre-existing error counts in databaseBackup.js and server.js are
unchanged.

Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc

* feat(faces): People strip, face filter and admin controls (#1074)

Frontend half. Renders nothing anywhere unless the `faces` feature flag is
on AND the photographer enabled detection for the gallery — the whole guest
surface hangs off one boolean, event.people_enabled, which the server
computes from the flag, the per-event toggle and the show-to-guests toggle
together.

Guest side: a People strip between the filter bar and the grid, circular
crops from each person's cover face, an active-filter chip row, and a "Show
all" bottom sheet. The face filter composes with category, search, media
type and the liked/saved/rated filters in the same useMemo rather than
replacing them, so "photos of Anna that I liked" works. Two people selected
means AND by default — that is what picking a second face almost always
asks for — with a toggle to OR that appears only once a second person is
picked.

Unnamed people show a photo count and never "Person 7". A number is honest
about what the system knows; an invented name is not. There is a test
asserting we don't do it.

The strip renders nothing below two people, collapses to one line when
dismissed (persisted per slug, so dismissing one gallery says nothing about
the next), and appears mid-backfill with a progress line rather than
blocking the gallery behind a spinner. Avatar crops are computed in ratios
of the source dimensions so they survive whatever rendition the browser
gets; without width/height they fall back to an uncropped thumbnail, since
a wrongly-offset crop is worse than no crop.

No new download endpoint: "download these N" rides the existing photoIds
path, which already enforces access level and per-category permissions
server-side. Adding a person_id selector would have been a second thing to
authorize for no gain.

Guest-facing copy never says "biometric" or "recognition" — those words
describe our implementation, not the guest's experience. The sheet's
footnote answers the first question every guest has (where does this go?)
inline. The admin card, by contrast, is explicit: it states the controller
obligation next to the toggle, and warns that scanning materializes the
preview tier on galleries that never generated one, which is real CPU and
disk an admin should know about before a 2,000-photo backfill.

EN + DE translations. 140 frontend tests green (8 new), tsc and eslint clean.

Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc

* fix(faces): measured match threshold, working build defaults, 89MB smaller image (#1074)

Ran the Phase 0 spike that had been outstanding, published the model, and
fixed what both turned up.

THRESHOLD IS NOW MEASURED, NOT GUESSED. LFW's standard 1000-pair protocol
run through this exact pipeline (YuNet -> Umeyama alignment -> FaceNet-512
ONNX), 100% detection on 2000 images:

  same person  cosine 0.6958 +/- 0.1415
  diff person  cosine 0.0849 +/- 0.1674   separation 0.6109
  peak accuracy 96.60% @ 0.405

So the pipeline separates people well — the thing I could not previously
claim, since every earlier number was the same face re-rendered.

Default moves 0.62 -> 0.50. The old value was a placeholder and a bad one:
it gave 0% false merges but 22.4% false splits, i.e. roughly one in four
same-person pairs failing to join, which fragments a gallery badly. 0.50
gives 1.0% false merge / 8.2% false split. Peak accuracy (0.405) is
deliberately NOT chosen: for clustering the two errors do not cost the same.
A false split is a duplicate row the photographer can merge away; a false
merge puts a stranger into someone's "download my photos" — and until the
Phase 2 merge/split UI ships, there is no way to undo one. So this sits on
the conservative side of the optimum.

The spike is committed as ml/tools/benchmark_threshold.py rather than
thrown away, so "why 0.50?" has an answer in six months and a re-tune is one
command.

BUILD DEFAULTS. FACENET_ONNX_URL/_SHA256 now default to the published
ml-models-v1 release asset, so `docker build ml/` and
`docker compose --profile faces up` work with no arguments. Blanking either
still fails loudly — a URL without a checksum is never acceptable, since the
checksum is what makes the URL safe to trust. Found by running compose for
real: it failed exactly as designed, which was correct behaviour and a bad
out-of-box experience now that a canonical artifact exists.

IMAGE SIZE. 389MB -> 300MB single-arch. `chown -R` after COPY rewrote every
copied file into a fresh layer, duplicating the 90MB model for nothing; the
user is now created before the copies and ownership set via COPY --chown.
Also drops pip/setuptools from the runtime image. Measured RSS is 186MiB
idle, and the container answers /faces end-to-end in well under the
80-150ms/photo the issue budgeted.

Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc

* fix(faces): threshold 0.50 -> 0.60 from real clustering, theme-aware People strip (#1074)

Both fixes come from running the feature on an actual gallery — 61 photos,
5 real identities — rather than reasoning about it.

THRESHOLD. The LFW pairwise sweep in the previous commit said 0.50, and it
was wrong. On a real gallery at 0.50, three of six visible clusters were
contaminated: two different people merged into one strip entry, which is the
exact failure that puts a stranger into someone's "download my photos".

Pairwise error rates do not predict cluster purity. Greedy assignment
compounds — one wrong face drags the centroid toward the midpoint between two
identities, making the next wrong face likelier. A 1% pairwise false-merge
rate is not a 1% chance of a clean gallery, and no amount of staring at an
ROC curve would have shown that.

Sweep against ground truth (5 identities):

    0.50 -> 6 clusters, 3 contaminated
    0.56 -> 6 clusters, 0 contaminated
    0.60 -> 5 clusters, 0 contaminated   <- exactly right
    0.64 -> 5 clusters, 0 contaminated, fewer faces assigned

0.60 recovers the right number of people with no contamination; higher only
loses coverage. Migration 177 carries the full reasoning so the next person
to touch this knows why the obvious pairwise answer is the wrong one.

THEME. The People strip hardcoded `text-neutral-800` for named people. On a
dark gallery — which the screenshot immediately showed — that renders a
named person's label almost invisibly, while UNNAMED people stayed legible.
Exactly backwards. Labels, headings, the collapsed summary, the scan line
and the filter chip row now read the gallery's own theme tokens
(--color-text / --color-muted-text / --color-accent / --color-surface-border)
like the rest of the gallery surface.

Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc

* fix(faces): keep the mobile filter row inside the viewport (#1074)

At 390px the photo count and Clear link were pushed against the right edge
by ml-auto and clipped. Only apply it from the sm breakpoint up, where
there is room; below that they flow after the chips.

Found by screenshotting the real thing on an iPhone-sized viewport.

Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc

* feat(faces): complete Phase 1, add People management and auto-categories (#1074)

Closes the two Phase 1 gaps, then builds Phase 2 and Phase 3.

PHASE 1 GAPS. "Download these N" was specified, described as done in an
earlier summary, and never actually built — I had verified the backend needed
no new endpoint and let that stand as if the button existed. It now hands the
filtered photo ids to the same path as a manual selection, so the server
re-applies access level and per-category permissions on the way through.
Photos in a downloads-disabled category are excluded client-side too, so the
number on the button is the number the guest receives. Hidden entirely when
downloads are off for the gallery.

Lightbox person chips ("In this photo: Anna") are the second way into the
face filter — a guest looking at a photo of themselves can act on it without
scrolling back to the strip. Tapping one closes the lightbox and filters the
grid behind it.

PHASE 2. A People management modal over the endpoints that already existed
and were already tested: rename inline, merge (multi-select, first pick is
the target so the name a photographer typed survives), split via a face
picker, hide, ignore. This matters more than it sounds — clustering
deliberately errs toward splitting because a wrong merge puts a stranger into
someone's download, and that trade only works if merging is easy.

PHASE 3. Rule engine over face_count plus face-area ratio: 0 -> Details,
1 large -> Portraits, 2-5 -> Small groups, >5 -> Groups. The area ratio is
what separates "a portrait of someone" from "someone is in this landscape".
Three guarantees, all tested: it only ever fills an EMPTY category (enforced
in the query AND re-checked in the UPDATE, so a photographer setting one
mid-run still wins), everything it touches is marked auto_categorized so undo
is exact, and it is a no-op unless separately enabled. Migration 178 adds the
column — separate from 177, which has already run wherever this branch is
deployed.

Verified on the real gallery: 61 photos -> 48 portraits + 13 small groups,
undo cleared exactly 61 and left the manual ones alone. Merge moved faces and
removed the source. Both confirmed against the database, not just the UI.

TWO BUGS THE BROWSER CAUGHT, both invisible to tsc:

- The lightbox destructure never landed — my patch targeted a line that has a
  default value, matched nothing, and failed silently. `people` resolved to
  something else entirely and the chips would never have rendered. eslint's
  "outer scope value" warning is what surfaced it.

- Admin face thumbnails 403'd because <AuthenticatedImage> attaches whatever
  gallery token is in session storage; an admin who has also opened one of
  their own galleries sends a type:"gallery" bearer to an admin route. Admin
  routes authenticate from the httpOnly cookie, which a plain same-origin
  <img> sends by itself. Worth noting AdminPhotoGrid has the same latent
  shape; not touched here.

Also: the admin card now reports "N people (M shown to guests)" when those
differ, so the settings page and the gallery stop disagreeing without
explanation.

45 backend tests (8 new) and 140 frontend tests green; tsc and eslint clean.
EN + DE for every new string.

Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc

* perf(faces): batch migration DDL and drop the face stack from server.js import (#1074)

CI's backend job timed out at 10 minutes on the first run of this branch.
Nothing failed — 132 of 182 suites passed and the wall clock ran out. Main
does the same 182 in 124s, and where main has 12 suites slow enough for jest
to print a duration, this branch had 77.

Two changes, both worth making regardless of how much of the gap they close:

- Migration 177 added its columns one ALTER TABLE at a time (four on photos,
  three on events, plus a separate index statement) and seeded settings with
  a SELECT and an INSERT per key. It now uses one alterTable per table and
  one SELECT plus one bulk INSERT. 178 folds its index into the same
  statement as its column. That chain replays in ~90 suites, so statement
  count there is multiplied by 90.

- server.js required faceQueue at module scope, which pulls in axios and —
  through imageProcessor — sharp. Every supertest suite that imports
  server.js was paying for a module graph it never uses. Now required inside
  the startup block, next to the call that needs it.

Honest about the evidence: locally the migration delta measures at zero
(1.15s vs 1.13s for the same suite, three runs each), so batching alone does
not explain an eight-minute regression. A fast local disk and many cores mask
per-statement and per-import costs that a two-core runner with a shared disk
does not. These are the two real costs this branch added to a path that runs
in almost every suite; whether they are sufficient is a question for CI, not
for another round of local speculation.

37 face tests still green after the change.

Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc

* i18n(faces): complete EN and DE coverage for the face feature (#1074)

The admin card and the Features toggle were rendering entirely from inline
English `defaultValue` fallbacks — 22 keys existed in no locale file at all,
so a German admin saw an English consent notice, English toggles and English
buttons. The gallery side was already translated; the admin side was not,
and nothing in the toolchain flags this because a `defaultValue` always
renders something.

Adds the missing `admin.faces.*` (19), `settings.features.faces.*` (2) and
shared `common.clear/saved/saveFailed` in both languages. Existing keys are
left alone (setdefault, not overwrite), so the shared `common` strings other
features rely on are untouched.

Committed the audit as frontend/scripts/i18n-faces-audit.py rather than
throwing it away: it extracts every t() key the face components actually use
and diffs it against each locale, and it also reports German values that are
byte-identical to English, which is the usual shape of an untranslated
copy-paste. Currently: 69 keys in use, EN complete, DE complete, no
identical pairs.

Verified in the browser, not just in the JSON — the German card reads
"61 / 61 Fotos durchsucht · 16 Personen (5 für Gäste sichtbar)" end to end.
Also checked the components for hardcoded user-facing text (JSX nodes,
title/aria-label/placeholder attributes) outside t(); there is none.

Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc

* fix(faces): 13 defects from external review — coordinates, counts, erasure, races (#1074)

Codex reviewed the branch against main. Thirteen findings, nine P1. I checked
every one against the code and could not dismiss a single one as a false
positive, so all thirteen are fixed here.

THE WORST ONE: bounding boxes were stored in the wrong coordinate system.
The sidecar reports coordinates in the space of the image it was HANDED —
which is the ≤1920px preview, not the original — while every consumer
compares them against photos.width/height, the original dimensions. A 6000px
photo therefore produced boxes ~3x too small and areas ~9x too small: avatar
crops landed in the wrong place and the Portraits rule could never fire. It
is invisible on any photo already under 1920px, which is exactly why the
demo gallery and every screenshot looked correct. Now scaled once in
faceProcessor so everything downstream can assume original-image coordinates.

ERASURE. The FK cascade on photo_faces is decorative on SQLite: PicPeak never
enables `PRAGMA foreign_keys`, so deleting a photo left its embeddings
behind. I first enabled the pragma globally and reverted it — six unrelated
suites immediately failed on pre-existing dangling references, and switching
it on would start rejecting inserts on every existing install. That is a real
change worth making, but it is its own PR, not a rider on this one. Instead
deletion purges explicitly: purgePhotoFaces in the photo paths (single, bulk,
service) and photo_faces/event_people in deleteEventCascade. Tests assert
this with the pragma explicitly OFF, so they can only pass if the code does
the work.

COUNTS. A re-scan deleted the old face rows without undoing their
contribution to event_people, so counts inflated on every re-scan and ghost
people survived. Now the affected people are recomputed before the
replacements are assigned. My own "must not double its faces" test only
checked photo_faces rows, which is why it passed throughout.

RACES. A worker that finished after an admin purged the event committed its
rows anyway — erasure reported success and the data reappeared. The commit is
now conditional on the row still being 'processing'. And assignFaces is
read-modify-write over an event's people, so two workers lost each other's
updates; it is now serialised per event with an in-process mutex plus a
Postgres advisory lock for the multi-pod case the queue advertises.

METADATA LOSS. Merging discarded the source's name and suppression flags, so
a merge could erase a typed name or un-hide someone. Reclustering remembered
only people with a label, so an unnamed-but-hidden bystander came back
guest-visible after one "Re-group people" — and suppression now propagates to
every descendant cluster, not just the majority one.

Also: export reset face_status so a restored gallery re-scans instead of
claiming to be scanned forever; manual category edits clear auto_categorized
so "undo automatic" cannot delete a photographer's own choice; external
photos are skipped rather than failed (resolvePhotoStorageKey returns null
for them by design); the gallery refetches photo memberships as a scan
progresses so filtering is not stale; a failed VACUUM now fails the backup
rather than publishing one that may retain biometric pages; and the ML
Dockerfile's `|| true` is scoped to the uninstall — as written it was
`(install && uninstall) || true`, so a failed dependency install produced a
green layer and an image with no onnxruntime.

Four new regression tests. Full backend suite failure set verified identical
to origin/main; frontend 140 green; tsc and eslint clean.

Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc

* fix(faces): 12 more defects from review round 2 — cross-event purge, leaks, lifecycle (#1074)

Second Codex round on the same diff, now including round 1's fixes. Twelve
findings, seven P1. Again none were false positives.

SECURITY, AND MINE FROM ROUND 1: the bulk-delete face purge iterated the raw
`photoIds` from the request instead of the event-scoped `photos` rows the
handler had already validated. purgePhotoFaces has no event scope of its own,
so an editor could pass another gallery's photo id and delete its face data —
even though the photo deletion right below it was correctly scoped. Fixing
one thing and introducing another is exactly why the second round was worth
running.

ANOTHER VISIBILITY LEAK, same class as the one round 1 fixed: /people returns
scan progress, and getScanStatus counted every photo with a face_status —
including hidden ones. Guests could read the hidden-photo count off the
progress bar while the people list and covers beside it were properly scoped.
Now scoped by the same predicate, with the caller passing its audience.

RECLUSTER, ROUND 1'S FIX WAS INCOMPLETE. I made suppression follow every
descendant but still copied the flags from the majority ANCESTOR. When
reclustering merges a visible named person with a hidden one, the majority
ancestor is often the visible one — republishing the hidden person's photos.
Suppression is now OR-ed across every ancestor contributing faces. The name
also now goes to the genuine largest descendant; the previous code took
whichever cluster came first in map order, which the comment already claimed
it did not.

LIFECYCLE. Face data is excluded from backups and exports, but photos.
face_status came across intact, so a restored install claimed every photo was
scanned while holding no faces — and the worker only claims 'pending', so it
stayed that way forever. Now: the SQLite backup requeues in the dump, restore
requeues after the pool reinit (the Postgres path cannot rewrite rows inside
pg_dump), the portable importer purges LOCAL face tables (they were excluded
from the replace list, so another instance's embeddings survived an import
with FK checks suspended) and requeues, and archiving disables detection so a
restored archive is honestly off rather than enabled-and-empty.

WRITE PATHS. Only processPhoto enqueued. The synchronous upload path
(chunked-upload completion, watch-folder) left photos unscanned, and
replacePhoto kept the OLD image's faces on a row now pointing at a different
picture — stale identities shown on the new photo.

FRONTEND. PeopleSheet and the admin manager rendered centred thumbnails and
ignored the bbox, so on group photos the avatar showed whoever stood in the
middle and two people from one photo were indistinguishable — in the manager
whose entire job is telling faces apart. The crop maths is now one shared
helper (faceCrop.ts) so the three surfaces cannot drift again. Full-page
layouts (gallery-premium, gallery-story) render their own lightbox and never
received the people props.

Backend failure set verified identical to origin/main; frontend 140 green;
tsc and eslint clean.

Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc

* fix(faces): round 3 — five of round 2's fixes were wrong or no-ops (#1074)

Third and final Codex round. Eight findings, four P1 — and the important part
is that FIVE of them are defects in round 2's fixes, not in the original code.

- The sync-upload enqueue I added was a silent no-op. It queried through
  `trx` after the transaction had already been committed, which throws
  "Transaction query already complete" straight into the catch I had wrapped
  it in. Chunked uploads and watch-folder imports were still never scanned,
  and the code read as though they were. Uses `db` now.

- The post-restore requeue ran BEFORE the files were restored, in both the
  portable importer and the native restore. The face worker is live during a
  restore, so it could claim those rows and scan the previous instance's
  files, or fail them for originals not yet on disk — with nothing to requeue
  them afterwards. Both now run after file restoration; the native one is
  extracted into requeueFaceScans() and called from the full and
  database-only paths.

- The admin face crop mixed coordinate spaces: an original-pixel bbox scaled
  against the THUMBNAIL's natural size. The API now returns the source
  dimensions alongside the box, so there is one space to reason about.

- Forwarding people props through layoutProps did not make them work — the
  full-page layouts never destructured them. GalleryStoryLayout now threads
  them to its own lightbox.

Genuinely new findings, all in the same class as ones already fixed:

- releaseToPending updated unconditionally, so a photo purged while its
  sidecar request was in flight came back as 'pending' and was rescanned —
  biometric rows reappearing after the purge reported success. Round 2 fixed
  exactly this on the COMMIT path and I did not carry it to the retry path.
  Now guarded on 'processing'.

- purgePhotoFaces left face_status alone, so a worker mid-scan still
  satisfied its commit guard and could write fresh faces into a photo being
  deleted — orphans, since the FK cascade is inert on SQLite. It now clears
  the claim as part of the purge.

- Phase 3 was unreachable: the migration seeds face_auto_categorize_enabled
  false and nothing could ever write it, so the rule engine and its undo
  endpoint returned "disabled" in every real flow. Added GET/PUT and a toggle
  on the admin card, EN + DE.

NOT fixed, deliberately: GalleryPremiumLayout uses yet-another-react-lightbox
rather than the shared PhotoLightbox, so person chips there are a real port
rather than a prop forward. Recorded as open rather than bodged.

Backend failure set identical to origin/main; 41 face tests and 140 frontend
tests green; i18n audit reports EN and DE complete at 71 keys.

Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc

* feat(faces): block face recognition on the all-in-one image (#1074, #1042)

The single-container image cannot run this feature, so it is refused there
rather than left to degrade.

WHY, since the reason is not obvious from the code: the AIO image runs the
backend, the frontend, SQLite and every background worker inside one
container aimed at "one photographer plus guests browsing". It has no Redis,
SQLite gives it a single writer, and it contains no ML sidecar to talk to.
Face detection would add a second image-processing pipeline competing with
Sharp for the same CPU and memory. That failure is not loud — the install
just becomes slow and looks broken, which is the worst possible shape for a
deployment whose whole promise is one container and no decisions.

Gated on an explicit PICPEAK_SINGLE_CONTAINER marker, NOT inferred from
SERVE_FRONTEND or a SQLite path: plenty of legitimate multi-container setups
serve the frontend from the backend or run SQLite, and none of them should
lose the feature by accident.

Three layers, because the first is the only one that enforces:

- faceSettings.isFeatureEnabled() returns false before consulting the flag,
  so a database restored from a full deployment with `faces` enabled still
  cannot switch it on here.
- The feature-flag API forces `faces: false` in both directions, so the admin
  UI reflects reality instead of offering a switch that refuses to stay on.
- The Features tab renders the card disabled with a plain-language reason,
  read from a new `single_container` field on /admin/system/version (an
  endpoint the admin UI already calls).

Documented in ml/README.md and .env.example. Three tests pin the behaviour,
including that the marker only accepts explicit truthy values.

NOTE FOR PR #1068: this expects `Dockerfile.aio` to set
`ENV PICPEAK_SINGLE_CONTAINER=true`. That one line lives on that branch and
is not in this commit — until it lands, an AIO build would still offer the
feature. Worth adding alongside the `Limits` section of docs/single-container.md.

44 face tests green; EN + DE complete at 72 keys.

Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc

* test(faces): pin the bbox coordinate space with a real scale factor (#1074)

The coordinate-space bug — boxes stored in preview space while every consumer
reads them as original-image pixels — had no test, and could not have been
caught by the ones that existed: every photo in the demo gallery is 750px, so
the scale factor was always exactly 1.0 and the correction never executed.

Verified by hand first, on a real 4000x3000 upload with the face placed
off-centre so a wrong crop would be unmistakable. Before the fix the stored
box was 1493,204 (preview space, face actually at x≈2850-3618); after, 3110,426
— a factor of 2.083, exactly 4000/1920, landing inside the face. The admin
crop then resolved to left=-395px/top=-46px on a 64px window, which is the
face centred.

That verification is now a test rather than a memory. Three cases: a 4000px
photo must scale by 4000/1920, a 1920px photo must NOT change (the case that
hid the bug), and a row with no width must fall back to unscaled rather than
storing NaN.

Note for anyone extending these: jest hoists mock factories above the file,
so anything they close over has to be `mock`-prefixed. Getting that wrong
fails at transform time with a message that does not name the variable.

47 face tests green.

Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc

---------

Co-authored-by: Paul Nothaft <[email protected]>
This commit is contained in:
Paul Nothaft
2026-08-18 22:37:28 +02:00
committed by GitHub
co-authored by Paul Nothaft
parent 6ebdd13dde
commit b69dd134d0
71 changed files with 8287 additions and 24 deletions
+49
View File
@@ -223,6 +223,55 @@ LOGS=./logs
# attempts is exponential: 1m, 5m, 30m, 2h, 12h. # attempts is exponential: 1m, 5m, 30m, 2h, 12h.
# WEBHOOK_MAX_ATTEMPTS=5 # WEBHOOK_MAX_ATTEMPTS=5
# -----------------------------------------------------------------------------
# Face recognition — "People in this gallery" (#1074, optional)
# -----------------------------------------------------------------------------
# Requires the optional picpeak-ml sidecar container:
# docker compose --profile faces up -d
#
# NONE of these variables do anything until the `faces` feature flag is
# enabled in Admin → Settings, AND the per-event "Detect people in this
# gallery" toggle is switched on. Both default to OFF. With the flag off the
# backend never contacts the sidecar, so leaving these at their defaults on an
# install without the container is completely inert.
#
# Face embeddings are biometric data (GDPR Art. 9 special category in the EU).
# The photographer is the controller and needs a lawful basis for the people
# in their photos — see docs/feature-face-recognition.md before enabling.
#
# NOT AVAILABLE ON THE ALL-IN-ONE IMAGE. The single-container build sets
# PICPEAK_SINGLE_CONTAINER=true and the backend refuses to enable face
# recognition there regardless of these variables or the feature flag: that
# image runs the backend, frontend, database and every worker in one
# container, with no ML sidecar to talk to, and face detection would compete
# with image processing for the same CPU and memory. Use the standard
# multi-container deployment if you want this feature.
#
# FACE_ML_TOKEN (no default — REQUIRED to run the sidecar)
# Shared secret between the backend and the sidecar. The sidecar refuses to
# start without it rather than serving anonymously, so an accidentally
# published port is never a free face-detection API. Generate with:
# openssl rand -hex 32
# FACE_ML_TOKEN=
#
# FACE_ML_URL (default: http://picpeak-ml:8000)
# Defaults to the sidecar's compose service name, so the standard
# deployment needs no configuration here. Only change it if you run the
# sidecar outside the default compose network.
# FACE_ML_URL=http://picpeak-ml:8000
#
# FACE_PROCESSOR_CONCURRENCY (default: 1)
# Face-detection workers in the backend. Defaults to 1 deliberately: face
# scanning shares a host with Sharp image processing, which is the real
# memory pressure (see UPLOAD_PROCESSOR_CONCURRENCY). Raise only on hosts
# with headroom to spare.
# FACE_PROCESSOR_CONCURRENCY=1
#
# FACE_ORT_THREADS (default: 1)
# ONNX Runtime threads inside the sidecar. More threads mean faster
# per-photo inference and higher RSS.
# FACE_ORT_THREADS=1
# Note on FRONTEND_API_URL (documentation only): # Note on FRONTEND_API_URL (documentation only):
# When using pre-built frontend images, runtime env vars cannot override the built JS. # When using pre-built frontend images, runtime env vars cannot override the built JS.
# Do NOT rely on FRONTEND_API_URL in Compose. Instead, keep VITE_API_URL=/api and # Do NOT rely on FRONTEND_API_URL in Compose. Instead, keep VITE_API_URL=/api and
+261 -1
View File
@@ -979,8 +979,245 @@ jobs:
if: failure() if: failure()
run: docker logs aio 2>&1 | tail -200 run: docker logs aio 2>&1 | tail -200
# -----------------------------------------------------------------------------
# ML sidecar (#1074): per-arch build, then merge into a multi-arch manifest
# -----------------------------------------------------------------------------
# Gated on the FACENET_ONNX_URL repository *variable* (Settings → Variables,
# not Secrets — it's a public release-asset URL). While it is unset, both ML
# jobs skip and the workflow behaves exactly as it did before this feature.
#
# Why a gate at all: deepface distributes FaceNet-512 as Keras .h5 only, so
# the ONNX has to be produced once by ml/tools/convert_facenet.py and
# published as a release asset before anything can build. Converting inside
# this workflow would drag TensorFlow (~600MB) through BOTH architecture legs
# of EVERY build to produce a file that is byte-identical each time.
#
# To activate, set two repository variables:
# FACENET_ONNX_URL https://github.com/PicPeak/picpeak/releases/download/<tag>/facenet512.onnx
# FACENET_ONNX_SHA256 <sha256 of that file>
# See ml/README.md for producing them.
build-ml:
if: vars.FACENET_ONNX_URL != ''
strategy:
fail-fast: false
matrix:
include:
- platform: linux/amd64
runner: ubuntu-latest
- platform: linux/arm64
runner: ubuntu-24.04-arm
runs-on: ${{ matrix.runner }}
permissions:
contents: read
packages: write
security-events: write
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Compute image names (lowercase for GHCR)
run: |
repo_lc="${GITHUB_REPOSITORY,,}"
echo "ML_IMAGE_NAME=${repo_lc}/ml" >> "$GITHUB_ENV"
- name: Prepare platform pair
run: |
platform="${{ matrix.platform }}"
echo "PLATFORM_PAIR=${platform//\//-}" >> "$GITHUB_ENV"
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to Container Registry
if: github.event_name != 'pull_request' || github.event.inputs.push == 'true'
id: login-ghcr
continue-on-error: true
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Determine if pushing
id: push-decision
run: |
if [[ "${{ github.event_name }}" == "pull_request" && "${{ github.event.inputs.push }}" != "true" ]]; then
echo "push=false" >> "$GITHUB_OUTPUT"
elif [[ "${{ steps.login-ghcr.outcome }}" != "success" ]]; then
echo "push=false" >> "$GITHUB_OUTPUT"
else
echo "push=true" >> "$GITHUB_OUTPUT"
fi
- name: Extract metadata for ML (labels only)
id: meta-ml
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.ML_IMAGE_NAME }}
labels: |
org.opencontainers.image.title=PicPeak ML
org.opencontainers.image.description=PicPeak face detection and embedding sidecar
org.opencontainers.image.vendor=PicPeak
maintainer=${{ github.repository_owner }}
- name: Build ML image (push by digest)
id: build
uses: docker/build-push-action@v5
with:
context: ./ml
file: ./ml/Dockerfile
platforms: ${{ matrix.platform }}
labels: ${{ steps.meta-ml.outputs.labels }}
cache-from: type=gha,scope=ml-${{ env.PLATFORM_PAIR }}
cache-to: type=gha,mode=max,scope=ml-${{ env.PLATFORM_PAIR }},ignore-error=true
outputs: ${{ steps.push-decision.outputs.push == 'true' && format('type=image,name={0}/{1},push-by-digest=true,name-canonical=true,push=true', env.REGISTRY, env.ML_IMAGE_NAME) || 'type=cacheonly' }}
build-args: |
CACHEBUST=${{ github.run_number }}
BUILD_DATE=${{ github.event.head_commit.timestamp }}
VCS_REF=${{ github.sha }}
VERSION=${{ steps.meta-ml.outputs.version }}
FACENET_ONNX_URL=${{ vars.FACENET_ONNX_URL }}
FACENET_ONNX_SHA256=${{ vars.FACENET_ONNX_SHA256 }}
- name: Export digest
if: steps.push-decision.outputs.push == 'true'
run: |
mkdir -p /tmp/digests
digest="${{ steps.build.outputs.digest }}"
touch "/tmp/digests/${digest#sha256:}"
- name: Upload digest artifact
if: steps.push-decision.outputs.push == 'true'
uses: actions/upload-artifact@v4
with:
name: digests-ml-${{ env.PLATFORM_PAIR }}
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1
# Per-arch scan by digest, same reasoning as the backend leg (#476).
# This image carries a Python/Debian dependency surface the other two
# don't, so it gets its own Security-tab category.
- name: Run Trivy vulnerability scanner (per-arch, by digest)
if: steps.push-decision.outputs.push == 'true'
uses: aquasecurity/[email protected]
env:
TRIVY_PLATFORM: ${{ matrix.platform }}
with:
image-ref: ${{ env.REGISTRY }}/${{ env.ML_IMAGE_NAME }}@${{ steps.build.outputs.digest }}
format: 'sarif'
output: 'trivy-ml-${{ env.PLATFORM_PAIR }}.sarif'
severity: 'CRITICAL,HIGH'
timeout: '10m'
- name: Upload Trivy scan results to GitHub Security tab
if: steps.push-decision.outputs.push == 'true'
uses: github/codeql-action/upload-sarif@v4
with:
sarif_file: 'trivy-ml-${{ env.PLATFORM_PAIR }}.sarif'
category: 'ml-vulnerabilities-${{ env.PLATFORM_PAIR }}'
merge-ml:
needs: build-ml
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
if: vars.FACENET_ONNX_URL != '' && (github.event_name != 'pull_request' || github.event.inputs.push == 'true')
steps:
- name: Compute image names (lowercase for GHCR)
run: |
repo_lc="${GITHUB_REPOSITORY,,}"
echo "ML_IMAGE_NAME=${repo_lc}/ml" >> "$GITHUB_ENV"
if [[ "$GITHUB_REPOSITORY" == "PicPeak/picpeak" ]]; then
echo "DOCKERHUB_ENABLED=true" >> "$GITHUB_ENV"
else
echo "DOCKERHUB_ENABLED=false" >> "$GITHUB_ENV"
fi
- name: Download digest artifacts
uses: actions/download-artifact@v4
with:
path: /tmp/digests
pattern: digests-ml-*
merge-multiple: true
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to Container Registry
id: login-ghcr
continue-on-error: true
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Determine build context
id: context
run: |
if [[ "${{ github.ref }}" == refs/tags/v*-beta* ]] || [[ "${{ github.ref }}" == refs/heads/main ]]; then
echo "channel=beta" >> $GITHUB_OUTPUT
echo "is_prerelease=true" >> $GITHUB_OUTPUT
else
echo "channel=stable" >> $GITHUB_OUTPUT
echo "is_prerelease=false" >> $GITHUB_OUTPUT
fi
- name: Log in to Docker Hub
if: env.DOCKERHUB_ENABLED == 'true'
uses: docker/login-action@v3
with:
registry: docker.io
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Extract metadata for ML
id: meta-ml
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.ML_IMAGE_NAME }}
${{ env.DOCKERHUB_ENABLED == 'true' && 'docker.io/picpeak/ml' || '' }}
labels: |
org.opencontainers.image.title=PicPeak ML
org.opencontainers.image.description=PicPeak face detection and embedding sidecar
org.opencontainers.image.vendor=PicPeak
maintainer=${{ github.repository_owner }}
# Identical tag scheme to backend/frontend: the sidecar's API contract
# is versioned with the backend that calls it, so PICPEAK_CHANNEL
# resolves the same string across all three images.
tags: |
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}},enable=${{ steps.context.outputs.is_prerelease == 'false' }}
type=semver,pattern={{major}},enable=${{ steps.context.outputs.is_prerelease == 'false' }}
type=ref,event=tag
type=sha,format=short
type=raw,value=latest,enable=${{ github.ref == 'refs/heads/stable' || (startsWith(github.ref, 'refs/tags/v') && steps.context.outputs.is_prerelease == 'false') }}
type=raw,value=stable,enable=${{ github.ref == 'refs/heads/stable' || (startsWith(github.ref, 'refs/tags/v') && steps.context.outputs.is_prerelease == 'false') }}
- name: Create and push multi-arch manifest
working-directory: /tmp/digests
run: |
docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
$(printf "${{ env.REGISTRY }}/${{ env.ML_IMAGE_NAME }}@sha256:%s " *)
- name: Inspect manifest (GHCR)
run: |
docker buildx imagetools inspect ${{ env.REGISTRY }}/${{ env.ML_IMAGE_NAME }}:${{ steps.meta-ml.outputs.version }}
- name: Inspect manifest (Docker Hub)
if: env.DOCKERHUB_ENABLED == 'true'
run: |
docker buildx imagetools inspect docker.io/picpeak/ml:${{ steps.meta-ml.outputs.version }}
summary: summary:
needs: [build-backend, merge-backend, build-frontend, merge-frontend, build-aio, merge-aio, smoke-aio] needs: [build-backend, merge-backend, build-frontend, merge-frontend, build-aio, merge-aio, smoke-aio, build-ml, merge-ml]
if: always() if: always()
runs-on: ubuntu-latest runs-on: ubuntu-latest
permissions: permissions:
@@ -992,6 +1229,7 @@ jobs:
repo_lc="${GITHUB_REPOSITORY,,}" repo_lc="${GITHUB_REPOSITORY,,}"
echo "BACKEND_IMAGE_NAME=${repo_lc}/backend" >> "$GITHUB_ENV" echo "BACKEND_IMAGE_NAME=${repo_lc}/backend" >> "$GITHUB_ENV"
echo "FRONTEND_IMAGE_NAME=${repo_lc}/frontend" >> "$GITHUB_ENV" echo "FRONTEND_IMAGE_NAME=${repo_lc}/frontend" >> "$GITHUB_ENV"
echo "ML_IMAGE_NAME=${repo_lc}/ml" >> "$GITHUB_ENV"
echo "AIO_IMAGE_NAME=${repo_lc}/aio" >> "$GITHUB_ENV" echo "AIO_IMAGE_NAME=${repo_lc}/aio" >> "$GITHUB_ENV"
# Mirror manifests to Docker Hub (picpeak/{backend,frontend}) only on the # Mirror manifests to Docker Hub (picpeak/{backend,frontend}) only on the
# canonical org repo, where the DOCKERHUB_* secrets live. Forks (and any # canonical org repo, where the DOCKERHUB_* secrets live. Forks (and any
@@ -1056,10 +1294,32 @@ jobs:
echo "❌ **AIO boot smoke**: ${{ needs.smoke-aio.result }}" >> $GITHUB_STEP_SUMMARY echo "❌ **AIO boot smoke**: ${{ needs.smoke-aio.result }}" >> $GITHUB_STEP_SUMMARY
fi fi
# The ML sidecar (#1074) is optional and only builds once the
# FACENET_ONNX_URL repository variable is set — "skipped" is the
# expected state, not a failure, so report it as such.
if [[ "${{ needs.build-ml.result }}" == "success" ]]; then
echo "✅ **ML sidecar build (per-arch)**: Successfully built" >> $GITHUB_STEP_SUMMARY
elif [[ "${{ needs.build-ml.result }}" == "skipped" ]]; then
echo "️ **ML sidecar build**: Skipped (FACENET_ONNX_URL repository variable not set — see ml/README.md)" >> $GITHUB_STEP_SUMMARY
else
echo "❌ **ML sidecar build (per-arch)**: ${{ needs.build-ml.result }}" >> $GITHUB_STEP_SUMMARY
fi
if [[ "${{ needs.merge-ml.result }}" == "success" ]]; then
echo "✅ **ML sidecar manifest merge**: Successfully published" >> $GITHUB_STEP_SUMMARY
elif [[ "${{ needs.merge-ml.result }}" == "skipped" ]]; then
echo "️ **ML sidecar manifest merge**: Skipped" >> $GITHUB_STEP_SUMMARY
else
echo "❌ **ML sidecar manifest merge**: ${{ needs.merge-ml.result }}" >> $GITHUB_STEP_SUMMARY
fi
echo "" >> $GITHUB_STEP_SUMMARY echo "" >> $GITHUB_STEP_SUMMARY
echo "### 📦 Images" >> $GITHUB_STEP_SUMMARY echo "### 📦 Images" >> $GITHUB_STEP_SUMMARY
echo "- Backend: \`${{ env.REGISTRY }}/${{ env.BACKEND_IMAGE_NAME }}\`" >> $GITHUB_STEP_SUMMARY echo "- Backend: \`${{ env.REGISTRY }}/${{ env.BACKEND_IMAGE_NAME }}\`" >> $GITHUB_STEP_SUMMARY
echo "- Frontend: \`${{ env.REGISTRY }}/${{ env.FRONTEND_IMAGE_NAME }}\`" >> $GITHUB_STEP_SUMMARY echo "- Frontend: \`${{ env.REGISTRY }}/${{ env.FRONTEND_IMAGE_NAME }}\`" >> $GITHUB_STEP_SUMMARY
if [[ "${{ needs.merge-ml.result }}" == "success" ]]; then
echo "- ML sidecar (optional): \`${{ env.REGISTRY }}/${{ env.ML_IMAGE_NAME }}\`" >> $GITHUB_STEP_SUMMARY
fi
echo "- All-in-one: \`${{ env.REGISTRY }}/${{ env.AIO_IMAGE_NAME }}\` (GHCR only — Docker Hub mirror pending)" >> $GITHUB_STEP_SUMMARY echo "- All-in-one: \`${{ env.REGISTRY }}/${{ env.AIO_IMAGE_NAME }}\` (GHCR only — Docker Hub mirror pending)" >> $GITHUB_STEP_SUMMARY
if [[ "$DOCKERHUB_ENABLED" == "true" ]]; then if [[ "$DOCKERHUB_ENABLED" == "true" ]]; then
echo "- Backend (Docker Hub): \`docker.io/picpeak/backend\`" >> $GITHUB_STEP_SUMMARY echo "- Backend (Docker Hub): \`docker.io/picpeak/backend\`" >> $GITHUB_STEP_SUMMARY
+30
View File
@@ -113,3 +113,33 @@ jobs:
- name: Run Vitest suite - name: Run Vitest suite
working-directory: ./frontend working-directory: ./frontend
run: npm test -- --run run: npm test -- --run
# Optional face-detection sidecar (#1074). Runs on every PR regardless of
# whether the feature is enabled anywhere — these tests need no model
# weights (they stub the pipeline out) and cover the auth boundary, the
# request guards and the alignment geometry, which is where a mistake is a
# security problem or a silent accuracy problem rather than a visible bug.
ml:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
# Matches ml/Dockerfile's base image, so a wheel that resolves here
# resolves in the image too.
python-version: '3.12'
cache: 'pip'
cache-dependency-path: ml/requirements.txt
- name: Install ml deps
working-directory: ./ml
run: pip install -r requirements.txt pytest httpx
- name: Run pytest suite
working-directory: ./ml
run: python -m pytest tests/ -q
+12
View File
@@ -133,3 +133,15 @@ new-layouts/
# Backend runtime storage (generated media, previews, thumbnails, # Backend runtime storage (generated media, previews, thumbnails,
# CRM/accounting documents) — never commit # CRM/accounting documents) — never commit
backend/storage/ backend/storage/
# Python artifacts — the picpeak-ml sidecar (#1074) is the only Python in
# this tree, but bytecode and virtualenvs must never be committed.
__pycache__/
*.py[cod]
.pytest_cache/
ml/.venv/
ml/venv/
# Locally produced model weights. The image fetches these by pinned URL and
# SHA-256 at build time; a 90MB blob must not end up in git history.
ml/*.onnx
ml/*.h5
@@ -0,0 +1,217 @@
/**
* Auto-category rule engine (#1074 phase 3).
*
* The rules themselves are simple enough to read. What needs testing is the
* promise around them: this engine may only ever fill an EMPTY category, and
* everything it touches must be reversible. A photographer's own assignment
* is a decision; this is a heuristic, and the heuristic never wins.
*/
const path = require('path');
const fs = require('fs');
const os = require('os');
process.env.NODE_ENV = 'test';
process.env.TEST_DATABASE_PATH = path.join(
fs.mkdtempSync(path.join(os.tmpdir(), 'picpeak-autocat-')), 'db.sqlite',
);
process.env.JWT_SECRET = process.env.JWT_SECRET || 'autocat-test-secret';
const { bootCrmDb } = require('./helpers/crmDb');
let db; let cleanup; let engine;
async function seedEvent(slug) {
const [row] = await db('events').insert({
slug,
event_type: 'wedding',
event_name: slug,
event_date: '2026-01-01',
host_email: '[email protected]',
admin_email: '[email protected]',
password_hash: 'x',
share_link: `${slug}-share`,
expires_at: new Date().toISOString(),
face_recognition_enabled: true,
}).returning('id');
return typeof row === 'object' ? row.id : row;
}
/** A scanned photo with `faceCount` faces, each `faceSide` px square. */
async function addScannedPhoto(eventId, faceCount, { faceSide = 400, categoryId = null } = {}) {
const [p] = await db('photos').insert({
event_id: eventId,
filename: `${Math.random()}.jpg`,
path: '/tmp/x.jpg',
type: 'individual',
width: 1000,
height: 1000,
processing_status: 'complete',
face_status: 'done',
face_count: faceCount,
category_id: categoryId,
}).returning('id');
const photoId = typeof p === 'object' ? p.id : p;
for (let i = 0; i < faceCount; i++) {
await db('photo_faces').insert({
photo_id: photoId,
event_id: eventId,
bbox_x: 10, bbox_y: 10, bbox_w: faceSide, bbox_h: faceSide,
det_score: 0.95,
model_version: 'test-v1',
created_at: new Date().toISOString(),
});
}
return photoId;
}
async function enable(on) {
const existing = await db('app_settings')
.where('setting_key', 'face_auto_categorize_enabled').first();
if (existing) {
await db('app_settings')
.where('setting_key', 'face_auto_categorize_enabled')
.update({ setting_value: JSON.stringify(on) });
}
}
async function categoryOf(photoId) {
const photo = await db('photos').where({ id: photoId }).first();
if (!photo.category_id) return null;
const cat = await db('photo_categories').where({ id: photo.category_id }).first();
return cat?.slug ?? null;
}
describe('faceAutoCategories (#1074 phase 3)', () => {
beforeAll(async () => {
({ db, cleanup } = await bootCrmDb());
engine = require('../../src/services/faceAutoCategories');
await enable(true);
}, 120000);
afterAll(async () => { if (cleanup) await cleanup(); });
describe('rules', () => {
it('sorts by face count, and by face size for portraits', async () => {
const eventId = await seedEvent('rules');
// 400px face in a 1000x1000 frame = 16% of the frame, over the 8% floor.
const portrait = await addScannedPhoto(eventId, 1, { faceSide: 400 });
const details = await addScannedPhoto(eventId, 0);
const small = await addScannedPhoto(eventId, 3);
const group = await addScannedPhoto(eventId, 9);
await engine.categorizeEvent(eventId);
expect(await categoryOf(details)).toBe('details');
expect(await categoryOf(portrait)).toBe('portraits');
expect(await categoryOf(small)).toBe('small-groups');
expect(await categoryOf(group)).toBe('groups');
});
it('does not call a distant single face a portrait', async () => {
// One person in a wide landscape is not a portrait of them. 60px in a
// 1000x1000 frame is 0.36% — far below the 8% floor.
const eventId = await seedEvent('small-face');
const distant = await addScannedPhoto(eventId, 1, { faceSide: 60 });
await engine.categorizeEvent(eventId);
expect(await categoryOf(distant)).toBeNull();
});
it('ignores photos that have not been scanned', async () => {
const eventId = await seedEvent('unscanned');
const [p] = await db('photos').insert({
event_id: eventId, filename: 'u.jpg', path: '/tmp/u.jpg', type: 'individual',
processing_status: 'complete', face_status: 'pending',
}).returning('id');
const photoId = typeof p === 'object' ? p.id : p;
await engine.categorizeEvent(eventId);
expect(await categoryOf(photoId)).toBeNull();
});
});
describe('the promise', () => {
it('NEVER overwrites a category a person chose', async () => {
// The single most important behaviour in this file.
const eventId = await seedEvent('no-overwrite');
const [c] = await db('photo_categories').insert({
name: 'Ceremony', slug: 'ceremony', is_global: false, event_id: eventId,
created_at: new Date().toISOString(),
}).returning('id');
const ceremonyId = typeof c === 'object' ? c.id : c;
// 9 faces — the rules would call this "groups" if they were allowed to.
const claimed = await addScannedPhoto(eventId, 9, { categoryId: ceremonyId });
await engine.categorizeEvent(eventId);
expect(await categoryOf(claimed)).toBe('ceremony');
const row = await db('photos').where({ id: claimed }).first();
expect(row.auto_categorized).toBeFalsy();
});
it('marks only what it assigned, so undo is exact', async () => {
const eventId = await seedEvent('undo');
const [c] = await db('photo_categories').insert({
name: 'Ceremony', slug: 'ceremony-2', is_global: false, event_id: eventId,
created_at: new Date().toISOString(),
}).returning('id');
const ceremonyId = typeof c === 'object' ? c.id : c;
const manual = await addScannedPhoto(eventId, 4, { categoryId: ceremonyId });
const auto = await addScannedPhoto(eventId, 4);
await engine.categorizeEvent(eventId);
expect(await categoryOf(auto)).toBe('small-groups');
const result = await engine.undoEvent(eventId);
expect(result.cleared).toBe(1);
// The automatic one is cleared...
expect(await categoryOf(auto)).toBeNull();
// ...and the photographer's own choice survives untouched.
expect(await categoryOf(manual)).toBe('ceremony-2');
});
it('is a no-op while the setting is off', async () => {
const eventId = await seedEvent('disabled');
const photoId = await addScannedPhoto(eventId, 0);
await enable(false);
const result = await engine.categorizeEvent(eventId);
await enable(true);
expect(result.skipped).toBe(true);
expect(await categoryOf(photoId)).toBeNull();
});
it('is idempotent — a second run assigns nothing new', async () => {
const eventId = await seedEvent('idempotent');
await addScannedPhoto(eventId, 0);
await addScannedPhoto(eventId, 7);
const first = await engine.categorizeEvent(eventId);
const second = await engine.categorizeEvent(eventId);
expect(first.assigned).toBe(2);
expect(second.assigned).toBe(0);
});
it('reuses one category per slug rather than creating duplicates', async () => {
const eventId = await seedEvent('reuse');
await addScannedPhoto(eventId, 0);
await addScannedPhoto(eventId, 0);
await addScannedPhoto(eventId, 0);
await engine.categorizeEvent(eventId);
const details = await db('photo_categories')
.where({ slug: 'details' })
.where(function () { this.where('event_id', eventId).orWhere('is_global', true); });
expect(details).toHaveLength(1);
});
});
});
@@ -0,0 +1,325 @@
/**
* Clustering engine (#1074).
*
* Uses synthetic embeddings with known identities rather than real faces: the
* question here is whether the ALGORITHM groups vectors correctly, which is
* separable from whether the model produces good vectors. Model quality is
* the spike's job.
*/
const path = require('path');
const fs = require('fs');
const os = require('os');
process.env.NODE_ENV = 'test';
process.env.TEST_DATABASE_PATH = path.join(
fs.mkdtempSync(path.join(os.tmpdir(), 'picpeak-faceclust-')), 'db.sqlite',
);
process.env.JWT_SECRET = process.env.JWT_SECRET || 'faceclust-test-secret';
const { bootCrmDb } = require('./helpers/crmDb');
let db; let cleanup; let clustering;
/** Deterministic unit vector for identity `id`, jittered by `variant`. */
function makeEmbedding(id, variant = 0, dim = 64) {
const vec = new Float32Array(dim);
for (let i = 0; i < dim; i++) {
vec[i] = Math.sin((i + 1) * (id + 1) * 0.7) + variant * 0.02 * Math.cos(i * 3.1);
}
let norm = 0;
for (let i = 0; i < dim; i++) norm += vec[i] * vec[i];
norm = Math.sqrt(norm);
for (let i = 0; i < dim; i++) vec[i] /= norm;
return vec;
}
async function seedEvent(slug) {
const [row] = await db('events').insert({
slug,
event_type: 'wedding',
event_name: slug,
event_date: '2026-01-01',
host_email: '[email protected]',
admin_email: '[email protected]',
password_hash: 'x',
share_link: `${slug}-share`,
expires_at: new Date().toISOString(),
}).returning('id');
return typeof row === 'object' ? row.id : row;
}
async function insertFace(eventId, embedding, overrides = {}) {
const [p] = await db('photos').insert({
event_id: eventId,
filename: `${Math.random()}.jpg`,
path: '/tmp/x.jpg',
type: 'individual',
}).returning('id');
const photoId = typeof p === 'object' ? p.id : p;
const row = {
photo_id: photoId,
event_id: eventId,
bbox_x: 0, bbox_y: 0, bbox_w: 200, bbox_h: 200,
det_score: 0.99,
embedding: clustering.packEmbedding(embedding),
model_version: 'test-v1',
created_at: new Date().toISOString(),
...overrides,
};
const [f] = await db('photo_faces').insert(row).returning('id');
return { ...row, id: typeof f === 'object' ? f.id : f };
}
describe('faceClustering (#1074)', () => {
beforeAll(async () => {
({ db, cleanup } = await bootCrmDb());
clustering = require('../../src/services/faceClustering');
}, 120000);
afterAll(async () => { if (cleanup) await cleanup(); });
describe('embedding round-trip', () => {
it('survives pack/unpack through the BLOB column exactly', async () => {
const original = makeEmbedding(1);
const eventId = await seedEvent('roundtrip');
const face = await insertFace(eventId, original);
const stored = await db('photo_faces').where({ id: face.id }).first();
const restored = clustering.unpackEmbedding(stored.embedding);
expect(restored).toHaveLength(original.length);
for (let i = 0; i < original.length; i++) {
expect(restored[i]).toBeCloseTo(original[i], 6);
}
});
it('returns null for a corrupt blob rather than throwing', () => {
expect(clustering.unpackEmbedding(Buffer.from([1, 2, 3]))).toBeNull();
expect(clustering.unpackEmbedding(null)).toBeNull();
});
});
describe('assignment', () => {
it('groups the same identity and separates different ones', async () => {
const eventId = await seedEvent('grouping');
const faces = [];
// Three identities, four shots each, interleaved so assignment order
// is not conveniently grouped.
for (let variant = 0; variant < 4; variant++) {
for (const identity of [1, 2, 3]) {
faces.push(await insertFace(eventId, makeEmbedding(identity, variant)));
}
}
await clustering.assignFaces(eventId, faces);
const people = await db('event_people').where({ event_id: eventId });
expect(people).toHaveLength(3);
// Every face of one identity must share a person id.
const rows = await db('photo_faces').where({ event_id: eventId }).select('id', 'person_id');
const byPerson = new Map();
for (const r of rows) {
byPerson.set(r.person_id, (byPerson.get(r.person_id) || 0) + 1);
}
expect([...byPerson.values()].sort()).toEqual([4, 4, 4]);
});
it('leaves low-quality faces unassigned instead of spawning junk people', async () => {
const eventId = await seedEvent('quality-floor');
const good = await insertFace(eventId, makeEmbedding(5));
// Tiny bbox — below the 40px floor.
const tiny = await insertFace(eventId, makeEmbedding(6), { bbox_w: 12, bbox_h: 12 });
// Weak detection score.
const weak = await insertFace(eventId, makeEmbedding(7), { det_score: 0.2 });
await clustering.assignFaces(eventId, [good, tiny, weak]);
const rows = await db('photo_faces')
.whereIn('id', [good.id, tiny.id, weak.id])
.select('id', 'person_id');
const map = Object.fromEntries(rows.map((r) => [r.id, r.person_id]));
expect(map[good.id]).not.toBeNull();
// Still stored — they show in "this photo contains" — just unassigned.
expect(map[tiny.id]).toBeNull();
expect(map[weak.id]).toBeNull();
expect(await db('event_people').where({ event_id: eventId })).toHaveLength(1);
});
it('never mixes embedding spaces from different model versions', async () => {
const eventId = await seedEvent('model-version');
const a = await insertFace(eventId, makeEmbedding(9), { model_version: 'v1' });
await clustering.assignFaces(eventId, [a]);
// Same vector, different model. Comparable numerically, meaningless
// semantically — it must NOT join the v1 cluster.
const b = await insertFace(eventId, makeEmbedding(9), { model_version: 'v2' });
await clustering.assignFaces(eventId, [b]);
const people = await db('event_people').where({ event_id: eventId });
expect(people).toHaveLength(2);
});
});
describe('merge and split', () => {
it('merge moves every face and removes the source person', async () => {
const eventId = await seedEvent('merge');
const f1 = await insertFace(eventId, makeEmbedding(11));
const f2 = await insertFace(eventId, makeEmbedding(21));
await clustering.assignFaces(eventId, [f1, f2]);
const people = await db('event_people').where({ event_id: eventId }).orderBy('id');
expect(people).toHaveLength(2);
await clustering.mergePeople(eventId, [people[1].id], people[0].id);
expect(await db('event_people').where({ event_id: eventId })).toHaveLength(1);
const remaining = await db('event_people').where({ event_id: eventId }).first();
expect(remaining.face_count_total).toBe(2);
const orphaned = await db('photo_faces')
.where({ event_id: eventId }).whereNull('person_id');
expect(orphaned).toHaveLength(0);
});
it('split pulls the named faces into a new person', async () => {
const eventId = await seedEvent('split');
const faces = [];
for (let v = 0; v < 4; v++) faces.push(await insertFace(eventId, makeEmbedding(13, v)));
await clustering.assignFaces(eventId, faces);
const person = await db('event_people').where({ event_id: eventId }).first();
expect(person.face_count_total).toBe(4);
const newId = await clustering.splitPerson(eventId, person.id, [faces[0].id, faces[1].id]);
expect(newId).toBeTruthy();
const original = await db('event_people').where({ id: person.id }).first();
const created = await db('event_people').where({ id: newId }).first();
expect(original.face_count_total).toBe(2);
expect(created.face_count_total).toBe(2);
});
it('deletes a person left with no faces rather than keeping a ghost', async () => {
const eventId = await seedEvent('empty-person');
const f = await insertFace(eventId, makeEmbedding(15));
await clustering.assignFaces(eventId, [f]);
const person = await db('event_people').where({ event_id: eventId }).first();
await db('photo_faces').where({ id: f.id }).update({ person_id: null });
await clustering.recomputeCentroid(person.id);
expect(await db('event_people').where({ id: person.id }).first()).toBeUndefined();
});
});
describe('regressions from external review', () => {
it('merge carries a name and suppression onto the survivor', async () => {
// A merge used to move the faces and delete the source outright, so a
// photographer-entered name vanished and a person they had hidden came
// back guest-visible.
const eventId = await seedEvent('merge-metadata');
const a = await insertFace(eventId, makeEmbedding(61));
const b = await insertFace(eventId, makeEmbedding(62));
await clustering.assignFaces(eventId, [a, b]);
const [p1, p2] = await db('event_people').where({ event_id: eventId }).orderBy('id');
// Target is unnamed and visible; the SOURCE carries the human state.
await db('event_people').where({ id: p2.id }).update({ label: 'Anna', is_hidden: true });
await clustering.mergePeople(eventId, [p2.id], p1.id);
const survivor = await db('event_people').where({ id: p1.id }).first();
expect(survivor.label).toBe('Anna');
expect(!!survivor.is_hidden).toBe(true);
});
it('recluster keeps hidden/ignored on people that were never named', async () => {
// The old query remembered only rows with a label, so a suppressed
// bystander came back visible after one "Re-group people".
const eventId = await seedEvent('recluster-suppression');
const faces = [];
for (let v = 0; v < 3; v++) faces.push(await insertFace(eventId, makeEmbedding(71, v)));
await clustering.assignFaces(eventId, faces);
const person = await db('event_people').where({ event_id: eventId }).first();
expect(person.label).toBeNull();
await db('event_people').where({ id: person.id }).update({ is_ignored: true });
await clustering.recluster(eventId);
const after = await db('event_people').where({ event_id: eventId });
expect(after.length).toBeGreaterThan(0);
expect(after.every((p) => !!p.is_ignored)).toBe(true);
});
});
describe('recluster', () => {
it('re-derives clusters and preserves photographer-assigned names', async () => {
// This is the property that makes re-clustering safe to offer as a
// button: without it, one click silently discards every typed name.
const eventId = await seedEvent('recluster');
const faces = [];
for (let v = 0; v < 3; v++) {
faces.push(await insertFace(eventId, makeEmbedding(31, v)));
faces.push(await insertFace(eventId, makeEmbedding(32, v)));
}
await clustering.assignFaces(eventId, faces);
const people = await db('event_people').where({ event_id: eventId }).orderBy('id');
expect(people).toHaveLength(2);
await db('event_people').where({ id: people[0].id }).update({ label: 'Anna' });
await db('event_people').where({ id: people[1].id }).update({ label: 'Ben' });
const count = await clustering.recluster(eventId);
expect(count).toBe(2);
const after = await db('event_people').where({ event_id: eventId });
const labels = after.map((p) => p.label).filter(Boolean).sort();
expect(labels).toEqual(['Anna', 'Ben']);
});
it('is stable across repeated runs', async () => {
const eventId = await seedEvent('recluster-stable');
const faces = [];
for (let v = 0; v < 3; v++) {
for (const id of [41, 42]) faces.push(await insertFace(eventId, makeEmbedding(id, v)));
}
await clustering.assignFaces(eventId, faces);
const first = await clustering.recluster(eventId);
const second = await clustering.recluster(eventId);
expect(second).toBe(first);
});
});
describe('consolidate', () => {
it('refuses to merge two people the photographer named differently', async () => {
// A human assertion this heuristic does not get to overrule.
const eventId = await seedEvent('consolidate-labels');
const a = await insertFace(eventId, makeEmbedding(51));
await clustering.assignFaces(eventId, [a]);
const first = await db('event_people').where({ event_id: eventId }).first();
// A near-identical centroid that would otherwise merge.
const [inserted] = await db('event_people').insert({
event_id: eventId,
centroid: clustering.packEmbedding(makeEmbedding(51, 0.01)),
face_count_total: 1,
model_version: 'test-v1',
created_at: new Date().toISOString(),
updated_at: new Date().toISOString(),
}).returning('id');
const secondId = typeof inserted === 'object' ? inserted.id : inserted;
await db('event_people').where({ id: first.id }).update({ label: 'Anna' });
await db('event_people').where({ id: secondId }).update({ label: 'Ben' });
await clustering.consolidate(eventId);
expect(await db('event_people').where({ event_id: eventId })).toHaveLength(2);
});
});
});
@@ -0,0 +1,339 @@
/**
* Privacy and visibility guarantees for face recognition (#1074).
*
* These are the tests that matter most in this feature. Two of them cover
* defects that would be invisible in normal use:
*
* - The people strip is computed from face rows, which have no concept of
* photo visibility. Handing a guest a raw count leaks how many hidden
* photos someone appears in, and a cover face picked without scoping
* renders a crop of a photo the guest may not open.
*
* - Face embeddings are biometric data. They must not ride along in a
* .picpeak export, which gets handed to clients and moved between
* operators.
*/
const path = require('path');
const fs = require('fs');
const os = require('os');
process.env.NODE_ENV = 'test';
process.env.TEST_DATABASE_PATH = path.join(
fs.mkdtempSync(path.join(os.tmpdir(), 'picpeak-faceprivacy-')), 'db.sqlite',
);
process.env.JWT_SECRET = process.env.JWT_SECRET || 'faceprivacy-test-secret';
const { bootCrmDb } = require('./helpers/crmDb');
let db; let cleanup; let clustering; let peopleService; let faceProcessor;
function makeEmbedding(id, variant = 0, dim = 64) {
const vec = new Float32Array(dim);
for (let i = 0; i < dim; i++) {
vec[i] = Math.sin((i + 1) * (id + 1) * 0.7) + variant * 0.02 * Math.cos(i * 3.1);
}
let norm = 0;
for (let i = 0; i < dim; i++) norm += vec[i] * vec[i];
norm = Math.sqrt(norm);
for (let i = 0; i < dim; i++) vec[i] /= norm;
return vec;
}
async function seedEvent(slug) {
const [row] = await db('events').insert({
slug,
event_type: 'wedding',
event_name: slug,
event_date: '2026-01-01',
host_email: '[email protected]',
admin_email: '[email protected]',
password_hash: 'x',
share_link: `${slug}-share`,
expires_at: new Date().toISOString(),
face_recognition_enabled: true,
}).returning('id');
return typeof row === 'object' ? row.id : row;
}
async function addPhotoWithFace(eventId, embedding, { visibility = 'visible', score = 0.99 } = {}) {
const [p] = await db('photos').insert({
event_id: eventId,
filename: `${Math.random()}.jpg`,
path: '/tmp/x.jpg',
type: 'individual',
visibility,
processing_status: 'complete',
}).returning('id');
const photoId = typeof p === 'object' ? p.id : p;
const row = {
photo_id: photoId,
event_id: eventId,
bbox_x: 0, bbox_y: 0, bbox_w: 200, bbox_h: 200,
det_score: score,
embedding: clustering.packEmbedding(embedding),
model_version: 'test-v1',
created_at: new Date().toISOString(),
};
const [f] = await db('photo_faces').insert(row).returning('id');
return { photoId, face: { ...row, id: typeof f === 'object' ? f.id : f } };
}
describe('face privacy and visibility (#1074)', () => {
beforeAll(async () => {
({ db, cleanup } = await bootCrmDb());
clustering = require('../../src/services/faceClustering');
peopleService = require('../../src/services/facePeopleService');
faceProcessor = require('../../src/services/faceProcessor');
}, 120000);
afterAll(async () => { if (cleanup) await cleanup(); });
describe('visibility scoping', () => {
it('counts only photos the audience can actually see', async () => {
const eventId = await seedEvent('visibility-count');
const faces = [];
// Same person: 3 visible photos, 4 hidden ones.
for (let v = 0; v < 3; v++) {
faces.push((await addPhotoWithFace(eventId, makeEmbedding(1, v))).face);
}
for (let v = 3; v < 7; v++) {
faces.push((await addPhotoWithFace(eventId, makeEmbedding(1, v), { visibility: 'hidden' })).face);
}
await clustering.assignFaces(eventId, faces);
const guestView = await peopleService.listPeople(eventId, { isClient: false, minClusterSize: 1 });
const clientView = await peopleService.listPeople(eventId, { isClient: true, minClusterSize: 1 });
expect(guestView).toHaveLength(1);
// The leak this test exists to prevent: 3, never 7.
expect(guestView[0].face_count).toBe(3);
expect(clientView[0].face_count).toBe(7);
});
it('never returns face_count_total to a guest', async () => {
const eventId = await seedEvent('no-total-leak');
const { face } = await addPhotoWithFace(eventId, makeEmbedding(2));
await clustering.assignFaces(eventId, [face]);
const [person] = await peopleService.listPeople(eventId, { isClient: false, minClusterSize: 1 });
expect(person).not.toHaveProperty('total_face_count');
expect(person).not.toHaveProperty('is_hidden');
});
it('picks a cover face from a photo the guest may open', async () => {
const eventId = await seedEvent('cover-scoping');
// The BEST face (highest score) is in a hidden photo — a naive
// implementation would hand its crop to the guest.
const hidden = await addPhotoWithFace(eventId, makeEmbedding(3, 0), {
visibility: 'hidden', score: 0.99,
});
const visible = await addPhotoWithFace(eventId, makeEmbedding(3, 1), {
visibility: 'visible', score: 0.80,
});
await clustering.assignFaces(eventId, [hidden.face, visible.face]);
const [guestPerson] = await peopleService.listPeople(eventId, { isClient: false, minClusterSize: 1 });
expect(guestPerson.cover.photo_id).toBe(visible.photoId);
expect(guestPerson.cover.photo_id).not.toBe(hidden.photoId);
});
it('drops a person entirely when all their photos are hidden', async () => {
const eventId = await seedEvent('all-hidden');
const faces = [];
for (let v = 0; v < 3; v++) {
faces.push((await addPhotoWithFace(eventId, makeEmbedding(4, v), { visibility: 'hidden' })).face);
}
await clustering.assignFaces(eventId, faces);
const guestView = await peopleService.listPeople(eventId, { isClient: false, minClusterSize: 1 });
expect(guestView).toHaveLength(0);
const clientView = await peopleService.listPeople(eventId, { isClient: true, minClusterSize: 1 });
expect(clientView).toHaveLength(1);
});
it('omits hidden and ignored people from the guest response', async () => {
const eventId = await seedEvent('hidden-people');
const a = (await addPhotoWithFace(eventId, makeEmbedding(5))).face;
const b = (await addPhotoWithFace(eventId, makeEmbedding(6))).face;
await clustering.assignFaces(eventId, [a, b]);
const people = await db('event_people').where({ event_id: eventId }).orderBy('id');
await db('event_people').where({ id: people[0].id }).update({ is_hidden: true });
await db('event_people').where({ id: people[1].id }).update({ is_ignored: true });
const guestView = await peopleService.listPeople(eventId, { isClient: false, minClusterSize: 1 });
expect(guestView).toHaveLength(0);
const adminView = await peopleService.listPeople(eventId, { isClient: true, forAdmin: true });
expect(adminView).toHaveLength(2);
});
it('does not attach a hidden person to a photo a guest can see', async () => {
const eventId = await seedEvent('person-ids-hidden');
const { photoId, face } = await addPhotoWithFace(eventId, makeEmbedding(7));
await clustering.assignFaces(eventId, [face]);
const person = await db('event_people').where({ event_id: eventId }).first();
await db('event_people').where({ id: person.id }).update({ is_hidden: true });
const guestMap = await peopleService.getPersonIdsByPhoto(eventId, [photoId], { forAdmin: false });
expect(guestMap.get(photoId)).toBeUndefined();
const adminMap = await peopleService.getPersonIdsByPhoto(eventId, [photoId], { forAdmin: true });
expect(adminMap.get(photoId)).toEqual([person.id]);
});
it('respects the minimum cluster size so one-off bystanders stay out', async () => {
const eventId = await seedEvent('min-cluster');
const solo = (await addPhotoWithFace(eventId, makeEmbedding(8))).face;
const crowd = [];
for (let v = 0; v < 4; v++) {
crowd.push((await addPhotoWithFace(eventId, makeEmbedding(9, v))).face);
}
await clustering.assignFaces(eventId, [solo, ...crowd]);
const people = await peopleService.listPeople(eventId, { isClient: false, minClusterSize: 3 });
expect(people).toHaveLength(1);
expect(people[0].face_count).toBe(4);
});
});
describe('erasure', () => {
it('purgeEvent removes every face row and resets the photos', async () => {
const eventId = await seedEvent('purge');
const faces = [];
for (let v = 0; v < 3; v++) {
faces.push((await addPhotoWithFace(eventId, makeEmbedding(10, v))).face);
}
await clustering.assignFaces(eventId, faces);
await db('photos').where({ event_id: eventId }).update({ face_status: 'done', face_count: 1 });
expect(await db('photo_faces').where({ event_id: eventId })).not.toHaveLength(0);
expect(await db('event_people').where({ event_id: eventId })).not.toHaveLength(0);
await faceProcessor.purgeEvent(eventId);
expect(await db('photo_faces').where({ event_id: eventId })).toHaveLength(0);
expect(await db('event_people').where({ event_id: eventId })).toHaveLength(0);
const photos = await db('photos').where({ event_id: eventId });
expect(photos.every((p) => p.face_status === null && p.face_count === null)).toBe(true);
});
it('purgePhotoFaces removes face rows WITHOUT relying on the FK cascade', async () => {
// The regression this guards: PicPeak does not enable
// `PRAGMA foreign_keys` on SQLite, so ON DELETE CASCADE never fires
// there and biometric embeddings outlived the photo. The pragma is
// explicitly OFF here so the assertion can only pass if the deletion
// path purges the rows itself.
await db.raw('PRAGMA foreign_keys = OFF');
const eventId = await seedEvent('purge-no-cascade');
const faces = [];
for (let v = 0; v < 3; v++) {
faces.push((await addPhotoWithFace(eventId, makeEmbedding(20, v))).face);
}
await clustering.assignFaces(eventId, faces);
const person = await db('event_people').where({ event_id: eventId }).first();
expect(person.face_count_total).toBe(3);
const victim = faces[0];
await faceProcessor.purgePhotoFaces(victim.photo_id);
expect(await db('photo_faces').where({ photo_id: victim.photo_id })).toHaveLength(0);
// …and the person it belonged to was rebuilt, not left with a stale count.
const after = await db('event_people').where({ id: person.id }).first();
expect(after.face_count_total).toBe(2);
});
it('purging the last face of a person removes the person too', async () => {
await db.raw('PRAGMA foreign_keys = OFF');
const eventId = await seedEvent('purge-last-face');
const { face, photoId } = await addPhotoWithFace(eventId, makeEmbedding(21));
await clustering.assignFaces(eventId, [face]);
expect(await db('event_people').where({ event_id: eventId })).toHaveLength(1);
await faceProcessor.purgePhotoFaces(photoId);
expect(await db('event_people').where({ event_id: eventId })).toHaveLength(0);
});
it('deleting an event removes its people and faces', async () => {
await db.raw('PRAGMA foreign_keys = ON');
const eventId = await seedEvent('event-delete');
const { face } = await addPhotoWithFace(eventId, makeEmbedding(11));
await clustering.assignFaces(eventId, [face]);
await db('photos').where({ event_id: eventId }).del();
await db('events').where({ id: eventId }).del();
expect(await db('photo_faces').where({ event_id: eventId })).toHaveLength(0);
expect(await db('event_people').where({ event_id: eventId })).toHaveLength(0);
});
});
describe('all-in-one image block (#1042 / PR #1068)', () => {
// Blocked for performance: the AIO image runs backend, frontend, SQLite
// and every worker in one container, with no ML sidecar to talk to. The
// failure there would not be loud — just a slow install that looks
// broken — so the gate is asserted rather than assumed.
const faceSettings = require('../../src/services/faceSettings');
afterEach(() => { delete process.env.PICPEAK_SINGLE_CONTAINER; });
it('reports the feature off regardless of the flag row', async () => {
process.env.PICPEAK_SINGLE_CONTAINER = 'true';
expect(faceSettings.isSingleContainerImage()).toBe(true);
// Even with the flag ON in the database.
await db('feature_flags').insert({ key: 'faces', value: true })
.onConflict('key').merge()
.catch(async () => {
await db('feature_flags').where({ key: 'faces' }).update({ value: true });
});
expect(await faceSettings.isFeatureEnabled()).toBe(false);
});
it('refuses per-event detection too', async () => {
process.env.PICPEAK_SINGLE_CONTAINER = 'true';
const eventId = await seedEvent('aio-block');
const event = await db('events').where({ id: eventId }).first();
expect(event.face_recognition_enabled).toBeTruthy();
expect(await faceSettings.isEnabledForEvent(event)).toBe(false);
});
it('accepts only explicit truthy markers', () => {
for (const v of ['true', '1', 'yes', 'TRUE']) {
process.env.PICPEAK_SINGLE_CONTAINER = v;
expect(faceSettings.isSingleContainerImage()).toBe(true);
}
for (const v of ['false', '0', '', 'no']) {
process.env.PICPEAK_SINGLE_CONTAINER = v;
expect(faceSettings.isSingleContainerImage()).toBe(false);
}
delete process.env.PICPEAK_SINGLE_CONTAINER;
expect(faceSettings.isSingleContainerImage()).toBe(false);
});
});
describe('export and backup exclusion', () => {
it('excludes both face tables from .picpeak exports', () => {
const { EXCLUDED_TABLES } = require('../../src/services/picpeakExportService');
expect(EXCLUDED_TABLES.has('photo_faces')).toBe(true);
expect(EXCLUDED_TABLES.has('event_people')).toBe(true);
});
it('excludes both face tables from the database backup table list', async () => {
const databaseBackup = require('../../src/services/databaseBackup');
const service = databaseBackup.DatabaseBackupService
? new databaseBackup.DatabaseBackupService()
: databaseBackup;
if (typeof service.getTables !== 'function') return; // shape differs; covered by the export test
const tables = await service.getTables();
expect(tables).not.toContain('photo_faces');
expect(tables).not.toContain('event_people');
// Sanity: the filter didn't eat everything.
expect(tables).toContain('events');
});
});
});
@@ -0,0 +1,146 @@
/**
* Bounding-box coordinate space (#1074).
*
* The sidecar reports boxes in the pixel space of the image it was HANDED —
* the ≤1920px preview — while every consumer (the strip's avatar crop, the
* admin manager, the auto-category portrait rule) compares them against
* photos.width/height, the ORIGINAL dimensions. faceProcessor scales once so
* everything downstream can assume original-image coordinates.
*
* This is the defect that survived longest in review, and it is invisible on
* any photo already under 1920px — the entire demo gallery was 750px, so the
* scale factor was always exactly 1.0 and the correction never ran. Verified
* by hand afterwards on a real 4000x3000 upload (stored box moved from
* 1493,204 to 3110,426 — a factor of 2.083, exactly 4000/1920). This test
* exists so that verification does not have to be repeated by hand.
*/
const path = require('path');
const fs = require('fs');
const os = require('os');
process.env.NODE_ENV = 'test';
process.env.TEST_DATABASE_PATH = path.join(
fs.mkdtempSync(path.join(os.tmpdir(), 'picpeak-facescale-')), 'db.sqlite',
);
process.env.JWT_SECRET = process.env.JWT_SECRET || 'facescale-test-secret';
// A 1920x1440 JPEG standing in for the preview rendition. faceProcessor reads
// its dimensions with sharp to derive the scale, so it must be a real image.
const sharp = require('sharp');
let mockPreviewBuffer;
const mockSidecarBox = [1493, 204, 131, 161]; // what the sidecar sees on the preview
jest.mock('../../src/services/imageProcessor', () => ({
...jest.requireActual('../../src/services/imageProcessor'),
ensurePreviewImage: jest.fn(async () => 'previews/preview_test.jpg'),
}));
jest.mock('../../src/services/storage', () => ({
getStorage: () => ({ get: async () => mockPreviewBuffer }),
}));
jest.mock('../../src/services/faceClient', () => ({
detectFaces: jest.fn(async () => ({
model_version: 'test-v1',
faces: [{
bbox: mockSidecarBox,
score: 0.99,
landmarks: [[0, 0], [0, 0], [0, 0], [0, 0], [0, 0]],
yaw: 0, pitch: 0, blur: 500,
embedding: Array.from({ length: 64 }, (_, i) => (i === 0 ? 1 : 0)),
}],
})),
SidecarUnavailableError: class extends Error {},
}));
const { bootCrmDb } = require('./helpers/crmDb');
let db; let cleanup; let faceProcessor;
async function seedPhoto(width, height) {
const [e] = await db('events').insert({
slug: `scale-${width}-${Math.random().toString(36).slice(2, 8)}`,
event_type: 'wedding',
event_name: 'scale',
event_date: '2026-01-01',
host_email: '[email protected]',
admin_email: '[email protected]',
password_hash: 'x',
share_link: `scale-${Math.random()}`,
expires_at: new Date().toISOString(),
face_recognition_enabled: true,
}).returning('id');
const eventId = typeof e === 'object' ? e.id : e;
const [p] = await db('photos').insert({
event_id: eventId,
filename: 'big.jpg',
path: '/tmp/big.jpg',
type: 'individual',
width,
height,
processing_status: 'complete',
face_status: 'processing',
}).returning('id');
return { eventId, photoId: typeof p === 'object' ? p.id : p };
}
describe('face bbox coordinate space (#1074)', () => {
beforeAll(async () => {
mockPreviewBuffer = await sharp({
create: { width: 1920, height: 1440, channels: 3, background: { r: 20, g: 40, b: 80 } },
}).jpeg().toBuffer();
({ db, cleanup } = await bootCrmDb());
// The faces flag gates everything; turn it on for this suite.
await db('feature_flags').insert({ key: 'faces', value: true })
.onConflict('key').merge()
.catch(async () => { await db('feature_flags').where({ key: 'faces' }).update({ value: true }); });
faceProcessor = require('../../src/services/faceProcessor');
}, 120000);
afterAll(async () => { if (cleanup) await cleanup(); });
it('scales preview-space boxes into ORIGINAL image coordinates', async () => {
// 4000px original, 1920px preview -> every coordinate must grow by 4000/1920.
const { photoId } = await seedPhoto(4000, 3000);
await faceProcessor.processPhotoFaces(photoId);
const face = await db('photo_faces').where({ photo_id: photoId }).first();
const scale = 4000 / 1920;
expect(face.bbox_x).toBeCloseTo(mockSidecarBox[0] * scale, 1);
expect(face.bbox_y).toBeCloseTo(mockSidecarBox[1] * scale, 1);
expect(face.bbox_w).toBeCloseTo(mockSidecarBox[2] * scale, 1);
expect(face.bbox_h).toBeCloseTo(mockSidecarBox[3] * scale, 1);
// The regression this guards: the raw preview-space value being stored.
expect(face.bbox_x).not.toBeCloseTo(mockSidecarBox[0], 1);
// And a sanity check that it lands inside the original frame.
expect(face.bbox_x + face.bbox_w).toBeLessThanOrEqual(4000);
});
it('leaves boxes untouched when the photo is already preview-sized', async () => {
// The case that hid the bug: no downscale, so scale is exactly 1 and the
// stored box equals what the sidecar reported.
const { photoId } = await seedPhoto(1920, 1440);
await faceProcessor.processPhotoFaces(photoId);
const face = await db('photo_faces').where({ photo_id: photoId }).first();
expect(face.bbox_x).toBeCloseTo(mockSidecarBox[0], 1);
expect(face.bbox_w).toBeCloseTo(mockSidecarBox[2], 1);
});
it('falls back to unscaled rather than corrupting when width is unknown', async () => {
// Pre-dimension-migration rows have no width. Storing a box scaled by
// NaN/0 would be worse than storing an unscaled one.
const { photoId } = await seedPhoto(null, null);
await faceProcessor.processPhotoFaces(photoId);
const face = await db('photo_faces').where({ photo_id: photoId }).first();
expect(Number.isFinite(face.bbox_x)).toBe(true);
expect(face.bbox_x).toBeCloseTo(mockSidecarBox[0], 1);
});
});
@@ -0,0 +1,166 @@
/**
* Migration 177 (#1074) — face recognition schema.
*
* The acceptance criteria for #1074 name three properties explicitly, so
* they get tests rather than a manual check:
*
* - idempotent on re-run,
* - a working down(),
* - and — the one that matters most — installing it must NOT enqueue
* anything. A `face_status` column defaulting to 'pending' would put
* every existing photo on every install into a queue the operator never
* asked for, on installs with no sidecar at all.
*/
const path = require('path');
const fs = require('fs');
const os = require('os');
process.env.NODE_ENV = 'test';
process.env.TEST_DATABASE_PATH = path.join(
fs.mkdtempSync(path.join(os.tmpdir(), 'picpeak-mig177-')), 'db.sqlite',
);
process.env.JWT_SECRET = process.env.JWT_SECRET || 'mig177-test-secret';
const { bootCrmDb } = require('../integration/helpers/crmDb');
const migration = require('../../migrations/core/177_add_face_recognition');
describe('migration 177 — face recognition schema', () => {
let db; let cleanup;
beforeAll(async () => {
({ db, cleanup } = await bootCrmDb());
}, 120000);
afterAll(async () => { if (cleanup) await cleanup(); });
it('creates both tables with the columns the pipeline writes', async () => {
expect(await db.schema.hasTable('photo_faces')).toBe(true);
expect(await db.schema.hasTable('event_people')).toBe(true);
for (const col of [
'photo_id', 'event_id', 'bbox_x', 'bbox_y', 'bbox_w', 'bbox_h',
'det_score', 'yaw', 'pitch', 'blur', 'embedding', 'model_version',
'person_id', 'created_at',
]) {
expect(await db.schema.hasColumn('photo_faces', col)).toBe(true);
}
for (const col of [
'event_id', 'label', 'cover_face_id', 'centroid', 'face_count_total',
'model_version', 'is_hidden', 'is_ignored',
]) {
expect(await db.schema.hasColumn('event_people', col)).toBe(true);
}
});
it('adds the photos and events columns', async () => {
for (const col of ['face_status', 'face_count', 'face_started_at', 'face_error']) {
expect(await db.schema.hasColumn('photos', col)).toBe(true);
}
for (const col of [
'face_recognition_enabled', 'faces_visible_to_guests', 'faces_last_scan_at',
]) {
expect(await db.schema.hasColumn('events', col)).toBe(true);
}
});
it('enqueues nothing — face_status has no default', async () => {
// The whole "zero behaviour change by default" guarantee rests on this.
const [{ id: eventId }] = await db('events').insert({
slug: 'mig177-event',
event_type: 'wedding',
event_name: 'Migration 177',
event_date: '2026-01-01',
host_email: '[email protected]',
admin_email: '[email protected]',
password_hash: 'x',
share_link: 'mig177-share',
expires_at: new Date().toISOString(),
}).returning('id');
const eid = typeof eventId === 'object' ? eventId.id : eventId;
await db('photos').insert({
event_id: eid, filename: 'a.jpg', path: '/tmp/a.jpg', type: 'individual',
});
const row = await db('photos').where({ event_id: eid }).first();
expect(row.face_status).toBeNull();
expect(await db('photo_faces').count({ c: '*' }).first()).toMatchObject({ c: 0 });
});
it('seeds the tunable thresholds rather than hardcoding them', async () => {
// Immich's clustering guide exists because no single threshold survives
// contact with every library — these must be operator-reachable.
const keys = [
'face_match_threshold', 'face_min_cluster_size',
'face_quality_min_score', 'face_quality_min_px',
];
const rows = await db('app_settings').whereIn('setting_key', keys);
expect(rows).toHaveLength(keys.length);
expect(rows.every((r) => r.setting_type === 'faces')).toBe(true);
});
it('is idempotent on re-run', async () => {
await expect(migration.up(db)).resolves.not.toThrow();
// And did not duplicate the settings rows.
const rows = await db('app_settings').where('setting_key', 'face_match_threshold');
expect(rows).toHaveLength(1);
});
it('down() removes everything it added, and up() restores it', async () => {
await migration.down(db);
expect(await db.schema.hasTable('photo_faces')).toBe(false);
expect(await db.schema.hasTable('event_people')).toBe(false);
expect(await db.schema.hasColumn('photos', 'face_status')).toBe(false);
expect(await db.schema.hasColumn('events', 'face_recognition_enabled')).toBe(false);
expect(await db('app_settings').where('setting_key', 'face_match_threshold')).toHaveLength(0);
await migration.up(db);
expect(await db.schema.hasTable('photo_faces')).toBe(true);
expect(await db.schema.hasColumn('photos', 'face_status')).toBe(true);
});
it('cascades face rows when a photo is deleted', async () => {
// #1074 acceptance criterion: deleting a photo removes its face rows.
//
// SQLite ignores foreign keys unless the pragma is on, and PicPeak does
// NOT enable it globally (a large amount of existing data and fixtures
// would start failing). So the cascade below proves only that the schema
// declares it correctly — the code does not RELY on it. Deletion paths
// purge face rows explicitly; see faceProcessor.purgeEvent /
// purgePhotoFaces and the erasure tests in facePrivacy.test.js.
await db.raw('PRAGMA foreign_keys = ON');
const [{ id: eventId }] = await db('events').insert({
slug: 'mig177-cascade',
event_type: 'wedding',
event_name: 'Cascade',
event_date: '2026-01-01',
host_email: '[email protected]',
admin_email: '[email protected]',
password_hash: 'x',
share_link: 'mig177-cascade-share',
expires_at: new Date().toISOString(),
}).returning('id');
const eid = typeof eventId === 'object' ? eventId.id : eventId;
const [{ id: photoId }] = await db('photos')
.insert({ event_id: eid, filename: 'c.jpg', path: '/tmp/c.jpg', type: 'individual' })
.returning('id');
const pid = typeof photoId === 'object' ? photoId.id : photoId;
await db('photo_faces').insert({
photo_id: pid,
event_id: eid,
bbox_x: 1, bbox_y: 2, bbox_w: 3, bbox_h: 4,
model_version: 'test',
created_at: new Date().toISOString(),
});
expect(await db('photo_faces').where({ photo_id: pid })).toHaveLength(1);
await db('photos').where({ id: pid }).del();
expect(await db('photo_faces').where({ photo_id: pid })).toHaveLength(0);
});
});
@@ -0,0 +1,268 @@
/**
* Migration 177: Face recognition — "People in this gallery" (#1074).
*
* Adds two tables and four columns for per-gallery face detection and
* clustering. People are clustered INSIDE one event; there is deliberately no
* cross-event person identity, so `event_people` hangs off `events` and every
* query is naturally scoped to one gallery.
*
* Three decisions worth knowing before reading the schema:
*
* 1. Embeddings are a plain binary column, not a vector type. 512 float32 =
* 2KB per face, and a 2,000-photo wedding at ~3 faces/photo is ~12MB —
* small enough that no pgvector extension is needed and the SQLite path
* behaves identically. Clustering runs in JS over hundreds to low
* thousands of rows per event.
*
* 2. `photos.face_status` is NULLABLE with no default, and this migration
* enqueues NOTHING. A default of 'pending' would silently queue every
* photo on every install the moment this ran — including installs with
* no sidecar and no intention of using the feature. Rows are enqueued
* only by an explicit per-event opt-in.
*
* 3. `event_people.face_count_total` is named for what it is: a count over
* ALL faces, including photos a guest cannot see. Guests are restricted
* to `photos.visibility = 'visible'` (gallery.js), so a guest-facing
* count and cover face MUST be computed per request against that same
* predicate. The name is a deterrent: anything reading
* `face_count_total` in a guest path is a bug.
*
* Face embeddings are biometric data — GDPR Art. 9 special category. Both
* tables are excluded from backups and from .picpeak exports (see
* picpeakExportService and databaseBackup): the data is derived, so a restore
* re-scans rather than carrying biometrics between deployments.
*/
const GLOBAL_DEFAULTS = [
// Cosine similarity above which a face joins an existing person.
//
// 0.60, measured twice — and the second measurement overruled the first,
// which is the whole reason this comment is long.
//
// LFW's standard 1000-pair protocol through this exact pipeline gives
// same-person cosine 0.696 +/- 0.142, different-person 0.085 +/- 0.167,
// peak accuracy 96.6% at 0.405. Reading that as a clustering threshold
// suggests ~0.50 (1.0% false merge, 8.2% false split pairwise).
//
// Running it on an actual gallery says otherwise. At 0.50, three of six
// visible clusters were contaminated — two different people merged into
// one strip entry. PAIRWISE ERROR RATES DO NOT PREDICT CLUSTER PURITY:
// greedy assignment compounds, because one wrong face drags the centroid
// toward the midpoint between two identities, which makes the next wrong
// face likelier. A 1% pairwise false-merge rate is not a 1% chance of a
// clean gallery.
//
// Sweep on a 61-photo / 5-identity gallery (ml/tools/benchmark_threshold.py
// covers the pairwise half; the cluster half is the `recluster` endpoint):
// 0.50 -> 6 clusters, 3 contaminated
// 0.56 -> 6 clusters, 0 contaminated
// 0.60 -> 5 clusters, 0 contaminated <- ground truth is 5
// 0.64 -> 5 clusters, 0 contaminated, fewer faces assigned
//
// 0.60 recovers the exact right number of people with no contamination.
// Higher only drops coverage. A false merge puts a stranger into someone's
// "download my photos" and cannot be undone until the Phase 2 merge/split
// UI ships, so contamination is the constraint being optimised against,
// not accuracy.
['face_match_threshold', 0.60],
// Faces a cluster needs before it appears in the guest-facing strip. Keeps
// one-off bystanders out of "People in this gallery".
['face_min_cluster_size', 3],
// Quality floor. Faces below any of these are still stored (so "this photo
// contains" stays accurate) but are left unassigned, so they cannot spawn
// junk people.
['face_quality_min_score', 0.7],
['face_quality_min_px', 40],
// Phase 3 rule engine. Present here so the settings block has one home.
['face_auto_categorize_enabled', false],
];
exports.up = async function (knex) {
// --- event_people -------------------------------------------------------
// Created before photo_faces: photo_faces.person_id references it. The
// reverse direction (event_people.cover_face_id → photo_faces.id) is a
// forward reference and is deliberately left WITHOUT a foreign key —
// Postgres rejects an FK to a table that doesn't exist yet, and adding it
// afterwards buys nothing here because the column is nullable and the
// clustering code always writes an id it just inserted. Migration 001 hit
// exactly this with events.hero_photo_id (#484).
if (!(await knex.schema.hasTable('event_people'))) {
await knex.schema.createTable('event_people', (table) => {
table.increments('id').primary();
table.integer('event_id').unsigned().notNullable()
.references('id').inTable('events').onDelete('CASCADE');
// NULL until a photographer names them. The UI shows a photo count
// instead — a number is honest, an invented "Person 7" is not.
table.string('label', 255);
table.integer('cover_face_id');
// Running-mean centroid of the cluster's embeddings, 512 float32.
table.binary('centroid');
// Count over ALL faces — see note 3 in the header. NEVER guest-facing.
table.integer('face_count_total').notNullable().defaultTo(0);
table.string('model_version', 64);
// Photographer-only: hidden people never reach a guest response.
table.boolean('is_hidden').notNullable().defaultTo(false);
// Bystanders and false positives — excluded from the strip entirely.
table.boolean('is_ignored').notNullable().defaultTo(false);
table.timestamp('created_at').defaultTo(knex.fn.now());
table.timestamp('updated_at').defaultTo(knex.fn.now());
table.index(['event_id'], 'event_people_event_idx');
});
}
// --- photo_faces --------------------------------------------------------
if (!(await knex.schema.hasTable('photo_faces'))) {
await knex.schema.createTable('photo_faces', (table) => {
table.increments('id').primary();
table.integer('photo_id').unsigned().notNullable()
.references('id').inTable('photos').onDelete('CASCADE');
// Denormalized from photos.event_id. Every clustering and people query
// is per-event; without this they would all join through photos.
table.integer('event_id').unsigned().notNullable()
.references('id').inTable('events').onDelete('CASCADE');
// Bounding box in ORIGINAL image pixels (the sidecar reports original
// coordinates even though it detects on a downscaled copy), so cover
// avatars can be cropped from any rendition by ratio.
table.float('bbox_x').notNullable();
table.float('bbox_y').notNullable();
table.float('bbox_w').notNullable();
table.float('bbox_h').notNullable();
table.float('det_score');
table.float('yaw');
table.float('pitch');
table.float('blur');
// 512 × float32, L2-normalized. See note 1.
table.binary('embedding');
// Which detector+embedder+alignment produced this. Embeddings from two
// pipelines are not comparable, so a model change re-derives rather
// than silently mixing two spaces.
table.string('model_version', 64);
// SET NULL rather than CASCADE: deleting a person must un-assign its
// faces, never delete the detections themselves.
table.integer('person_id').unsigned()
.references('id').inTable('event_people').onDelete('SET NULL');
table.timestamp('created_at').defaultTo(knex.fn.now());
table.index(['photo_id'], 'photo_faces_photo_idx');
table.index(['event_id', 'person_id'], 'photo_faces_event_person_idx');
});
}
// --- photos -------------------------------------------------------------
//
// Every column added in ONE alterTable, not one call each. This migration
// replays in ~90 test suites that boot a database from scratch, and each
// ALTER is its own statement and its own fsync — the per-column version of
// this was measurably slower on CI's shared disk than the whole rest of the
// chain. Same reason the settings seed below is one round trip.
if (await knex.schema.hasTable('photos')) {
const wanted = {
// NULL = never queued. See note 2 — this migration enqueues nothing.
// pending | processing | done | failed | skipped
face_status: (t) => t.string('face_status', 16),
face_count: (t) => t.integer('face_count'),
// Set when a row is claimed, so the janitor can recover it after a
// worker dies mid-photo. Mirrors photos.processing_started_at.
face_started_at: (t) => t.timestamp('face_started_at'),
face_error: (t) => t.text('face_error'),
};
const missing = [];
for (const name of Object.keys(wanted)) {
if (!(await knex.schema.hasColumn('photos', name))) missing.push(name);
}
if (missing.length) {
await knex.schema.alterTable('photos', (table) => {
for (const name of missing) wanted[name](table);
// Index for the queue claim, added in the same statement. The worker
// polls face_status='pending' constantly; without it that is a full
// scan of `photos` on every tick.
if (missing.includes('face_status')) {
table.index(['face_status'], 'photos_face_status_idx');
}
});
}
}
// --- events -------------------------------------------------------------
if (await knex.schema.hasTable('events')) {
const wanted = {
// Per-event opt-in. NULL and false both mean off; the column is
// nullable only so an existing row doesn't need backfilling.
face_recognition_enabled: (t) => t.boolean('face_recognition_enabled'),
// When detection is on, does the GUEST see the people strip? Off means
// the photographer gets the tool and guests see an unchanged gallery.
// Defaults to on, but only matters once the above is enabled.
faces_visible_to_guests: (t) => t.boolean('faces_visible_to_guests'),
faces_last_scan_at: (t) => t.timestamp('faces_last_scan_at'),
};
const missing = [];
for (const name of Object.keys(wanted)) {
if (!(await knex.schema.hasColumn('events', name))) missing.push(name);
}
if (missing.length) {
await knex.schema.alterTable('events', (table) => {
for (const name of missing) wanted[name](table);
});
}
}
// --- app_settings -------------------------------------------------------
if (!(await knex.schema.hasTable('app_settings'))) return;
// One SELECT and at most one INSERT, rather than a SELECT+INSERT per key.
const keys = GLOBAL_DEFAULTS.map(([k]) => k);
const present = new Set(
(await knex('app_settings').whereIn('setting_key', keys).select('setting_key'))
.map((r) => r.setting_key)
);
const rows = GLOBAL_DEFAULTS
.filter(([key]) => !present.has(key))
.map(([key, value]) => ({
setting_key: key,
// JSON-stringified so SQLite (TEXT) and Postgres (JSONB) round-trip
// the same shape — matches migrations 104 and 173.
setting_value: JSON.stringify(value),
setting_type: 'faces',
// ISO string, not a Date: under Jest, Date objects handed to the
// sqlite3 binding store as the literal "[object Object]".
updated_at: new Date().toISOString(),
}));
if (rows.length) await knex('app_settings').insert(rows);
};
exports.down = async function (knex) {
// photo_faces first — it holds the FK into event_people.
await knex.schema.dropTableIfExists('photo_faces');
await knex.schema.dropTableIfExists('event_people');
if (await knex.schema.hasTable('photos')) {
for (const name of ['face_status', 'face_count', 'face_started_at', 'face_error']) {
if (await knex.schema.hasColumn('photos', name)) {
await knex.schema.alterTable('photos', (table) => table.dropColumn(name));
}
}
}
if (await knex.schema.hasTable('events')) {
for (const name of [
'face_recognition_enabled',
'faces_visible_to_guests',
'faces_last_scan_at',
]) {
if (await knex.schema.hasColumn('events', name)) {
await knex.schema.alterTable('events', (table) => table.dropColumn(name));
}
}
}
if (await knex.schema.hasTable('app_settings')) {
await knex('app_settings')
.whereIn('setting_key', GLOBAL_DEFAULTS.map(([k]) => k))
.del();
}
};
@@ -0,0 +1,41 @@
/**
* Migration 178: auto-categorised flag (#1074 phase 3).
*
* Marks photos whose category was assigned by the face-count rule engine
* rather than by a person. That distinction is what makes "undo all automatic
* categories" a single query instead of an archaeology exercise, and it is
* why the engine can promise never to touch a photographer's own choices.
*
* Separate from 177 rather than folded into it: 177 has already run wherever
* this branch has been deployed, so editing it would be a no-op there and a
* silent schema divergence between installs.
*
* Nullable with no default. NULL and false both mean "a human chose this (or
* nothing did)", which is the safe reading for every pre-existing row —
* backfilling `false` would say the same thing at the cost of rewriting the
* whole table.
*/
exports.up = async function (knex) {
if (!(await knex.schema.hasTable('photos'))) return;
if (await knex.schema.hasColumn('photos', 'auto_categorized')) return;
// Column and index in ONE statement — this replays in ~90 test suites that
// build a database from scratch, and each separate ALTER costs a statement
// and an fsync there. See the note in 177.
await knex.schema.alterTable('photos', (table) => {
table.boolean('auto_categorized');
// The undo path filters on this alone across a whole event, so it is
// worth an index on installs where most photos are NOT auto-categorised.
table.index(['auto_categorized'], 'photos_auto_categorized_idx');
});
};
exports.down = async function (knex) {
if (!(await knex.schema.hasTable('photos'))) return;
if (!(await knex.schema.hasColumn('photos', 'auto_categorized'))) return;
await knex.schema.alterTable('photos', (table) => {
table.dropColumn('auto_categorized');
});
};
+11
View File
@@ -1186,6 +1186,17 @@ async function startServer() {
// sharp/ffmpeg/EXIF pipeline off the request thread. // sharp/ffmpeg/EXIF pipeline off the request thread.
backgroundProcessor.start(); backgroundProcessor.start();
// Face detection (#1074). Starts alongside the photo processor but stays
// idle — every worker tick re-checks the `faces` feature flag, which is
// off by default. It is safe to start unconditionally precisely because
// it never touches FACE_ML_URL until that flag is on.
//
// Required HERE rather than at module scope: the face stack pulls in
// axios and (via imageProcessor) sharp, and server.js is imported by a
// large number of supertest suites that never start a worker. Keeping it
// lazy means they don't pay for a module graph they never use.
require('./src/services/faceQueue').start();
app.listen(PORT, () => { app.listen(PORT, () => {
logger.info(`Server running on port ${PORT}`); logger.info(`Server running on port ${PORT}`);
logger.info(`Admin interface: ${process.env.ADMIN_URL || 'http://localhost:3000'}`); logger.info(`Admin interface: ${process.env.ADMIN_URL || 'http://localhost:3000'}`);
+475
View File
@@ -0,0 +1,475 @@
// Per-event face recognition — "People in this gallery" (#1074). Same
// sub-router shape as ./downloadResolutions.js — see ./index.js for the
// registration-order contract.
//
// Every route here is gated on the `faces` feature flag as well as the usual
// auth/ownership chain. The frontend hides these surfaces when the flag is
// off, but a direct API call must be refused too — this feature touches
// biometric data, so "the UI doesn't show it" is not a control.
const { body, validationResult } = require('express-validator');
const { db, logActivity } = require('../../database/db');
const { adminAuth } = require('../../middleware/auth');
const { requirePermission } = require('../../middleware/permissions');
const { requireFeatureFlag } = require('../../middleware/requireFeatureFlag');
const { requireEventOwnership } = require('../../middleware/ownership');
const { errorResponse } = require('../../utils/routeHelpers');
const { parseBooleanInput } = require('../../utils/parsers');
const logger = require('../../utils/logger');
const faceSettings = require('../../services/faceSettings');
const faceProcessor = require('../../services/faceProcessor');
const faceClustering = require('../../services/faceClustering');
const facePeopleService = require('../../services/facePeopleService');
const faceClient = require('../../services/faceClient');
const requireFaces = requireFeatureFlag('faces', 'FACES_DISABLED');
async function loadOwnedEvent(req) {
let q = db('events').where('id', req.params.id);
if (req.admin.roleName === 'editor') {
q = q.where('created_by', req.admin.id);
}
return q.first();
}
module.exports = (router) => {
/**
* Sidecar connection test for the global settings panel.
*
* Registered FIRST because it is the only literal-prefix route in this
* module — see ./index.js on why registration order is load-bearing. It
* does not currently collide with '/:id/faces' (the second segment differs)
* but relying on that would be one refactor away from a silent 404.
*/
router.get('/faces/health', adminAuth, requirePermission('events.view'), requireFaces, async (req, res) => {
try {
const result = await faceClient.checkHealth();
res.json({ url: faceSettings.getSidecarUrl(), ...result });
} catch (error) {
errorResponse(res, error, 500, 'Failed to check sidecar health');
}
});
/**
* Settings + scan status for the Event → Settings → People panel.
*/
router.get('/:id/faces', adminAuth, requirePermission('events.view'), requireFaces, requireEventOwnership,
async (req, res) => {
try {
const event = await loadOwnedEvent(req);
if (!event) return res.status(404).json({ error: 'Event not found' });
const status = await facePeopleService.getScanStatus(event.id);
res.json({
enabled: event.face_recognition_enabled === true || event.face_recognition_enabled === 1,
visible_to_guests: faceSettings.areFacesVisibleToGuests(event),
last_scan_at: event.faces_last_scan_at || null,
status,
});
} catch (error) {
errorResponse(res, error, 500, 'Failed to load face settings');
}
});
/**
* Flip the two per-event toggles.
*
* Enabling detection backfills existing photos. Disabling it stops there —
* it does NOT delete anything, because "stop scanning" and "destroy the
* clusters I already named" are different intentions and the destructive
* one gets its own explicit endpoint.
*/
router.patch('/:id/faces',
adminAuth,
requirePermission('events.edit'),
requireFaces,
requireEventOwnership,
[
body('enabled').optional().isBoolean(),
body('visible_to_guests').optional().isBoolean(),
],
async (req, res) => {
const errors = validationResult(req);
if (!errors.isEmpty()) return res.status(400).json({ errors: errors.array() });
try {
const event = await loadOwnedEvent(req);
if (!event) return res.status(404).json({ error: 'Event not found' });
const update = {};
const wasEnabled = event.face_recognition_enabled === true || event.face_recognition_enabled === 1;
let nowEnabled = wasEnabled;
if (req.body.enabled !== undefined) {
nowEnabled = parseBooleanInput(req.body.enabled, false);
update.face_recognition_enabled = nowEnabled;
}
if (req.body.visible_to_guests !== undefined) {
update.faces_visible_to_guests = parseBooleanInput(req.body.visible_to_guests, true);
}
if (!Object.keys(update).length) {
return res.status(400).json({ error: 'Nothing to update' });
}
await db('events').where({ id: event.id }).update(update);
let queued = 0;
if (!wasEnabled && nowEnabled) {
queued = await faceProcessor.enqueueEvent(event.id, { onlyUnscanned: true });
await db('events').where({ id: event.id })
.update({ faces_last_scan_at: new Date().toISOString() });
}
await logActivity('event_faces_updated', {
actorType: 'admin',
actorId: req.admin.id,
eventId: event.id,
metadata: { ...update, queued },
}).catch(() => {});
res.json({ success: true, queued });
} catch (error) {
errorResponse(res, error, 500, 'Failed to update face settings');
}
});
/**
* People management grid. Unlike the gallery endpoint this returns hidden
* and ignored people, plus the total (all-visibility) face count — the
* photographer is allowed to see their own hidden photos.
*/
router.get('/:id/people', adminAuth, requirePermission('events.view'), requireFaces, requireEventOwnership,
async (req, res) => {
try {
const event = await loadOwnedEvent(req);
if (!event) return res.status(404).json({ error: 'Event not found' });
const people = await facePeopleService.listPeople(event.id, {
isClient: true,
forAdmin: true,
});
res.json({ people });
} catch (error) {
errorResponse(res, error, 500, 'Failed to fetch people');
}
});
/**
* Rename / hide / ignore / set cover.
*/
router.patch('/:id/people/:personId',
adminAuth,
requirePermission('events.edit'),
requireFaces,
requireEventOwnership,
[
body('label').optional({ nullable: true }).isString().isLength({ max: 255 }),
body('is_hidden').optional().isBoolean(),
body('is_ignored').optional().isBoolean(),
body('cover_face_id').optional({ nullable: true }).isInt(),
],
async (req, res) => {
const errors = validationResult(req);
if (!errors.isEmpty()) return res.status(400).json({ errors: errors.array() });
try {
const event = await loadOwnedEvent(req);
if (!event) return res.status(404).json({ error: 'Event not found' });
const person = await db('event_people')
.where({ id: req.params.personId, event_id: event.id }).first();
if (!person) return res.status(404).json({ error: 'Person not found' });
const update = { updated_at: new Date().toISOString() };
if (req.body.label !== undefined) {
const label = req.body.label === null ? null : String(req.body.label).trim();
update.label = label || null;
}
if (req.body.is_hidden !== undefined) update.is_hidden = parseBooleanInput(req.body.is_hidden, false);
if (req.body.is_ignored !== undefined) update.is_ignored = parseBooleanInput(req.body.is_ignored, false);
if (req.body.cover_face_id !== undefined && req.body.cover_face_id !== null) {
// The cover must belong to THIS person in THIS event — otherwise a
// crafted id could point the avatar at any face in the database.
const face = await db('photo_faces')
.where({ id: req.body.cover_face_id, event_id: event.id, person_id: person.id })
.first();
if (!face) return res.status(400).json({ error: 'cover_face_id does not belong to this person' });
update.cover_face_id = face.id;
}
await db('event_people').where({ id: person.id }).update(update);
res.json({ success: true });
} catch (error) {
errorResponse(res, error, 500, 'Failed to update person');
}
});
/**
* Merge people. The two operations that make automatic clustering survive
* contact with reality are this and split — a well-behaved wedding gallery
* still produces "Anna in daylight" and "Anna at the party".
*/
router.post('/:id/people/merge',
adminAuth,
requirePermission('events.edit'),
requireFaces,
requireEventOwnership,
[body('source_ids').isArray({ min: 1 }), body('target_id').isInt()],
async (req, res) => {
const errors = validationResult(req);
if (!errors.isEmpty()) return res.status(400).json({ errors: errors.array() });
try {
const event = await loadOwnedEvent(req);
if (!event) return res.status(404).json({ error: 'Event not found' });
// Every id must belong to this event — never merge across galleries.
const ids = [...req.body.source_ids.map(Number), Number(req.body.target_id)];
const owned = await db('event_people').where({ event_id: event.id }).whereIn('id', ids).pluck('id');
if (owned.length !== new Set(ids).size) {
return res.status(400).json({ error: 'One or more people do not belong to this event' });
}
const result = await faceClustering.mergePeople(
event.id, req.body.source_ids.map(Number), Number(req.body.target_id)
);
res.json({ success: true, ...result });
} catch (error) {
errorResponse(res, error, 500, 'Failed to merge people');
}
});
router.post('/:id/people/:personId/split',
adminAuth,
requirePermission('events.edit'),
requireFaces,
requireEventOwnership,
[body('face_ids').isArray({ min: 1 })],
async (req, res) => {
const errors = validationResult(req);
if (!errors.isEmpty()) return res.status(400).json({ errors: errors.array() });
try {
const event = await loadOwnedEvent(req);
if (!event) return res.status(404).json({ error: 'Event not found' });
const newPersonId = await faceClustering.splitPerson(
event.id, Number(req.params.personId), req.body.face_ids.map(Number)
);
if (!newPersonId) return res.status(400).json({ error: 'No matching faces to split' });
res.json({ success: true, person_id: newPersonId });
} catch (error) {
errorResponse(res, error, 500, 'Failed to split person');
}
});
/**
* Faces of one person, for the split picker.
*/
router.get('/:id/people/:personId/faces', adminAuth, requirePermission('events.view'), requireFaces, requireEventOwnership,
async (req, res) => {
try {
const event = await loadOwnedEvent(req);
if (!event) return res.status(404).json({ error: 'Event not found' });
const faces = await db('photo_faces')
.where({ event_id: event.id, person_id: req.params.personId })
.orderBy('det_score', 'desc')
.limit(500)
.join('photos', 'photos.id', 'photo_faces.photo_id')
.select(
'photo_faces.id', 'photo_faces.photo_id',
'photo_faces.bbox_x', 'photo_faces.bbox_y',
'photo_faces.bbox_w', 'photo_faces.bbox_h',
'photo_faces.det_score', 'photo_faces.blur',
// Needed to crop the box — it is in original-image pixels.
'photos.width as photo_width', 'photos.height as photo_height'
);
res.json({
faces: faces.map((f) => ({
id: f.id,
photo_id: f.photo_id,
bbox: [f.bbox_x, f.bbox_y, f.bbox_w, f.bbox_h],
photo_width: f.photo_width ?? null,
photo_height: f.photo_height ?? null,
score: f.det_score,
blur: f.blur,
})),
});
} catch (error) {
errorResponse(res, error, 500, 'Failed to fetch faces');
}
});
/**
* Re-scan: re-runs detection through the sidecar. Expensive.
*/
router.post('/:id/faces/rescan', adminAuth, requirePermission('events.edit'), requireFaces, requireEventOwnership,
async (req, res) => {
try {
const event = await loadOwnedEvent(req);
if (!event) return res.status(404).json({ error: 'Event not found' });
if (!(await faceSettings.isEnabledForEvent(event))) {
return res.status(400).json({ error: 'Face detection is not enabled for this event' });
}
const queued = await faceProcessor.enqueueEvent(event.id, { onlyUnscanned: false });
await db('events').where({ id: event.id })
.update({ faces_last_scan_at: new Date().toISOString() });
await logActivity('event_faces_rescan', {
actorType: 'admin', actorId: req.admin.id, eventId: event.id, metadata: { queued },
}).catch(() => {});
res.json({ success: true, queued });
} catch (error) {
errorResponse(res, error, 500, 'Failed to queue re-scan');
}
});
/**
* Re-cluster: re-derives people from stored embeddings. Cheap — no sidecar
* call — and the thing to run after changing the match threshold.
*/
router.post('/:id/faces/recluster', adminAuth, requirePermission('events.edit'), requireFaces, requireEventOwnership,
async (req, res) => {
try {
const event = await loadOwnedEvent(req);
if (!event) return res.status(404).json({ error: 'Event not found' });
const people = await faceClustering.recluster(event.id);
await logActivity('event_faces_recluster', {
actorType: 'admin', actorId: req.admin.id, eventId: event.id, metadata: { people },
}).catch(() => {});
res.json({ success: true, people });
} catch (error) {
errorResponse(res, error, 500, 'Failed to recluster');
}
});
/**
* Read/write the global auto-category setting.
*
* The migration seeds it false and nothing else ever wrote it, so the whole
* rule engine — and its undo endpoint — were unreachable in normal product
* flows: every call returned `skipped: disabled`. A feature with no way to
* turn it on is not shipped.
*/
router.get('/faces/auto-categories', adminAuth, requirePermission('settings.view'), requireFaces,
async (req, res) => {
try {
const thresholds = await faceSettings.getThresholds();
res.json({ enabled: thresholds.face_auto_categorize_enabled === true });
} catch (error) {
errorResponse(res, error, 500, 'Failed to read the auto-category setting');
}
});
router.put('/faces/auto-categories',
adminAuth,
requirePermission('settings.edit'),
requireFaces,
[body('enabled').isBoolean()],
async (req, res) => {
const errors = validationResult(req);
if (!errors.isEmpty()) return res.status(400).json({ errors: errors.array() });
try {
const enabled = parseBooleanInput(req.body.enabled, false);
const existing = await db('app_settings')
.where('setting_key', 'face_auto_categorize_enabled').first();
if (existing) {
await db('app_settings').where('setting_key', 'face_auto_categorize_enabled')
.update({ setting_value: JSON.stringify(enabled), updated_at: new Date().toISOString() });
} else {
await db('app_settings').insert({
setting_key: 'face_auto_categorize_enabled',
setting_value: JSON.stringify(enabled),
setting_type: 'faces',
updated_at: new Date().toISOString(),
});
}
await logActivity('face_auto_categories_toggled', {
actorType: 'admin', actorId: req.admin.id, metadata: { enabled },
}).catch(() => {});
res.json({ success: true, enabled });
} catch (error) {
errorResponse(res, error, 500, 'Failed to update the auto-category setting');
}
});
/**
* Run the auto-category rules over this event now.
*
* Normally happens automatically after each photo is scanned; this is for
* applying the rules to a gallery that was scanned before the setting was
* turned on.
*/
router.post('/:id/faces/categorize', adminAuth, requirePermission('events.edit'), requireFaces, requireEventOwnership,
async (req, res) => {
try {
const event = await loadOwnedEvent(req);
if (!event) return res.status(404).json({ error: 'Event not found' });
const { categorizeEvent } = require('../../services/faceAutoCategories');
const result = await categorizeEvent(event.id);
res.json({ success: true, ...result });
} catch (error) {
errorResponse(res, error, 500, 'Failed to apply automatic categories');
}
});
/**
* Undo every automatic category assignment. Only clears what the rule
* engine set — categories chosen by a person are untouched.
*/
router.delete('/:id/faces/categorize', adminAuth, requirePermission('events.edit'), requireFaces, requireEventOwnership,
async (req, res) => {
try {
const event = await loadOwnedEvent(req);
if (!event) return res.status(404).json({ error: 'Event not found' });
const { undoEvent } = require('../../services/faceAutoCategories');
const result = await undoEvent(event.id);
await logActivity('event_faces_categories_undone', {
actorType: 'admin', actorId: req.admin.id, eventId: event.id, metadata: result,
}).catch(() => {});
res.json({ success: true, ...result });
} catch (error) {
errorResponse(res, error, 500, 'Failed to undo automatic categories');
}
});
/**
* Delete all face data for this event. The GDPR erasure path.
*/
router.delete('/:id/faces', adminAuth, requirePermission('events.edit'), requireFaces, requireEventOwnership,
async (req, res) => {
try {
const event = await loadOwnedEvent(req);
if (!event) return res.status(404).json({ error: 'Event not found' });
const result = await faceProcessor.purgeEvent(event.id);
await db('events').where({ id: event.id }).update({ faces_last_scan_at: null });
await logActivity('event_faces_purged', {
actorType: 'admin', actorId: req.admin.id, eventId: event.id, metadata: result,
}).catch(() => {});
logger.info(`adminEvents: face data purged for event ${event.id} by admin ${req.admin.id}`);
res.json({ success: true, ...result });
} catch (error) {
errorResponse(res, error, 500, 'Failed to delete face data');
}
});
};
@@ -232,6 +232,14 @@ async function deleteEventCascade(eventId, adminContext) {
// 3. Delete email queue entries // 3. Delete email queue entries
await trx('email_queue').where('event_id', eventId).del(); await trx('email_queue').where('event_id', eventId).del();
// 4. Delete photos (also handles hero_photo_id foreign key) // 4. Delete photos (also handles hero_photo_id foreign key)
// Face data (#1074). The FK declares ON DELETE CASCADE, but SQLite only
// honours that when `PRAGMA foreign_keys = ON`, which PicPeak does not set
// — so on the SQLite path the cascade is inert and biometric embeddings
// would outlive the gallery they belong to. Delete explicitly, before the
// photos, so the guarantee holds on both engines.
await trx('photo_faces').where('event_id', eventId).del();
await trx('event_people').where('event_id', eventId).del();
await trx('photos').where('event_id', eventId).del(); await trx('photos').where('event_id', eventId).del();
// 5. Finally delete the event row // 5. Finally delete the event row
await trx('events').where('id', eventId).del(); await trx('events').where('id', eventId).del();
+1
View File
@@ -15,5 +15,6 @@ require('./resets')(router);
require('./archiveBulk')(router); require('./archiveBulk')(router);
require('./logo')(router); require('./logo')(router);
require('./qr')(router); require('./qr')(router);
require('./faces')(router);
module.exports = router; module.exports = router;
+23
View File
@@ -98,6 +98,18 @@ const KNOWN_FLAGS = [
// kill-switch for the Workflows admin area AND the engine's runtime side // kill-switch for the Workflows admin area AND the engine's runtime side
// effects (no run is created/resumed while off). // effects (no run is created/resumed while off).
'workflows', 'workflows',
// Face recognition — "People in this gallery" (migration 177, #1074).
// Requires the optional picpeak-ml sidecar container. THIS FLAG IS THE
// GATE for the whole feature: FACE_ML_URL has a working default (the
// compose service name), so the variable's presence proves nothing and
// cannot be used to detect intent. While this is off the backend never
// contacts the sidecar, the face queue idles, no face UI renders anywhere
// and no face_status is ever written.
//
// Face embeddings are biometric data (GDPR Art. 9). Turning this on is only
// the first of two deliberate actions — detection still has to be enabled
// per event. Strictly opt-in.
'faces',
]; ];
// Spec defaults for any flag missing from the DB (e.g. a row added by a // Spec defaults for any flag missing from the DB (e.g. a row added by a
@@ -129,6 +141,8 @@ const DEFAULT_FLAGS = {
slideshow: false, slideshow: false,
transfers: false, transfers: false,
workflows: false, workflows: false,
// #1074 — off by default is the whole "zero behaviour change" guarantee.
faces: false,
}; };
async function readAllFlags() { async function readAllFlags() {
@@ -146,6 +160,15 @@ function applyDependencyRules(flags) {
const out = { ...flags }; const out = { ...flags };
// Galleries is the foundation — never off. // Galleries is the foundation — never off.
out.galleries = true; out.galleries = true;
// Face recognition is unavailable on the all-in-one single-container image
// (#1042 / PR #1068) for performance reasons — see
// faceSettings.isSingleContainerImage. Forced false in BOTH directions:
// GET reports it off so the UI can show it as unavailable rather than a
// switch that silently does nothing, and PUT cannot turn it on. The backend
// gate refuses independently, so this is presentation plus defence in
// depth, not the enforcement itself.
const { isSingleContainerImage } = require('../services/faceSettings');
if (isSingleContainerImage()) out.faces = false;
// Sub-features can't outlive their parents. // Sub-features can't outlive their parents.
if (out.quotes === false) out.bills = false; if (out.quotes === false) out.bills = false;
if (out.calendar === false) out.calendarBooking = false; if (out.calendar === false) out.calendarBooking = false;
+38
View File
@@ -684,6 +684,15 @@ router.delete('/:eventId/photos/:photoId', adminAuth, requirePermission('photos.
} }
// Remove from database // Remove from database
// Face data (#1074): the FK cascade is inert on SQLite, and cannot fix up
// event_people counts anyway. See faceProcessor.purgePhotoFaces.
try {
const { purgePhotoFaces } = require('../services/faceProcessor');
await purgePhotoFaces(photoId);
} catch (err) {
logger.warn(`deletePhoto: face purge failed for photo ${photoId}`, { error: err.message });
}
await db('photos').where({ id: photoId }).delete(); await db('photos').where({ id: photoId }).delete();
// Log activity (event was fetched above for storage key resolution) // Log activity (event was fetched above for storage key resolution)
@@ -752,6 +761,12 @@ router.patch('/:eventId/photos/:photoId', adminAuth, requirePermission('photos.e
} }
} }
// A human just set (or cleared) this category, so it is no longer an
// automatic assignment (#1074 phase 3). Without resetting the flag,
// "undo automatic categories" would later wipe the photographer's own
// choice — exactly the guarantee the rule engine advertises.
updateData.auto_categorized = false;
// Update photo // Update photo
await db('photos') await db('photos')
.where({ id: photoId, event_id: eventId }) .where({ id: photoId, event_id: eventId })
@@ -828,6 +843,23 @@ router.post('/:eventId/photos/bulk-delete', adminAuth, requirePermission('photos
} }
} }
// Face data (#1074), bulk path. Same reasoning as the single delete: the
// SQLite FK cascade never fires, and event_people counts need rebuilding
// regardless of engine.
// Iterate the VALIDATED rows, not the raw request ids. `photos` is already
// scoped to this event; `photoIds` is user input, and purgePhotoFaces has
// no event scope of its own — so looping the raw ids let an editor delete
// face data (and recompute people) in a gallery they do not own, even
// though the photo deletion below is correctly scoped.
for (const photo of photos) {
try {
const { purgePhotoFaces } = require('../services/faceProcessor');
await purgePhotoFaces(photo.id);
} catch (err) {
logger.warn(`bulk delete: face purge failed for photo ${photo.id}`, { error: err.message });
}
}
// Delete from database // Delete from database
await db('photos') await db('photos')
.whereIn('id', photoIds) .whereIn('id', photoIds)
@@ -908,6 +940,12 @@ router.post('/:eventId/photos/bulk-update', adminAuth, requirePermission('photos
updateData.category_id = null; updateData.category_id = null;
} }
} }
// A human just set (or cleared) this category, so it is no longer an
// automatic assignment (#1074 phase 3). Without resetting the flag,
// "undo automatic categories" would later wipe the photographer's own
// choice — exactly the guarantee the rule engine advertises.
updateData.auto_categorized = false;
} }
await db('photos') await db('photos')
+9 -1
View File
@@ -35,12 +35,20 @@ router.get('/version', adminAuth, requirePermission(['settings.view', 'system.vi
const channel = getCurrentChannel(backendVersion); const channel = getCurrentChannel(backendVersion);
// `single_container` lets the admin UI explain why a feature is
// unavailable instead of rendering a switch that silently refuses to
// stay on. Currently only face recognition is gated this way (#1074 on
// the all-in-one image, #1042 / PR #1068) — see
// faceSettings.isSingleContainerImage for the reasoning.
const { isSingleContainerImage } = require('../services/faceSettings');
res.json({ res.json({
backend: backendVersion, backend: backendVersion,
frontend: '1.0.0', // This will be set by frontend frontend: '1.0.0', // This will be set by frontend
node: process.version, node: process.version,
environment: process.env.NODE_ENV || 'production', environment: process.env.NODE_ENV || 'production',
channel: channel channel: channel,
single_container: isSingleContainerImage()
}); });
} catch (error) { } catch (error) {
logger.error('Error fetching version:', error); logger.error('Error fetching version:', error);
+96
View File
@@ -810,6 +810,35 @@ router.get('/:slug/photos', verifyGalleryAccess, resolveGuest, async (req, res)
likedRows.forEach(row => likedPhotoIds.add(row.photo_id)); likedRows.forEach(row => likedPhotoIds.add(row.photo_id));
} }
// People in each photo (#1074). Two independent gates: the feature must
// be on for this event AND, for a plain guest, the photographer must have
// left the strip visible. A client (PIN access) is the photographer's own
// view, so faces_visible_to_guests doesn't restrict them.
//
// `photos` is already visibility-filtered above, and this only ever asks
// about ids in that set, so it cannot widen what the caller sees.
let peopleEnabled = false;
let personIdsByPhoto = new Map();
try {
const { isEnabledForEvent, areFacesVisibleToGuests } = require('../services/faceSettings');
if (photos.length > 0 && await isEnabledForEvent(req.event)) {
peopleEnabled = isClient || areFacesVisibleToGuests(req.event);
if (peopleEnabled) {
const { getPersonIdsByPhoto } = require('../services/facePeopleService');
personIdsByPhoto = await getPersonIdsByPhoto(
req.event.id,
photos.map(p => p.id),
{ forAdmin: isClient }
);
}
}
} catch (err) {
// A face-feature failure must never take down the gallery payload.
logger.warn(`gallery: person_ids lookup failed for event ${req.event.id}`, { error: err.message });
peopleEnabled = false;
personIdsByPhoto = new Map();
}
// Get actual categories used by photos in this event // Get actual categories used by photos in this event
// This includes both global categories and event-specific ones // This includes both global categories and event-specific ones
const usedCategoryIds = hiddenForGuest ? [] : await db('photos') const usedCategoryIds = hiddenForGuest ? [] : await db('photos')
@@ -961,6 +990,11 @@ router.get('/:slug/photos', verifyGalleryAccess, resolveGuest, async (req, res)
// Mirror of the admin-side toggle so the lightbox can decide // Mirror of the admin-side toggle so the lightbox can decide
// whether to surface original camera filenames (#508). // whether to surface original camera filenames (#508).
use_original_filenames: useOriginalFilenames, use_original_filenames: useOriginalFilenames,
// "People in this gallery" (#1074). False whenever the global flag
// is off, detection is off for this event, or the photographer chose
// to keep the strip to themselves — the frontend renders no face UI
// at all in that case.
people_enabled: peopleEnabled,
...protectionSettings ...protectionSettings
}, },
// Reveal mode (#838): the guest UI switches to the upload-only view // Reveal mode (#838): the guest UI switches to the upload-only view
@@ -1050,6 +1084,12 @@ router.get('/:slug/photos', verifyGalleryAccess, resolveGuest, async (req, res)
// grid seed its lifted likedPhotoIds correctly on hard refresh. // grid seed its lifted likedPhotoIds correctly on hard refresh.
is_liked: showFeedbackToGuests ? likedPhotoIds.has(photo.id) : false, is_liked: showFeedbackToGuests ? likedPhotoIds.has(photo.id) : false,
favorite_count: showFeedbackToGuests ? (photo.favorite_count || 0) : 0, favorite_count: showFeedbackToGuests ? (photo.favorite_count || 0) : 0,
// People in this photo (#1074). Empty array when the feature is
// off for this event or hidden from guests, so the frontend has
// one shape to handle. Riding along on this payload is what keeps
// face filtering client-side and instant, like the category and
// liked/rated filters.
person_ids: personIdsByPhoto.get(photo.id) || [],
// Visibility (only included for clients) // Visibility (only included for clients)
...(isClient ? { visibility: photo.visibility || 'visible' } : {}) ...(isClient ? { visibility: photo.visibility || 'visible' } : {})
}; };
@@ -1060,6 +1100,62 @@ router.get('/:slug/photos', verifyGalleryAccess, resolveGuest, async (req, res)
} }
}); });
/**
* People in this gallery (#1074).
*
* Returns [] rather than 403 whenever the feature is unavailable — a guest
* must not be able to tell "this gallery has no people" from "this gallery
* has the feature switched off". Same reasoning as reveal mode returning an
* empty photo set rather than an error.
*
* Counts and cover faces are computed against the caller's own visibility
* scope inside facePeopleService; nothing here reads face_count_total.
*/
router.get('/:slug/people', verifyGalleryAccess, resolveGuest, async (req, res) => {
try {
const isClient = req.accessLevel === 'client';
const { isEnabledForEvent, areFacesVisibleToGuests, getThresholds } =
require('../services/faceSettings');
if (!(await isEnabledForEvent(req.event))) {
return res.json({ people: [] });
}
if (!isClient && !areFacesVisibleToGuests(req.event)) {
return res.json({ people: [] });
}
// While a gallery is hidden behind reveal mode (#838), a plain guest sees
// no photos — so they see no people either.
if (guestBlockedByReveal(req)) {
return res.json({ people: [] });
}
const { listPeople, getScanStatus } = require('../services/facePeopleService');
const thresholds = await getThresholds();
const people = await listPeople(req.event.id, {
isClient,
forAdmin: false,
minClusterSize: thresholds.face_min_cluster_size,
});
// Drives the "Finding people… 240/1200" progress line during a backfill.
// Scoped to what this viewer may see — an unscoped total would leak the
// number of hidden photos through the progress bar.
const status = await getScanStatus(req.event.id, { isClient });
res.json({
people,
scan: {
in_progress: status.in_progress,
scanned: status.scanned,
total: status.total,
},
});
} catch (error) {
errorResponse(res, error, 500, 'Failed to fetch people');
}
});
// Toggle photo visibility (client-only) // Toggle photo visibility (client-only)
router.patch('/:slug/photos/:photoId/visibility', verifyGalleryAccess, async (req, res) => { router.patch('/:slug/photos/:photoId/visibility', verifyGalleryAccess, async (req, res) => {
try { try {
+29
View File
@@ -225,6 +225,35 @@ async function archiveEvent(event) {
} }
} }
// Purge face data (#1074). photo_faces cascades off photos, but archiving
// does NOT delete the photo rows — and event_people hangs off the event,
// which also survives. So neither would go without an explicit purge, and
// an archived gallery would keep its biometric data indefinitely.
//
// Face data is derived: if the event is ever restored, re-enabling
// detection re-scans. Nothing irreplaceable is lost except assigned
// names, which is the same trade already accepted for backups/exports.
try {
const { purgeEvent } = require('./faceProcessor');
await purgeEvent(event.id);
// Turn detection OFF as well. purgeEvent clears the rows but leaves the
// toggle on, so restoring the archive would bring back a gallery that
// claims face detection is enabled while having no people and no queued
// work — indistinguishable from a broken scan. Off is the honest state:
// the photographer re-enables it and gets a fresh backfill, which is
// exactly the flow the toggle already implements.
await db('events').where({ id: event.id })
.update({ face_recognition_enabled: false, faces_last_scan_at: null });
} catch (err) {
// Never fail an archive over this — but say so loudly, because it
// means biometric data outlived the gallery.
logger.error(
`Archive: failed to purge face data for event ${event.slug}` +
`face rows may remain. ${err.message}`
);
}
// Queue completion email — admin_email is nullable on events (migration 073); // Queue completion email — admin_email is nullable on events (migration 073);
// skip queueing rather than violating email_queue.recipient_email NOT NULL. // skip queueing rather than violating email_queue.recipient_email NOT NULL.
// //
+71 -2
View File
@@ -16,6 +16,20 @@ const packageJson = require('../../package.json');
const CHUNK_SIZE = 1024 * 1024; // 1MB chunks for streaming const CHUNK_SIZE = 1024 * 1024; // 1MB chunks for streaming
const PROGRESS_INTERVAL = 100; // Report progress every 100 rows const PROGRESS_INTERVAL = 100; // Report progress every 100 rows
// Face recognition tables (#1074). Their SCHEMA is backed up, their CONTENTS
// are not: embeddings are biometric data (GDPR Art. 9) and fully derived from
// the photos, so a restore re-scans instead of carrying biometrics forward.
//
// The schema must survive, which is why Postgres uses --exclude-table-DATA
// rather than --exclude-table: knex_migrations records 177 as applied, so a
// restore whose dump lacked the CREATE TABLE would fail on the first query
// instead of merely coming back empty.
//
// SQLite cannot filter at all — `sqlite3 .backup` is a whole-file binary copy
// — so the rows are deleted from the temp copy before it is finalised. See
// createSQLiteBackup below.
const FACE_TABLES = ['photo_faces', 'event_people'];
/** /**
* Database Backup Service * Database Backup Service
* Supports both SQLite and PostgreSQL with proper escaping, * Supports both SQLite and PostgreSQL with proper escaping,
@@ -149,7 +163,7 @@ class DatabaseBackupService {
AND name != 'knex_migrations_lock' AND name != 'knex_migrations_lock'
ORDER BY name ORDER BY name
`); `);
return result.map(row => row.name); return result.map(row => row.name).filter((t) => !FACE_TABLES.includes(t));
} else { } else {
// PostgreSQL // PostgreSQL
const result = await db.raw(` const result = await db.raw(`
@@ -160,7 +174,9 @@ class DatabaseBackupService {
AND table_name NOT IN ('knex_migrations', 'knex_migrations_lock') AND table_name NOT IN ('knex_migrations', 'knex_migrations_lock')
ORDER BY table_name ORDER BY table_name
`); `);
return result.rows.map(row => row.table_name); return result.rows
.map(row => row.table_name)
.filter((t) => !FACE_TABLES.includes(t));
} }
} }
@@ -175,6 +191,48 @@ class DatabaseBackupService {
// Use SQLite's backup API for consistency // Use SQLite's backup API for consistency
await spawnAsync('sqlite3', [dbPath, `.backup '${tempPath}'`]); await spawnAsync('sqlite3', [dbPath, `.backup '${tempPath}'`]);
// Strip face data from the COPY (#1074). `.backup` is a whole-file
// binary copy with no way to exclude a table, so the rows come out and
// are deleted here — the live database is never touched.
//
// VACUUM is not cosmetic: without it the deleted pages remain in the
// file and "not backed up" would be false on disk, which is the exact
// claim this code exists to make true.
for (const table of FACE_TABLES) {
await spawnAsync('sqlite3', [
tempPath,
`DELETE FROM ${table} WHERE 1=1;`,
]).catch(() => {
// Table absent on installs that predate migration 177 — fine.
});
}
// Reset the DERIVED state on photos as well. Without this the restored
// database says every photo is scanned ('done') while the face tables
// are empty, and the worker only ever claims 'pending' — so the gallery
// reports a finished scan and shows nobody, permanently, until an admin
// works out that a manual re-scan is needed. Requeue instead.
await spawnAsync('sqlite3', [
tempPath,
"UPDATE photos SET face_status = CASE WHEN face_status IS NULL THEN NULL ELSE 'pending' END, "
+ 'face_count = NULL, face_started_at = NULL, face_error = NULL;',
]).catch(() => {});
// FATAL, not a warning. Deleting rows leaves their pages in the file
// until VACUUM rewrites it, so a backup that skipped the VACUUM can
// still contain recoverable face embeddings. Publishing it would break
// the explicit promise that biometric data is never backed up — better
// to fail the backup and say so than to hand over an artifact that
// quietly violates it.
try {
await spawnAsync('sqlite3', [tempPath, 'VACUUM;']);
} catch (err) {
await fs.unlink(tempPath).catch(() => {});
throw new Error(
'SQLite backup aborted: could not VACUUM after removing face data, so the '
+ `backup could still contain recoverable biometric pages (${err.message})`
);
}
// Verify the backup // Verify the backup
const verifyResult = await spawnAsync('sqlite3', [tempPath, 'PRAGMA integrity_check']); const verifyResult = await spawnAsync('sqlite3', [tempPath, 'PRAGMA integrity_check']);
if (!verifyResult.stdout.includes('ok')) { if (!verifyResult.stdout.includes('ok')) {
@@ -232,6 +290,17 @@ class DatabaseBackupService {
// callers (scheduled cron, dedicated admin DB-backup page) hit // callers (scheduled cron, dedicated admin DB-backup page) hit
// the same failure but on installs that had never exercised them. // the same failure but on installs that had never exercised them.
// Face data (#1074): schema yes, rows no. --exclude-table-DATA, not
// --exclude-table — see the FACE_TABLES comment at the top of this file
// for why dropping the CREATE TABLE would break restore outright.
for (const table of FACE_TABLES) {
pgDumpOptions.push(`--exclude-table-data=public.${table}`);
}
// NOTE: the Postgres path cannot rewrite rows inside pg_dump the way the
// SQLite path can, so photos.face_status is restored as-is here. The
// restore path compensates — see resetDerivedFaceState in restoreService.
// Both engines must end up requeued, not "done with no people".
// Add compression if not doing it separately // Add compression if not doing it separately
if (options.compress && !options.separateCompression) { if (options.compress && !options.separateCompression) {
pgDumpOptions.push('--compress=6'); pgDumpOptions.push('--compress=6');
+178
View File
@@ -0,0 +1,178 @@
/**
* Rule-based auto-categorisation from face counts (#1074 phase 3).
*
* Once faces are detected, `photos.face_count` and the landmark yaw are free
* inputs for the sorting @bluecow94 asked for in discussion #1069. This is a
* small rule list, not a classifier — the point is that it is predictable and
* explainable, which a photographer can work with.
*
* THREE NON-NEGOTIABLE RULES, in order of importance:
*
* 1. It only ever fills an EMPTY category. It never overwrites a category
* a photographer set, because a human assignment is a decision and this
* is a heuristic. There is no configuration that changes this.
*
* 2. Everything it does is marked `auto_categorized = true`, so "undo all
* automatic categories" is one query rather than an archaeology
* exercise.
*
* 3. It is a separate step from detection and separately enableable. Some
* photographers want the sorting and not the people strip.
*/
const { db } = require('../database/db');
const logger = require('../utils/logger');
const { getThresholds } = require('./faceSettings');
/**
* Category slugs this engine manages, in evaluation order. First match wins.
*
* `minFaceAreaRatio` on Portraits is what separates "a portrait of someone"
* from "someone happens to be in this landscape" — one face in the frame
* means nothing without knowing how much of the frame it occupies.
*/
const RULES = [
{
slug: 'details',
name: 'Details',
match: (photo) => photo.face_count === 0,
},
{
slug: 'portraits',
name: 'Portraits',
match: (photo, ctx) => photo.face_count === 1 && ctx.largestFaceAreaRatio >= 0.08,
},
{
slug: 'small-groups',
name: 'Small groups',
match: (photo) => photo.face_count >= 2 && photo.face_count <= 5,
},
{
slug: 'groups',
name: 'Groups',
match: (photo) => photo.face_count > 5,
},
];
/**
* Find or create the event-scoped category for a rule.
*
* Event-scoped rather than global: these are generated categories, and
* polluting every gallery's category list with them would be worse than the
* duplication. An existing GLOBAL category with the same slug is reused when
* present, so an operator who already has "Portraits" keeps one.
*/
async function resolveCategory(eventId, rule, cache, trx = db) {
if (cache.has(rule.slug)) return cache.get(rule.slug);
let category = await trx('photo_categories')
.where('slug', rule.slug)
.where(function () {
this.where('event_id', eventId).orWhere('is_global', true);
})
.first();
if (!category) {
const [inserted] = await trx('photo_categories').insert({
name: rule.name,
slug: rule.slug,
is_global: false,
event_id: eventId,
created_at: new Date().toISOString(),
}).returning('id');
const id = typeof inserted === 'object' ? inserted.id : inserted;
category = { id, slug: rule.slug };
}
cache.set(rule.slug, category);
return category;
}
/**
* Apply the rules to one event.
*
* Only touches photos that have been face-scanned (`face_status = 'done'`)
* and have no category. Returns a per-slug tally.
*/
async function categorizeEvent(eventId) {
const thresholds = await getThresholds();
if (!thresholds.face_auto_categorize_enabled) {
return { skipped: true, reason: 'disabled' };
}
const photos = await db('photos')
.where({ event_id: eventId, face_status: 'done' })
// Rule 1, enforced in the query rather than trusted to a later branch.
.whereNull('category_id')
.select('id', 'face_count', 'width', 'height');
if (!photos.length) return { assigned: 0, byCategory: {} };
// One query for every face box in the event, rather than one per photo.
const faceRows = await db('photo_faces')
.where({ event_id: eventId })
.whereIn('photo_id', photos.map((p) => p.id))
.select('photo_id', 'bbox_w', 'bbox_h');
const largestByPhoto = new Map();
for (const row of faceRows) {
const area = (row.bbox_w || 0) * (row.bbox_h || 0);
if (area > (largestByPhoto.get(row.photo_id) || 0)) {
largestByPhoto.set(row.photo_id, area);
}
}
const cache = new Map();
const byCategory = {};
let assigned = 0;
for (const photo of photos) {
const frameArea = (photo.width || 0) * (photo.height || 0);
const ctx = {
largestFaceAreaRatio: frameArea > 0
? (largestByPhoto.get(photo.id) || 0) / frameArea
: 0,
};
const rule = RULES.find((r) => r.match(photo, ctx));
if (!rule) continue;
const category = await resolveCategory(eventId, rule, cache);
// Guard the UPDATE on category_id still being NULL. Between the SELECT
// above and here a photographer may have set one by hand, and their
// choice wins — rule 1 is not a best-effort.
const updated = await db('photos')
.where({ id: photo.id })
.whereNull('category_id')
.update({ category_id: category.id, auto_categorized: true });
if (updated > 0) {
assigned += updated;
byCategory[rule.slug] = (byCategory[rule.slug] || 0) + updated;
}
}
logger.info(
`faceAutoCategories: assigned ${assigned} photo(s) in event ${eventId} `
+ `(${Object.entries(byCategory).map(([k, v]) => `${k}=${v}`).join(', ') || 'none'})`
);
return { assigned, byCategory };
}
/**
* Undo every automatic assignment for an event.
*
* Clears only rows this engine set — a photographer's own categories are
* untouched, which is the entire reason `auto_categorized` exists.
*/
async function undoEvent(eventId) {
const cleared = await db('photos')
.where({ event_id: eventId, auto_categorized: true })
.update({ category_id: null, auto_categorized: false });
logger.info(`faceAutoCategories: cleared ${cleared} automatic category assignment(s) in event ${eventId}`);
return { cleared };
}
module.exports = { RULES, categorizeEvent, undoEvent };
+134
View File
@@ -0,0 +1,134 @@
/**
* HTTP client for the picpeak-ml sidecar (#1074).
*
* The important behaviour here is the error taxonomy, because the queue
* treats the two classes completely differently:
*
* SidecarUnavailableError — the sidecar is down, unreachable, timing out,
* or returned 5xx. The photo goes BACK to 'pending' and is retried later.
* Turning the container off for a week must not require a manual re-scan.
*
* Everything else (4xx) — this image is a lost cause. The photo is marked
* 'failed' and never retried, because retrying an undecodable file
* forever is just a busy loop.
*
* Log volume matters too: an hour of downtime at a 1s poll is 3,600 identical
* warnings. Unavailability is logged at most once per LOG_INTERVAL_MS.
*/
const axios = require('axios');
const logger = require('../utils/logger');
const { getSidecarUrl, getSidecarToken } = require('./faceSettings');
const REQUEST_TIMEOUT_MS = parseInt(process.env.FACE_ML_TIMEOUT_MS || '30000', 10);
const LOG_INTERVAL_MS = 5 * 60 * 1000;
let lastUnavailableLogAt = 0;
class SidecarUnavailableError extends Error {
constructor(message) {
super(message);
this.name = 'SidecarUnavailableError';
}
}
class SidecarRejectedError extends Error {
constructor(message, status) {
super(message);
this.name = 'SidecarRejectedError';
this.status = status;
}
}
function logUnavailable(message) {
const now = Date.now();
if (now - lastUnavailableLogAt < LOG_INTERVAL_MS) return;
lastUnavailableLogAt = now;
logger.warn(
`faceClient: ML sidecar unavailable (${message}). Photos stay queued and will ` +
'be retried; no action needed unless this persists. Further identical warnings ' +
'are suppressed for 5 minutes.'
);
}
function authHeaders() {
const token = getSidecarToken();
return token ? { 'X-Face-ML-Token': token } : {};
}
/**
* Translate an axios failure into our two-class taxonomy.
*/
function classify(err) {
const status = err.response?.status;
if (status && status >= 400 && status < 500) {
// 401 is a configuration error, not a bad image — but it is also not
// something retrying fixes, so it surfaces loudly and stops the photo.
if (status === 401) {
logger.error(
'faceClient: sidecar rejected our token (401). FACE_ML_TOKEN must match ' +
'on both the backend and the picpeak-ml container.'
);
}
return new SidecarRejectedError(
err.response?.data?.detail || `Sidecar rejected the request (${status})`,
status
);
}
logUnavailable(err.code || err.message || `HTTP ${status}`);
return new SidecarUnavailableError(err.message || 'Sidecar unreachable');
}
/**
* Detect faces in an image. `buffer` is the preview rendition's bytes.
* Returns the sidecar's `{ model_version, faces: [...] }`.
*/
async function detectFaces(buffer, filename = 'photo.jpg') {
const form = new FormData();
// Node 18+ ships FormData/Blob globally, so no multipart dependency is
// needed for the one endpoint that uploads anything.
form.append('image', new Blob([buffer]), filename);
try {
const response = await axios.post(`${getSidecarUrl()}/faces`, form, {
headers: authHeaders(),
timeout: REQUEST_TIMEOUT_MS,
// A 45MP preview is ~2MB; the cap is generous but not unbounded.
maxBodyLength: 64 * 1024 * 1024,
maxContentLength: 64 * 1024 * 1024,
});
return response.data;
} catch (err) {
throw classify(err);
}
}
/**
* Sidecar identity + liveness, for the admin connection test.
* Returns { ok: true, info } or { ok: false, error } — never throws, because
* the caller is a UI button and a stack trace helps nobody there.
*/
async function checkHealth() {
try {
const { data } = await axios.get(`${getSidecarUrl()}/info`, {
headers: authHeaders(),
timeout: 5000,
});
return { ok: true, info: data };
} catch (err) {
const status = err.response?.status;
if (status === 401) {
return { ok: false, error: 'Sidecar rejected the token (check FACE_ML_TOKEN on both containers)' };
}
return { ok: false, error: err.message || 'Sidecar unreachable' };
}
}
module.exports = {
detectFaces,
checkHealth,
SidecarUnavailableError,
SidecarRejectedError,
};
+533
View File
@@ -0,0 +1,533 @@
/**
* Per-event face clustering (#1074).
*
* Plain JS over a few thousand rows. A gallery is hundreds to low-thousands
* of faces, so this needs no vector database and no native extension — the
* embeddings live in a BLOB column and the maths is a dot product.
*
* Algorithm: greedy incremental assignment. A new face is compared against
* every existing person centroid in the event; above the match threshold it
* joins the nearest one and updates that centroid as a running mean,
* otherwise it opens a new person.
*
* Greedy assignment is order-dependent by construction — the same photos
* uploaded in a different order can produce different clusters. That is
* accepted here (it is what Immich, PhotoPrism and Ente all do) and mitigated
* by `consolidate()`, which merges centroid pairs that have drifted together,
* plus the admin merge/split tools. What it buys is that a photo can be
* clustered the moment it is scanned, so the strip fills in during a backfill
* instead of after it.
*
* Embeddings arrive L2-normalized from the sidecar, so cosine similarity is a
* plain dot product. Centroids are re-normalized after every update to keep
* that true.
*/
const { db } = require('../database/db');
const logger = require('../utils/logger');
const { getThresholds } = require('./faceSettings');
const FLOAT_BYTES = 4;
// -- serialization -----------------------------------------------------------
/**
* Float32Array → Buffer for the BLOB column. Little-endian on every platform
* we ship (x86_64, aarch64), and the value never leaves the deployment, so
* no byte-order header is needed.
*/
function packEmbedding(vec) {
const arr = vec instanceof Float32Array ? vec : Float32Array.from(vec);
return Buffer.from(arr.buffer, arr.byteOffset, arr.byteLength);
}
function unpackEmbedding(buf) {
if (!buf) return null;
// Postgres bytea comes back as Buffer; SQLite may hand back a Uint8Array.
const b = Buffer.isBuffer(buf) ? buf : Buffer.from(buf);
if (b.length % FLOAT_BYTES !== 0) return null;
// Copy rather than aliasing: a Buffer from the driver may be a view into a
// larger pooled allocation, and Float32Array over an unaligned offset
// throws.
const copy = Buffer.from(b);
return new Float32Array(copy.buffer, copy.byteOffset, copy.length / FLOAT_BYTES);
}
// -- vector maths ------------------------------------------------------------
function dot(a, b) {
let sum = 0;
const n = Math.min(a.length, b.length);
for (let i = 0; i < n; i++) sum += a[i] * b[i];
return sum;
}
function normalize(vec) {
let sumSq = 0;
for (let i = 0; i < vec.length; i++) sumSq += vec[i] * vec[i];
const norm = Math.sqrt(sumSq);
if (norm === 0) return vec;
const out = new Float32Array(vec.length);
for (let i = 0; i < vec.length; i++) out[i] = vec[i] / norm;
return out;
}
/**
* Running mean of `count` existing vectors with one new vector, re-normalized.
*/
function updateCentroid(centroid, count, incoming) {
const out = new Float32Array(centroid.length);
for (let i = 0; i < centroid.length; i++) {
out[i] = (centroid[i] * count + incoming[i]) / (count + 1);
}
return normalize(out);
}
// -- quality -----------------------------------------------------------------
/**
* Is this face good enough to define a person?
*
* Low-quality faces are still STORED and still shown in "this photo contains"
* — they just don't get assigned, so they can't spawn junk people or drag a
* good centroid off course. A blurry profile at 30px is a real detection and
* a terrible identity.
*/
function meetsQualityFloor(face, thresholds) {
if (face.det_score != null && face.det_score < thresholds.face_quality_min_score) return false;
const size = Math.min(face.bbox_w ?? 0, face.bbox_h ?? 0);
if (size < thresholds.face_quality_min_px) return false;
return true;
}
// -- clustering --------------------------------------------------------------
/**
* Assign a set of freshly-inserted faces to people within one event.
*
* Loads the event's people once, mutates centroids in memory across the whole
* batch, then writes back — so a photo with five faces costs one read and one
* write pass rather than five of each.
*/
/**
* Per-event serialization for cluster assignment.
*
* assignFaces is read-modify-write over an event's people: it loads every
* centroid, mutates them in memory across the batch, then writes back. Two
* workers on the same event therefore lose updates — both read the same
* snapshot, and the second write clobbers the first (or both open a duplicate
* person for the same face). The default concurrency is 1, but the queue
* advertises multi-pod safety and the tunable exists, so this cannot rely on
* there only ever being one writer.
*
* In-process mutex + (on Postgres) a transaction-scoped advisory lock keyed on
* the event. The advisory lock is what covers multiple pods; the mutex avoids
* pointless lock round-trips within one process. SQLite is single-writer
* anyway, so the mutex alone is sufficient there.
*/
const eventLocks = new Map();
async function withEventLock(eventId, trx, fn) {
const previous = eventLocks.get(eventId) || Promise.resolve();
let release;
const current = new Promise((resolve) => { release = resolve; });
eventLocks.set(eventId, previous.then(() => current));
await previous;
try {
// pg_advisory_xact_lock is released automatically when the transaction
// ends, including on rollback — no leak if the caller throws.
if (trx && typeof trx.raw === 'function') {
const client = db.client.config.client;
const isPg = client === 'pg' || (typeof client === 'string' && client.includes('postgres'));
if (isPg) {
await trx.raw('SELECT pg_advisory_xact_lock(?, ?)', [1074, Number(eventId)]);
}
}
return await fn();
} finally {
release();
if (eventLocks.get(eventId) === current) eventLocks.delete(eventId);
}
}
async function assignFaces(eventId, faceRows, options = {}) {
const thresholds = options.thresholds || (await getThresholds());
const trx = options.trx || db;
return withEventLock(eventId, options.trx, () => assignFacesLocked(eventId, faceRows, thresholds, trx));
}
async function assignFacesLocked(eventId, faceRows, thresholds, trx) {
const people = await trx('event_people')
.where({ event_id: eventId })
.select('id', 'centroid', 'face_count_total', 'model_version');
const state = people.map((p) => ({
id: p.id,
centroid: unpackEmbedding(p.centroid),
count: p.face_count_total || 0,
dirty: false,
modelVersion: p.model_version,
})).filter((p) => p.centroid);
const assignments = [];
for (const face of faceRows) {
const embedding = unpackEmbedding(face.embedding);
if (!embedding) continue;
if (!meetsQualityFloor(face, thresholds)) {
assignments.push({ faceId: face.id, personId: null });
continue;
}
let best = null;
let bestScore = -Infinity;
for (const person of state) {
// Never compare across embedding spaces — a model change makes old
// centroids meaningless rather than merely stale.
if (person.modelVersion && face.model_version && person.modelVersion !== face.model_version) {
continue;
}
const score = dot(embedding, person.centroid);
if (score > bestScore) {
bestScore = score;
best = person;
}
}
if (best && bestScore >= thresholds.face_match_threshold) {
best.centroid = updateCentroid(best.centroid, best.count, embedding);
best.count += 1;
best.dirty = true;
assignments.push({ faceId: face.id, personId: best.id });
} else {
const [inserted] = await trx('event_people').insert({
event_id: eventId,
centroid: packEmbedding(embedding),
face_count_total: 1,
model_version: face.model_version,
cover_face_id: face.id,
created_at: new Date().toISOString(),
updated_at: new Date().toISOString(),
}).returning('id');
const personId = typeof inserted === 'object' ? inserted.id : inserted;
state.push({
id: personId,
centroid: embedding,
count: 1,
dirty: false,
modelVersion: face.model_version,
});
assignments.push({ faceId: face.id, personId });
}
}
for (const { faceId, personId } of assignments) {
await trx('photo_faces').where({ id: faceId }).update({ person_id: personId });
}
for (const person of state) {
if (!person.dirty) continue;
await trx('event_people').where({ id: person.id }).update({
centroid: packEmbedding(person.centroid),
face_count_total: person.count,
updated_at: new Date().toISOString(),
});
}
return assignments;
}
/**
* Merge people whose centroids have drifted together.
*
* Greedy assignment can open "Anna in daylight" and "Anna at the party" as
* two clusters when the first few photos of each were dissimilar. Once both
* have absorbed enough faces their centroids converge, and this pass catches
* that. Runs at a slightly stricter threshold than initial assignment:
* merging two established clusters is a bigger claim than adding one face to
* one of them, and an over-eager merge is much harder for a photographer to
* unpick than a missed one.
*/
async function consolidate(eventId, options = {}) {
const thresholds = options.thresholds || (await getThresholds());
const mergeThreshold = Math.min(0.95, thresholds.face_match_threshold + 0.08);
const people = await db('event_people')
.where({ event_id: eventId })
.select('id', 'centroid', 'face_count_total', 'model_version', 'label');
const state = people
.map((p) => ({ ...p, vec: unpackEmbedding(p.centroid) }))
.filter((p) => p.vec);
const merged = [];
const absorbed = new Set();
for (let i = 0; i < state.length; i++) {
if (absorbed.has(state[i].id)) continue;
for (let j = i + 1; j < state.length; j++) {
if (absorbed.has(state[j].id)) continue;
const a = state[i];
const b = state[j];
if (a.model_version !== b.model_version) continue;
// Never silently merge two people the photographer has NAMED
// differently — that is a human assertion this heuristic does not get
// to overrule.
if (a.label && b.label && a.label !== b.label) continue;
if (dot(a.vec, b.vec) >= mergeThreshold) {
await mergePeople(eventId, [b.id], a.id);
absorbed.add(b.id);
merged.push({ from: b.id, into: a.id });
}
}
}
if (merged.length) {
logger.info(`faceClustering: consolidated ${merged.length} person pair(s) in event ${eventId}`);
}
return merged;
}
/**
* Move every face from `sourceIds` onto `targetId` and delete the sources.
* Recomputes the target centroid from its actual members rather than
* averaging the two centroids — cheap at this scale, and exact.
*/
async function mergePeople(eventId, sourceIds, targetId) {
const ids = sourceIds.filter((id) => id !== targetId);
if (!ids.length) return { moved: 0 };
return db.transaction(async (trx) => {
// Carry metadata forward before the sources are deleted. Without this a
// merge silently discards a photographer-entered name, or un-hides a
// person they had suppressed — the target keeps its own values where it
// has them, and inherits from a source only where it does not.
const target = await trx('event_people').where({ id: targetId }).first();
const sources = await trx('event_people').whereIn('id', ids).select('label', 'is_hidden', 'is_ignored');
const inherited = {};
if (target && !target.label) {
const named = sources.find((p) => p.label);
if (named) inherited.label = named.label;
}
// Suppression is one-way on merge: if ANY party was hidden or ignored,
// the survivor stays that way. Re-exposing someone by merging is the
// failure that matters; leaving them hidden is trivially reversible.
if (sources.some((p) => p.is_hidden) || target?.is_hidden) inherited.is_hidden = true;
if (sources.some((p) => p.is_ignored) || target?.is_ignored) inherited.is_ignored = true;
const moved = await trx('photo_faces')
.where({ event_id: eventId })
.whereIn('person_id', ids)
.update({ person_id: targetId });
await trx('event_people').where({ event_id: eventId }).whereIn('id', ids).del();
if (Object.keys(inherited).length) {
await trx('event_people').where({ id: targetId })
.update({ ...inherited, updated_at: new Date().toISOString() });
}
await recomputeCentroid(targetId, trx);
return { moved };
});
}
/**
* Pull `faceIds` out of their cluster into a brand-new person.
*/
async function splitPerson(eventId, personId, faceIds) {
if (!faceIds?.length) return null;
return db.transaction(async (trx) => {
const faces = await trx('photo_faces')
.where({ event_id: eventId, person_id: personId })
.whereIn('id', faceIds)
.select('id', 'embedding', 'model_version');
if (!faces.length) return null;
const [inserted] = await trx('event_people').insert({
event_id: eventId,
face_count_total: 0,
model_version: faces[0].model_version,
created_at: new Date().toISOString(),
updated_at: new Date().toISOString(),
}).returning('id');
const newPersonId = typeof inserted === 'object' ? inserted.id : inserted;
await trx('photo_faces')
.whereIn('id', faces.map((f) => f.id))
.update({ person_id: newPersonId });
await recomputeCentroid(newPersonId, trx);
await recomputeCentroid(personId, trx);
return newPersonId;
});
}
/**
* Recompute one person's centroid and count from its member faces.
* Deletes the person if it has no members left.
*/
async function recomputeCentroid(personId, trx = db) {
const faces = await trx('photo_faces')
.where({ person_id: personId })
.select('id', 'embedding', 'det_score');
if (!faces.length) {
await trx('event_people').where({ id: personId }).del();
return;
}
const vectors = faces.map((f) => unpackEmbedding(f.embedding)).filter(Boolean);
if (!vectors.length) return;
const mean = new Float32Array(vectors[0].length);
for (const vec of vectors) {
for (let i = 0; i < mean.length; i++) mean[i] += vec[i];
}
for (let i = 0; i < mean.length; i++) mean[i] /= vectors.length;
// Cover face: the highest-scoring detection in the cluster, so the avatar
// is the sharpest available crop of that person rather than whichever face
// happened to arrive first.
const cover = faces.reduce((a, b) => ((b.det_score ?? 0) > (a.det_score ?? 0) ? b : a));
await trx('event_people').where({ id: personId }).update({
centroid: packEmbedding(normalize(mean)),
face_count_total: faces.length,
cover_face_id: cover.id,
updated_at: new Date().toISOString(),
});
}
/**
* Re-derive every cluster in an event from stored embeddings, without
* touching the sidecar. Needed after threshold tuning — the expensive part
* (inference) is already done and cached in the rows.
*
* Preserves labels by re-attaching them to whichever new cluster inherited
* the most faces from the old one. Without this, re-clustering a gallery
* would silently discard every name the photographer typed.
*/
async function recluster(eventId) {
const thresholds = await getThresholds();
// Remember every person carrying HUMAN state — a name, or a hidden/ignored
// decision. Keying this on `label` alone silently dropped the privacy flags
// of unnamed people: a bystander the photographer had suppressed came back
// guest-visible after one "Re-group people".
const previousLabels = await db('event_people')
.where({ event_id: eventId })
.where(function () {
this.whereNotNull('label').orWhere('is_hidden', true).orWhere('is_ignored', true);
})
.select('id', 'label', 'is_hidden', 'is_ignored');
const priorMembership = new Map();
if (previousLabels.length) {
const rows = await db('photo_faces')
.where({ event_id: eventId })
.whereIn('person_id', previousLabels.map((p) => p.id))
.select('id', 'person_id');
for (const row of rows) priorMembership.set(row.id, row.person_id);
}
await db.transaction(async (trx) => {
await trx('photo_faces').where({ event_id: eventId }).update({ person_id: null });
await trx('event_people').where({ event_id: eventId }).del();
});
const faces = await db('photo_faces')
.where({ event_id: eventId })
.orderBy('id', 'asc')
.select('id', 'embedding', 'model_version', 'det_score', 'bbox_w', 'bbox_h');
const assignments = await assignFaces(eventId, faces, { thresholds });
// Re-attach labels by majority inheritance.
if (previousLabels.length) {
const tally = new Map(); // newPersonId -> Map<oldPersonId, count>
for (const { faceId, personId } of assignments) {
if (personId == null) continue;
const oldId = priorMembership.get(faceId);
if (oldId == null) continue;
if (!tally.has(personId)) tally.set(personId, new Map());
const inner = tally.get(personId);
inner.set(oldId, (inner.get(oldId) || 0) + 1);
}
// Suppression is OR-ed across EVERY ancestor that contributed faces to a
// new cluster — not copied from the majority one. Reclustering can merge a
// visible named person with a hidden one; taking the majority ancestor's
// flags would then publish the hidden person's photos. Erring toward
// staying hidden is trivially reversible; erring toward visible is not.
const suppression = new Map(); // newId -> { is_hidden, is_ignored }
for (const [newId, inner] of tally) {
let hidden = false;
let ignored = false;
for (const oldId of inner.keys()) {
const old = previousLabels.find((p) => p.id === oldId);
if (!old) continue;
if (old.is_hidden) hidden = true;
if (old.is_ignored) ignored = true;
}
suppression.set(newId, { is_hidden: hidden, is_ignored: ignored });
}
// The NAME goes to exactly one cluster: the descendant that inherited the
// MOST of that old person's faces, chosen globally. Iterating `tally` and
// taking the first match gave the name to whichever cluster happened to
// come first — an early outlier could take it while the real majority
// cluster ended up unnamed.
const bestDescendant = new Map(); // oldId -> { newId, count }
for (const [newId, inner] of tally) {
for (const [oldId, count] of inner) {
const current = bestDescendant.get(oldId);
if (!current || count > current.count) bestDescendant.set(oldId, { newId, count });
}
}
const labelFor = new Map(); // newId -> label
for (const [oldId, { newId }] of bestDescendant) {
const old = previousLabels.find((p) => p.id === oldId);
if (old?.label && !labelFor.has(newId)) labelFor.set(newId, old.label);
}
for (const [newId, flags] of suppression) {
const update = { ...flags, updated_at: new Date().toISOString() };
if (labelFor.has(newId)) update.label = labelFor.get(newId);
await db('event_people').where({ id: newId }).update(update);
}
}
for (const personId of new Set(assignments.map((a) => a.personId).filter(Boolean))) {
await recomputeCentroid(personId);
}
await consolidate(eventId, { thresholds });
const count = await db('event_people').where({ event_id: eventId }).count({ c: '*' }).first();
logger.info(`faceClustering: reclustered event ${eventId}${count?.c ?? 0} people`);
return Number(count?.c ?? 0);
}
module.exports = {
packEmbedding,
unpackEmbedding,
dot,
normalize,
meetsQualityFloor,
assignFaces,
consolidate,
mergePeople,
splitPerson,
recomputeCentroid,
recluster,
};
+247
View File
@@ -0,0 +1,247 @@
/**
* People queries for the gallery and admin surfaces (#1074).
*
* THE RULE THIS MODULE EXISTS TO ENFORCE: a guest's view of "who is in this
* gallery" must be computed over the photos that guest can actually see.
*
* Guests are restricted to `photos.visibility = 'visible'` (gallery.js);
* PIN-clients see everything. `event_people.face_count_total` counts ALL
* faces, so handing it to a guest leaks the existence and volume of hidden
* photos — and a cover face chosen from a hidden photo would render a crop of
* an image the guest was never allowed to open. Neither is theoretical: a
* photographer hides shots precisely because someone should not see them.
*
* So the guest-facing count and cover are recomputed per request against the
* same predicate the photo query uses. `face_count_total` is never returned
* to a guest.
*/
const { db } = require('../database/db');
/**
* Apply the visibility predicate a given audience is allowed to see.
* Mirrors gallery.js: guests get 'visible' (or NULL, pre-migration rows),
* clients get everything. Also excludes photos still being processed, which
* are invisible in the gallery payload too.
*/
function applyVisibilityScope(query, { isClient }) {
query.where(function () {
this.where('photos.processing_status', 'complete').orWhereNull('photos.processing_status');
});
if (!isClient) {
query.where(function () {
this.where('photos.visibility', 'visible').orWhereNull('photos.visibility');
});
}
return query;
}
/**
* People in an event, scoped to what this audience can see.
*
* Returns [{ id, label, face_count, cover: { photo_id, bbox } }] ordered by
* visible photo count. People whose visible count falls below `minClusterSize`
* are dropped — a person who appears in three hidden photos and one visible
* one is not "in this gallery" from the guest's point of view.
*/
async function listPeople(eventId, { isClient = false, forAdmin = false, minClusterSize = 3 } = {}) {
const people = await db('event_people')
.where({ event_id: eventId })
.modify((q) => {
if (!forAdmin) {
// Hidden = photographer-only. Ignored = bystander/false positive.
// Neither ever reaches a guest response.
q.where('is_hidden', false).where('is_ignored', false);
}
})
.select('id', 'label', 'is_hidden', 'is_ignored', 'face_count_total', 'cover_face_id');
if (!people.length) return [];
// One grouped query for the audience-scoped counts, rather than N queries.
const counts = await applyVisibilityScope(
db('photo_faces')
.join('photos', 'photos.id', 'photo_faces.photo_id')
.where('photo_faces.event_id', eventId)
.whereNotNull('photo_faces.person_id'),
{ isClient }
)
.groupBy('photo_faces.person_id')
.select('photo_faces.person_id')
.countDistinct({ visible_photos: 'photos.id' });
const countByPerson = new Map(
counts.map((r) => [r.person_id, Number(r.visible_photos) || 0])
);
// Cover faces, also scoped: the stored cover_face_id may point at a face in
// a photo this audience cannot see. Pick the best VISIBLE face instead.
const covers = await applyVisibilityScope(
db('photo_faces')
.join('photos', 'photos.id', 'photo_faces.photo_id')
.where('photo_faces.event_id', eventId)
.whereNotNull('photo_faces.person_id'),
{ isClient }
)
.orderBy('photo_faces.det_score', 'desc')
.select(
'photo_faces.id',
'photo_faces.person_id',
'photo_faces.photo_id',
'photo_faces.bbox_x',
'photo_faces.bbox_y',
'photo_faces.bbox_w',
'photo_faces.bbox_h',
// The bbox is in ORIGINAL image pixels, so any consumer cropping it has
// to know what those pixels were measured against. Without these the
// admin manager was scaling an original-space box by a THUMBNAIL's
// natural size and rendering the wrong region entirely.
'photos.width as photo_width',
'photos.height as photo_height'
);
const coverByPerson = new Map();
for (const row of covers) {
// Rows arrive best-score-first, so the first hit per person wins.
if (!coverByPerson.has(row.person_id)) coverByPerson.set(row.person_id, row);
}
const out = [];
for (const person of people) {
const count = countByPerson.get(person.id) || 0;
const cover = coverByPerson.get(person.id);
// A person with no visible photos, or too few to be worth a face in the
// strip, simply does not exist for this audience.
if (!forAdmin && (count < minClusterSize || !cover)) continue;
if (forAdmin && !cover && count === 0) {
out.push({
id: person.id,
label: person.label || null,
face_count: 0,
total_face_count: person.face_count_total,
is_hidden: !!person.is_hidden,
is_ignored: !!person.is_ignored,
cover: null,
});
continue;
}
const entry = {
id: person.id,
label: person.label || null,
face_count: count,
cover: cover
? {
face_id: cover.id,
photo_id: cover.photo_id,
bbox: [cover.bbox_x, cover.bbox_y, cover.bbox_w, cover.bbox_h],
photo_width: cover.photo_width ?? null,
photo_height: cover.photo_height ?? null,
}
: null,
};
if (forAdmin) {
// face_count_total is admin-only by design — see the module header.
entry.total_face_count = person.face_count_total;
entry.is_hidden = !!person.is_hidden;
entry.is_ignored = !!person.is_ignored;
}
out.push(entry);
}
return out.sort((a, b) => b.face_count - a.face_count);
}
/**
* Map of photoId → person ids, for the photos in `photoIds`.
*
* Only ever called with ids already present in the gallery payload, which is
* itself visibility-filtered — so this cannot widen what a caller sees. It
* still filters hidden/ignored people, so a hidden person leaves no trace on
* a photo the guest CAN see.
*/
async function getPersonIdsByPhoto(eventId, photoIds, { forAdmin = false } = {}) {
if (!photoIds?.length) return new Map();
const rows = await db('photo_faces')
.join('event_people', 'event_people.id', 'photo_faces.person_id')
.where('photo_faces.event_id', eventId)
.whereIn('photo_faces.photo_id', photoIds)
.whereNotNull('photo_faces.person_id')
.modify((q) => {
if (!forAdmin) {
q.where('event_people.is_hidden', false).where('event_people.is_ignored', false);
}
})
.select('photo_faces.photo_id', 'photo_faces.person_id');
const map = new Map();
for (const row of rows) {
if (!map.has(row.photo_id)) map.set(row.photo_id, []);
const list = map.get(row.photo_id);
if (!list.includes(row.person_id)) list.push(row.person_id);
}
return map;
}
/**
* Scan progress for the admin status line and the gallery's "Finding
* people… 240/1200" indicator.
*/
async function getScanStatus(eventId, { isClient = true } = {}) {
// `isClient` defaults to TRUE (the admin/photographer view) because every
// existing caller is an admin surface. A guest must pass isClient:false:
// counting every photo with a face_status would otherwise tell them how
// many hidden photos the gallery holds — the same leak the people list and
// covers are already scoped against, arriving through the progress bar.
const rows = await applyVisibilityScope(
db('photos').where({ event_id: eventId }).whereNotNull('face_status'),
{ isClient }
)
.groupBy('face_status')
.select('face_status')
.count({ c: '*' });
const byStatus = Object.fromEntries(rows.map((r) => [r.face_status, Number(r.c) || 0]));
const done = byStatus.done || 0;
const total = Object.values(byStatus).reduce((a, b) => a + b, 0);
const peopleRow = await db('event_people')
.where({ event_id: eventId })
.count({ c: '*' })
.first();
// Two counts, deliberately. `people` is every cluster that exists;
// `people_visible_to_guests` applies the minimum-cluster-size floor and the
// hidden/ignored flags, i.e. what the gallery strip actually shows. An
// admin comparing the settings page against their own gallery will
// otherwise see two numbers that disagree with no explanation.
const { getThresholds } = require('./faceSettings');
const thresholds = await getThresholds();
const visible = await listPeople(eventId, {
isClient: false,
forAdmin: false,
minClusterSize: thresholds.face_min_cluster_size,
});
return {
scanned: done,
total,
pending: (byStatus.pending || 0) + (byStatus.processing || 0),
failed: byStatus.failed || 0,
skipped: byStatus.skipped || 0,
people: Number(peopleRow?.c ?? 0),
people_visible_to_guests: visible.length,
in_progress: ((byStatus.pending || 0) + (byStatus.processing || 0)) > 0,
};
}
module.exports = {
listPeople,
getPersonIdsByPhoto,
getScanStatus,
applyVisibilityScope,
};
+325
View File
@@ -0,0 +1,325 @@
/**
* Per-photo face detection (#1074).
*
* Runs on the PREVIEW rendition, not the original. A 1920px JPEG is ~40x
* cheaper to decode than a 45MP original and costs nothing in recall at the
* face sizes an event gallery actually contains.
*
* Note the side effect that has: on an install with `lightbox_preview_enabled`
* off, no previews exist, so a face backfill generates the whole preview tier
* as it goes — a real Sharp workload and real disk. That is why the admin
* toggle warns about it and why the face queue defaults to concurrency 1.
*/
const sharp = require('sharp');
const { db } = require('../database/db');
const logger = require('../utils/logger');
const { getStorage } = require('./storage');
const { ensurePreviewImage } = require('./imageProcessor');
// SidecarUnavailableError is deliberately NOT caught here — it propagates to
// faceQueue, which is the only layer that knows to retry rather than fail.
const { detectFaces } = require('./faceClient');
const { assignFaces, packEmbedding, recomputeCentroid } = require('./faceClustering');
const { getThresholds, isEnabledForEvent } = require('./faceSettings');
/**
* Thrown when a scan finishes but the photo is no longer the row we claimed —
* the event was purged, archived or re-queued mid-flight. Not a failure of
* the photo, so the queue must not mark it 'failed'.
*/
class StaleScanError extends Error {
constructor(photoId) {
super(`Face scan for photo ${photoId} was superseded`);
this.name = 'StaleScanError';
}
}
async function streamToBuffer(stream) {
if (Buffer.isBuffer(stream)) return stream;
const chunks = [];
for await (const chunk of stream) {
chunks.push(Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk));
}
return Buffer.concat(chunks);
}
/**
* Detect and cluster the faces in one photo.
*
* Throws SidecarUnavailableError when the sidecar is down — the caller must
* return the row to 'pending' rather than failing it.
*/
async function processPhotoFaces(photoId) {
const photo = await db('photos').where({ id: photoId }).first();
if (!photo) {
logger.warn(`faceProcessor: photo ${photoId} disappeared before processing`);
return { status: 'skipped' };
}
const event = await db('events').where({ id: photo.event_id }).first();
if (!(await isEnabledForEvent(event))) {
// The toggle was switched off between enqueue and claim. Not an error.
await db('photos').where({ id: photoId }).update({
face_status: 'skipped', face_started_at: null, face_error: null,
});
return { status: 'skipped' };
}
// Videos have no preview tier and no meaningful single frame to scan.
const isVideo = photo.media_type === 'video'
|| (photo.mime_type && photo.mime_type.startsWith('video/'));
if (isVideo) {
await db('photos').where({ id: photoId }).update({
face_status: 'skipped', face_started_at: null, face_count: 0,
});
return { status: 'skipped' };
}
// External / reference photos live outside managed storage, and
// resolvePhotoStorageKey returns null for them by design (photoResolver.js).
// ensurePreviewImage therefore cannot build a preview, so scanning them is
// unsupported rather than broken — 'skipped', not 'failed', so an external
// gallery does not report every photo as an error.
if (photo.source_origin === 'external' || photo.source_origin === 'reference') {
await db('photos').where({ id: photoId }).update({
face_status: 'skipped', face_started_at: null, face_error: null,
});
return { status: 'skipped' };
}
const previewKey = await ensurePreviewImage(photo);
if (!previewKey) {
// No preview and no way to make one — the source is missing or corrupt.
// That is a property of this photo, so it fails rather than retries.
await db('photos').where({ id: photoId }).update({
face_status: 'failed',
face_started_at: null,
face_error: 'Could not generate a preview rendition to scan',
});
return { status: 'failed' };
}
const storage = getStorage();
const buffer = await streamToBuffer(await storage.get(previewKey));
// Throws SidecarUnavailableError upward on a down sidecar — deliberately
// not caught here, so the queue can distinguish "retry" from "failed".
const result = await detectFaces(buffer, photo.filename || 'photo.jpg');
const faces = result?.faces || [];
const modelVersion = result?.model_version || null;
// Bounding boxes come back in the pixel space of the image the SIDECAR was
// given — which is the preview (≤1920px long edge), not the original. Every
// consumer compares them against photos.width/height, which are the
// ORIGINAL dimensions: the strip's avatar crop works in ratios of them, and
// the auto-category portrait rule divides face area by frame area.
//
// Left unscaled, a 6000px photo yields boxes ~3x too small (and areas ~9x
// too small), so avatars crop to the wrong place and "Portraits" never
// fires. It is invisible on any photo already under 1920px, which is why
// the demo gallery looked correct.
//
// Scale here, once, so everything downstream can assume original-image
// coordinates.
let boxScale = 1;
try {
const previewMeta = await sharp(buffer).metadata();
if (previewMeta?.width && photo.width) {
boxScale = photo.width / previewMeta.width;
}
} catch (err) {
logger.warn(`faceProcessor: could not read preview dimensions for photo ${photoId}`, {
error: err.message,
});
}
const thresholds = await getThresholds();
try {
await db.transaction(async (trx) => {
// Replace rather than append: a re-scan of the same photo must not
// double its faces. Detaching first keeps the FK to event_people clean.
//
// Remember which people the OLD faces belonged to. Deleting the rows does
// not undo their contribution to event_people.face_count_total or to the
// running-mean centroid, so without recomputing those afterwards a
// re-scan inflates every count (typically doubling it) and leaves ghost
// people behind when a face is no longer detected.
const affectedPeople = await trx('photo_faces')
.where({ photo_id: photoId })
.whereNotNull('person_id')
.distinct('person_id')
.pluck('person_id');
await trx('photo_faces').where({ photo_id: photoId }).del();
const inserted = [];
for (const face of faces) {
const [bx, by, bw, bh] = (face.bbox || [0, 0, 0, 0]).map((v) => v * boxScale);
const row = {
photo_id: photoId,
event_id: photo.event_id,
bbox_x: bx, bbox_y: by, bbox_w: bw, bbox_h: bh,
det_score: face.score ?? null,
yaw: face.yaw ?? null,
pitch: face.pitch ?? null,
blur: face.blur ?? null,
embedding: face.embedding ? packEmbedding(face.embedding) : null,
model_version: modelVersion,
// ISO string, not a Date — under Jest a Date handed to the sqlite3
// binding stores as the literal "[object Object]" (see CLAUDE.md).
created_at: new Date().toISOString(),
};
const [id] = await trx('photo_faces').insert(row).returning('id');
inserted.push({ ...row, id: typeof id === 'object' ? id.id : id });
}
// Rebuild the people the removed faces belonged to BEFORE assigning the
// replacements, so assignment compares against honest centroids.
for (const personId of affectedPeople) {
await recomputeCentroid(personId, trx);
}
if (inserted.length) {
await assignFaces(photo.event_id, inserted, { thresholds, trx });
}
// Commit the photo only if it is STILL the row we claimed. An admin who
// purges or archives the event while this worker was waiting on the
// sidecar has already cleared face_status; without this guard the worker
// would write its rows back moments after "all face data deleted"
// reported success, so an erasure request would silently not stick.
const committed = await trx('photos')
.where({ id: photoId, face_status: 'processing' })
.update({
face_status: 'done',
face_count: faces.length,
face_started_at: null,
face_error: null,
});
if (committed === 0) {
logger.info(
`faceProcessor: photo ${photoId} was purged or re-queued while scanning — discarding results`
);
// Undo this transaction's inserts rather than leaving orphans behind.
throw new StaleScanError(photoId);
}
});
} catch (err) {
// Losing a race with a purge is an expected outcome, not a photo that
// failed to scan. The transaction has already rolled back, so nothing was
// written; leave face_status exactly as the purge left it.
if (err instanceof StaleScanError) return { status: 'skipped' };
throw err;
}
// Auto-categories (#1074 phase 3) run as a distinct step AFTER detection,
// outside the transaction: they are a convenience, and a rule-engine
// failure must never roll back the faces we just paid the sidecar for.
// No-ops unless separately enabled.
try {
const { categorizeEvent } = require('./faceAutoCategories');
await categorizeEvent(photo.event_id);
} catch (err) {
logger.warn(`faceProcessor: auto-categorisation failed for event ${photo.event_id}`, {
error: err.message,
});
}
return { status: 'done', faceCount: faces.length };
}
/**
* Queue every eligible photo in an event for (re-)scanning.
*
* `onlyUnscanned` is the backfill case — enabling the toggle on a gallery
* that already has photos. Without it, this is a full re-scan.
*/
async function enqueueEvent(eventId, { onlyUnscanned = false } = {}) {
const query = db('photos')
.where({ event_id: eventId })
// Photos still being processed have no preview and no dimensions yet;
// photoProcessor enqueues them itself on completion.
.where(function () {
this.where('processing_status', 'complete').orWhereNull('processing_status');
});
if (onlyUnscanned) {
query.where(function () {
this.whereNull('face_status').orWhereIn('face_status', ['failed', 'skipped']);
});
}
const updated = await query.update({
face_status: 'pending',
face_started_at: null,
face_error: null,
});
logger.info(`faceProcessor: queued ${updated} photo(s) for face detection in event ${eventId}`);
return updated;
}
/**
* Remove every face row for an event. Used by "delete all face data", by the
* per-event toggle going off, and by archival.
*/
async function purgeEvent(eventId) {
return db.transaction(async (trx) => {
// Detach before deleting people so the SET NULL FK never fires against
// rows that are about to go anyway.
await trx('photo_faces').where({ event_id: eventId }).update({ person_id: null });
const faces = await trx('photo_faces').where({ event_id: eventId }).del();
const people = await trx('event_people').where({ event_id: eventId }).del();
await trx('photos').where({ event_id: eventId }).update({
face_status: null, face_count: null, face_started_at: null, face_error: null,
});
logger.info(`faceProcessor: purged ${faces} face(s) and ${people} person(s) from event ${eventId}`);
return { faces, people };
});
}
/**
* Remove the face rows belonging to one photo, and rebuild the people that
* lose members as a result.
*
* Called from every photo-deletion path. The schema declares ON DELETE
* CASCADE, but SQLite only enforces foreign keys when `PRAGMA foreign_keys`
* is on and PicPeak does not enable it globally — so on SQLite the cascade
* never fires and biometric embeddings would outlive the photo. Even where it
* does fire (Postgres), the cascade cannot fix up event_people counts or
* centroids, which is the other half of this.
*
* Safe to call for photos that were never scanned: it simply deletes nothing.
*/
async function purgePhotoFaces(photoId, trx = db) {
const affected = await trx('photo_faces')
.where({ photo_id: photoId })
.whereNotNull('person_id')
.distinct('person_id')
.pluck('person_id');
// Drop any in-flight claim as well. A worker holding this photo would
// otherwise still satisfy its `face_status = 'processing'` commit guard and
// write fresh faces straight after the purge — and with FK enforcement off
// on SQLite, the subsequent photo delete cannot cascade them away, leaving
// orphaned biometric rows.
await trx('photos').where({ id: photoId }).update({
face_status: null, face_count: null, face_started_at: null, face_error: null,
});
const removed = await trx('photo_faces').where({ photo_id: photoId }).del();
if (!removed) return { removed: 0 };
const { recomputeCentroid: recompute } = require('./faceClustering');
for (const personId of affected) {
// Deletes the person outright when it has no members left.
await recompute(personId, trx);
}
return { removed, peopleTouched: affected.length };
}
module.exports = {
processPhotoFaces, enqueueEvent, purgeEvent, purgePhotoFaces, StaleScanError,
};
+210
View File
@@ -0,0 +1,210 @@
/**
* Background face-detection worker pool (#1074).
*
* Deliberately a near-copy of backgroundProcessor.js rather than a shared
* abstraction: the two differ in exactly one behaviour (below), and inventing
* a generic queue framework to save forty lines would make both harder to
* read. Claim semantics, the janitor and the tunable shape are identical, so
* anyone who understands one understands the other.
*
* THE ONE DIFFERENCE — a sidecar that is unreachable puts the photo back to
* 'pending' with backoff, never 'failed'. Turning the ML container off for a
* week must not require a manual re-scan afterwards. Only a 4xx (a genuinely
* unprocessable image) marks a row failed.
*
* Nothing here starts unless the `faces` feature flag is on. That matters
* more than usual because FACE_ML_URL now has a working default, so "is the
* variable set" is no longer a signal — without the flag check, every install
* would poll a hostname that does not resolve.
*
* Tunables (env, all optional):
* FACE_PROCESSOR_CONCURRENCY default 1. Face scanning shares a host
* with Sharp; 1 is the safe default on
* the 2GB VPS class this project supports.
* FACE_PROCESSOR_POLL_MS default 2000
* FACE_PROCESSOR_STUCK_TIMEOUT_MS default 600000 (10 minutes)
* FACE_PROCESSOR_DISABLED default false ('true' to opt out, e.g. CI)
*/
const { db } = require('../database/db');
const logger = require('../utils/logger');
const { processPhotoFaces } = require('./faceProcessor');
const { SidecarUnavailableError } = require('./faceClient');
const { isFeatureEnabled } = require('./faceSettings');
const POLL_INTERVAL_MS = parseInt(process.env.FACE_PROCESSOR_POLL_MS || '2000', 10);
const CONCURRENCY = Math.max(1, parseInt(process.env.FACE_PROCESSOR_CONCURRENCY || '1', 10));
const STUCK_TIMEOUT_MS = parseInt(process.env.FACE_PROCESSOR_STUCK_TIMEOUT_MS || '600000', 10);
const JANITOR_INTERVAL_MS = 60 * 1000;
// Backoff after the sidecar goes away. Without it a down sidecar turns into a
// hot loop: claim, fail, release, claim again, thousands of times a minute.
const UNAVAILABLE_BACKOFF_MS = parseInt(process.env.FACE_PROCESSOR_BACKOFF_MS || '30000', 10);
let running = false;
let workerHandles = [];
let janitorHandle = null;
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
function isPostgres() {
const c = db.client.config.client;
return c === 'pg' || (typeof c === 'string' && c.includes('postgres'));
}
/**
* Atomically claim the oldest pending photo. Returns the row or null.
* Same two-path approach as backgroundProcessor: SKIP LOCKED on Postgres so
* multiple pods race cleanly, a status-guarded UPDATE on SQLite.
*/
async function claimNextPhoto() {
if (isPostgres()) {
return db.transaction(async (trx) => {
const row = await trx('photos')
.where('face_status', 'pending')
.orderBy('id', 'asc')
.forUpdate()
.skipLocked()
.first();
if (!row) return null;
await trx('photos').where('id', row.id).update({
face_status: 'processing',
face_started_at: new Date().toISOString(),
});
return row;
});
}
return db.transaction(async (trx) => {
const row = await trx('photos')
.where('face_status', 'pending')
.orderBy('id', 'asc')
.first();
if (!row) return null;
const updated = await trx('photos')
.where({ id: row.id, face_status: 'pending' })
.update({
face_status: 'processing',
face_started_at: new Date().toISOString(),
});
return updated > 0 ? row : null;
});
}
async function releaseToPending(photoId) {
// Guarded on 'processing' — the state THIS worker put the row in. If the
// event was purged while the sidecar request was in flight, purgeEvent has
// already set face_status to NULL, and an unconditional update would put it
// back to 'pending' and rescan it: biometric rows reappearing after the
// purge endpoint reported success. Same reasoning as the commit guard in
// faceProcessor; the retry path needed it too.
await db('photos')
.where({ id: photoId, face_status: 'processing' })
.update({ face_status: 'pending', face_started_at: null });
}
async function workerLoop(workerIdx) {
while (running) {
// Re-checked every tick, not once at startup: an admin turning the flag
// off must stop the workers without a restart.
if (!(await isFeatureEnabled())) {
await sleep(POLL_INTERVAL_MS * 5);
continue;
}
let claimed;
try {
claimed = await claimNextPhoto();
} catch (e) {
logger.warn(`faceQueue[${workerIdx}]: claim error`, { error: e.message });
await sleep(POLL_INTERVAL_MS);
continue;
}
if (!claimed) {
await sleep(POLL_INTERVAL_MS);
continue;
}
try {
await processPhotoFaces(claimed.id);
} catch (err) {
if (err instanceof SidecarUnavailableError) {
// Retry, don't fail. faceClient already rate-limits the log line.
await releaseToPending(claimed.id).catch(() => {});
await sleep(UNAVAILABLE_BACKOFF_MS);
continue;
}
logger.error(`faceQueue[${workerIdx}]: photo ${claimed.id} failed`, {
error: err.message,
stack: err.stack,
});
try {
await db('photos').where({ id: claimed.id }).update({
face_status: 'failed',
face_started_at: null,
face_error: String(err.message || err).slice(0, 1000),
});
} catch (updateErr) {
logger.error(`faceQueue[${workerIdx}]: failed to mark photo ${claimed.id} as failed`, {
error: updateErr.message,
});
}
}
}
}
async function janitorLoop() {
while (running) {
try {
const cutoff = new Date(Date.now() - STUCK_TIMEOUT_MS).toISOString();
const reset = await db('photos')
.where('face_status', 'processing')
.where('face_started_at', '<', cutoff)
.update({ face_status: 'pending', face_started_at: null });
if (reset > 0) {
logger.warn(`faceQueue: janitor reset ${reset} stuck photo(s) from 'processing' to 'pending'`);
}
} catch (e) {
logger.warn('faceQueue: janitor error', { error: e.message });
}
await sleep(JANITOR_INTERVAL_MS);
}
}
function start() {
if (running) return;
if (process.env.FACE_PROCESSOR_DISABLED === 'true') {
logger.info('faceQueue: disabled via FACE_PROCESSOR_DISABLED');
return;
}
running = true;
workerHandles = [];
for (let i = 0; i < CONCURRENCY; i++) {
workerHandles.push(
workerLoop(i).catch((e) =>
logger.error(`faceQueue[${i}]: crashed`, { error: e.message, stack: e.stack })
)
);
}
janitorHandle = janitorLoop().catch((e) =>
logger.error('faceQueue: janitor crashed', { error: e.message, stack: e.stack })
);
logger.info(
`faceQueue: started ${CONCURRENCY} worker(s), poll=${POLL_INTERVAL_MS}ms, stuck=${STUCK_TIMEOUT_MS}ms ` +
'(idle until the `faces` feature flag is enabled)'
);
}
async function stop() {
if (!running) return;
running = false;
await Promise.all([...workerHandles, janitorHandle].filter(Boolean));
workerHandles = [];
janitorHandle = null;
}
module.exports = { start, stop, claimNextPhoto };
+170
View File
@@ -0,0 +1,170 @@
/**
* Face-recognition settings and the feature gate (#1074).
*
* One module owns the answer to "is this feature on?", because the answer has
* three independent parts and getting any of them wrong is either a privacy
* problem or a log-spam problem:
*
* 1. The `faces` feature flag (global, default OFF).
* 2. `FACE_ML_URL` — which now has a DEFAULT (the compose service name),
* so its presence proves nothing. This is exactly why the flag is the
* gate: if the URL were the gate, every install would try to reach a
* hostname that doesn't resolve.
* 3. The per-event `face_recognition_enabled` toggle.
*
* Nothing may contact the sidecar unless (1) is true. `isFeatureEnabled()` is
* the only correct way to ask.
*/
const { db } = require('../database/db');
const {
isFeatureEnabled: isFlagEnabled,
invalidateFeatureFlagCache,
} = require('../middleware/requireFeatureFlag');
const logger = require('../utils/logger');
// Canonical flag key. Must match the entry in adminFeatureFlags KNOWN_FLAGS
// and the FeatureKey union on the frontend.
const FLAG_KEY = 'faces';
/**
* Is this the all-in-one single-container image (#1042 / PR #1068)?
*
* Face recognition is BLOCKED there, on performance grounds. The AIO image
* runs the backend, the frontend, SQLite and every background worker inside
* one container sized for "one photographer plus guests browsing" — it has no
* Redis and SQLite gives it a single writer. Face detection would add a
* second image-processing pipeline competing with Sharp for the same CPU and
* RAM, on top of an ML sidecar the image does not contain and cannot start.
*
* Enabling it there would not fail loudly; it would just make the whole
* install slow and appear broken, which is the worst shape for a deployment
* aimed at people who want one container and no decisions.
*
* Detected from an explicit marker rather than inferred from SERVE_FRONTEND
* or a SQLite path: plenty of legitimate multi-container setups serve the
* frontend from the backend or run SQLite, and none of them should lose the
* feature by accident.
*/
function isSingleContainerImage() {
const v = String(process.env.PICPEAK_SINGLE_CONTAINER || '').toLowerCase();
return v === 'true' || v === '1' || v === 'yes';
}
const DEFAULTS = {
// Measured on a real gallery, not just on LFW pairs — see migration 177
// for why those two disagree and why cluster purity wins.
face_match_threshold: 0.60,
face_min_cluster_size: 3,
face_quality_min_score: 0.7,
face_quality_min_px: 40,
face_auto_categorize_enabled: false,
};
const SETTING_KEYS = Object.keys(DEFAULTS);
function parseSetting(raw, fallback) {
if (raw === undefined || raw === null) return fallback;
try {
return JSON.parse(raw);
} catch {
// Pre-JSON rows (and plain strings written by hand) — coerce by the
// shape of the default so callers always get the type they expect.
if (typeof fallback === 'number') {
const n = Number(raw);
return Number.isFinite(n) ? n : fallback;
}
if (typeof fallback === 'boolean') return raw === 'true' || raw === '1';
return raw;
}
}
/**
* Is the `faces` feature flag on? This is THE gate — see the header.
*
* Delegates to the shared `feature_flags` reader (already cached, already
* invalidated by PUT /admin/feature-flags) rather than keeping a second
* cache that could disagree with the middleware guarding the same routes.
*/
async function isFeatureEnabled() {
// Hard block on the all-in-one image, ahead of the flag. Even if the row
// says true — restored from a backup taken on a full deployment, say — the
// feature stays off here.
if (isSingleContainerImage()) return false;
try {
return await isFlagEnabled(FLAG_KEY);
} catch (e) {
// A missing table (pre-migration install) means off, not a crash.
logger.debug?.('faceSettings: feature flag read failed', { error: e.message });
return false;
}
}
/**
* Effective sidecar URL. Callers must check isFeatureEnabled() first — this
* always returns a string, because the default is a hostname rather than an
* absence.
*/
function getSidecarUrl() {
return (process.env.FACE_ML_URL || 'http://picpeak-ml:8000').replace(/\/+$/, '');
}
function getSidecarToken() {
return process.env.FACE_ML_TOKEN || '';
}
/**
* Clustering/quality thresholds, with defaults. Deliberately settings rather
* than constants: no single threshold survives contact with every library.
*/
async function getThresholds() {
const out = { ...DEFAULTS };
try {
const rows = await db('app_settings')
.whereIn('setting_key', SETTING_KEYS)
.select('setting_key', 'setting_value');
for (const row of rows) {
out[row.setting_key] = parseSetting(row.setting_value, DEFAULTS[row.setting_key]);
}
} catch (e) {
logger.debug?.('faceSettings: threshold read failed, using defaults', { error: e.message });
}
return out;
}
/**
* Is detection enabled for this specific event? Requires BOTH the global flag
* and the per-event toggle — the "two deliberate actions" rule from #1074 §6.
*/
async function isEnabledForEvent(event) {
if (!(await isFeatureEnabled())) return false;
if (!event) return false;
return event.face_recognition_enabled === true || event.face_recognition_enabled === 1;
}
/**
* Do GUESTS of this event see the people strip? Separate from the above so a
* photographer can use clustering as a private tool.
*
* Defaults to true when detection is on, because the column is nullable and
* NULL means "not explicitly set" rather than "off" — matching the tri-state
* convention used by show_watermark / show_qr.
*/
function areFacesVisibleToGuests(event) {
if (!event) return false;
return event.faces_visible_to_guests !== false && event.faces_visible_to_guests !== 0;
}
module.exports = {
DEFAULTS,
FLAG_KEY,
isSingleContainerImage,
isFeatureEnabled,
invalidateFeatureFlagCache,
getSidecarUrl,
getSidecarToken,
getThresholds,
isEnabledForEvent,
areFacesVisibleToGuests,
};
+36
View File
@@ -283,6 +283,25 @@ async function processUploadedPhotos(files, eventId, uploadedBy = 'admin', categ
}); });
} catch (e) { /* non-fatal */ } } catch (e) { /* non-fatal */ }
// Face detection (#1074). processPhoto() — the ASYNC path — enqueues on
// completion, but this synchronous path (chunked-upload completion,
// watch-folder import) writes a finished photo row directly and never
// reaches it, so those photos stayed unscanned in a face-enabled event
// until someone ran a manual re-scan.
try {
const { isEnabledForEvent } = require('./faceSettings');
if (!isVideo && await isEnabledForEvent(event)) {
// `db`, NOT `trx`: the transaction is committed above, so a query
// through it throws "Transaction query already complete" — which the
// catch below swallowed, making this whole enqueue a silent no-op.
await db('photos').where({ id: photoId }).update({ face_status: 'pending' });
}
} catch (err) {
logger.warn(`processUploadedPhotos: face enqueue failed for photo ${photoId}`, {
error: err.message,
});
}
uploadedPhotos.push({ uploadedPhotos.push({
id: photoId, id: photoId,
filename: newFilename, filename: newFilename,
@@ -541,6 +560,23 @@ async function processPhoto(photoId) {
// Mark complete // Mark complete
updateData.processing_status = 'complete'; updateData.processing_status = 'complete';
updateData.processing_error = null; updateData.processing_error = null;
// Face detection (#1074): this is the only correct place to enqueue.
// Earlier and there is no preview rendition to scan; later and there is no
// hook at all. Same UPDATE rather than a follow-up write, so a crash
// between the two can't leave a complete photo permanently unqueued.
// Guarded on BOTH the global flag and the per-event toggle, so installs
// without the feature never write a face_status at all.
try {
const { isEnabledForEvent } = require('./faceSettings');
if (!isVideo && await isEnabledForEvent(event)) {
updateData.face_status = 'pending';
}
} catch (err) {
// Never let the face feature block a photo from completing.
logger.warn(`processPhoto: face enqueue check failed for ${photoId}`, { error: err.message });
}
await db('photos').where({ id: photoId }).update(updateData); await db('photos').where({ id: photoId }).update(updateData);
// Side effects (best-effort, never fail the photo if these break) // Side effects (best-effort, never fail the photo if these break)
@@ -122,6 +122,26 @@ async function replacePhoto(existingPhoto, newFileTempPath, { originalFilename,
media_type: mimeType?.startsWith('video/') ? 'video' : 'image', media_type: mimeType?.startsWith('video/') ? 'video' : 'image',
}; };
// Face data (#1074): the row keeps its id but now points at a DIFFERENT
// image, so every stored face describes the old picture. Left alone the
// gallery would keep showing the previous subject's identities on the new
// photo. Drop them, then re-queue if the event has detection on.
try {
const { purgePhotoFaces } = require('./faceProcessor');
const { isEnabledForEvent } = require('./faceSettings');
await purgePhotoFaces(existingPhoto.id);
const event = await db('events').where({ id: existingPhoto.event_id }).first();
updates.face_status = (await isEnabledForEvent(event)) ? 'pending' : null;
updates.face_count = null;
updates.face_started_at = null;
updates.face_error = null;
} catch (err) {
logger.warn(`replacePhoto: face reset failed for photo ${existingPhoto.id}`, {
error: err.message,
});
}
await db('photos').where({ id: existingPhoto.id }).update(updates); await db('photos').where({ id: existingPhoto.id }).update(updates);
const updatedPhoto = await db('photos').where({ id: existingPhoto.id }).first(); const updatedPhoto = await db('photos').where({ id: existingPhoto.id }).first();
+10
View File
@@ -6,6 +6,7 @@
*/ */
const path = require('path'); const path = require('path');
const logger = require('../utils/logger');
const fs = require('fs').promises; const fs = require('fs').promises;
const { db } = require('../database/db'); const { db } = require('../database/db');
const { formatBoolean } = require('../utils/dbCompat'); const { formatBoolean } = require('../utils/dbCompat');
@@ -120,6 +121,15 @@ const deletePhoto = async (photoId, options = {}) => {
} }
} }
// Face data (#1074) — see purgePhotoFaces on why the FK cascade is not
// relied on. Must run BEFORE the photo row goes.
try {
const { purgePhotoFaces } = require('./faceProcessor');
await purgePhotoFaces(photoId);
} catch (err) {
logger.warn(`deletePhoto: face purge failed for photo ${photoId}`, { error: err.message });
}
// Delete from database // Delete from database
await db('photos').where('id', photoId).delete(); await db('photos').where('id', photoId).delete();
} }
+27 -2
View File
@@ -29,7 +29,20 @@ const packageJson = require('../../package.json');
const PICPEAK_FORMAT_VERSION = 1; const PICPEAK_FORMAT_VERSION = 1;
// Never exported as data — the target owns these (its own migrations set them). // Never exported as data — the target owns these (its own migrations set them).
const EXCLUDED_TABLES = new Set(['knex_migrations', 'knex_migrations_lock']); //
// photo_faces / event_people (#1074) are excluded for a different reason:
// face embeddings are biometric data (GDPR Art. 9), and this export gets
// handed to clients and moved between operators. The data is fully derived
// from the photos, so the target re-scans rather than receiving biometrics it
// has no lawful basis for. NOTE the cost of that decision: any names the
// photographer assigned to people are lost too, since they live in
// event_people. That is accepted — see ml/README.md and the #1074 thread.
const EXCLUDED_TABLES = new Set([
'knex_migrations',
'knex_migrations_lock',
'photo_faces',
'event_people',
]);
// Storage subdirs holding non-recalculable blobs — always included. // Storage subdirs holding non-recalculable blobs — always included.
const DOC_DIRS = ['business-docs', 'uploads']; const DOC_DIRS = ['business-docs', 'uploads'];
@@ -90,7 +103,19 @@ async function writeTableNdjson(table, dataDir) {
const hash = crypto.createHash('sha256'); const hash = crypto.createHash('sha256');
const rows = await db(table).select('*'); const rows = await db(table).select('*');
const lines = rows.map((row) => { const lines = rows.map((row) => {
const line = JSON.stringify(row); // photo_faces / event_people are excluded from the export (#1074), so a
// restored install has no face data — but `photos.face_status = 'done'`
// would come across intact and the worker only ever claims 'pending'.
// The gallery would then report itself fully scanned while showing no
// people at all, permanently, with no way to tell why.
//
// Reset the derived state so the target simply re-scans once the operator
// enables the feature there.
const line = JSON.stringify(
table === 'photos' && (row.face_status !== null && row.face_status !== undefined)
? { ...row, face_status: null, face_count: null, face_started_at: null, face_error: null }
: row
);
hash.update(`${line}\n`); hash.update(`${line}\n`);
return line; return line;
}); });
@@ -351,6 +351,21 @@ async function replaceAllTables(tables, dataDir, currentAdmin, roleSnapshot, { c
for (const table of tables) { for (const table of tables) {
await trx(table).del(); await trx(table).del();
} }
// Face data (#1074) is excluded from the archive, which also excludes it
// from `tables` — so the LOCAL rows would survive a whole-DB replace.
// FK enforcement is deliberately suspended during import, so those
// orphans can end up attached to reused photo/event ids from the incoming
// archive: one instance's biometric data silently adopted by another's
// galleries. Purge them explicitly.
for (const faceTable of ['photo_faces', 'event_people']) {
try {
await trx(faceTable).del();
} catch (err) {
// Absent on targets that predate migration 177 — nothing to purge.
}
}
for (const table of tables) { for (const table of tables) {
const rows = parseNdjson(path.join(dataDir, `${table}.ndjson`)); const rows = parseNdjson(path.join(dataDir, `${table}.ndjson`));
if (!rows.length) continue; if (!rows.length) continue;
@@ -493,7 +508,30 @@ async function importFromPicpeak({ picpeakPath, currentAdminId }) {
await resyncSequences(tables); await resyncSequences(tables);
await setSessionsValidAfter(Math.floor(Date.now() / 1000)); await setSessionsValidAfter(Math.floor(Date.now() / 1000));
const filesRestored = await restoreFiles(staging); const filesRestored = await restoreFiles(staging);
// Face data (#1074): queue ONLY once the files are on disk. The archive
// carries no face rows and the export blanked photos.face_status, but the
// event toggles come across enabled, so the "enable" transition that
// normally triggers a backfill never happens here.
//
// Ordering matters: the worker is live during a restore. Queued before
// restoreFiles, it races the copy and either scans the PREVIOUS
// instance's files or marks photos failed for originals that are not
// there yet — and nothing re-queues them afterwards.
try {
const requeued = await db('photos')
.whereIn('event_id', db('events').select('id').where('face_recognition_enabled', true))
.update({
face_status: 'pending', face_count: null, face_started_at: null, face_error: null,
});
if (requeued > 0) {
logger.info(`picpeakImport: queued ${requeued} photo(s) for face detection after import`);
}
} catch (err) {
logger.debug?.(`picpeakImport: face requeue skipped: ${err.message}`);
}
const usesExternalMedia = await detectExternalMedia(); const usesExternalMedia = await detectExternalMedia();
logger.info( logger.info(
+46
View File
@@ -212,9 +212,15 @@ class RestoreService {
switch (options.restoreType) { switch (options.restoreType) {
case 'full': case 'full':
restoreResult = await this.performFullRestore(localBackupPath, manifest, options); restoreResult = await this.performFullRestore(localBackupPath, manifest, options);
// After the FILES too — see requeueFaceScans on why the ordering
// matters with a live worker.
await this.requeueFaceScans();
break; break;
case 'database': case 'database':
restoreResult = await this.performDatabaseRestore(localBackupPath, manifest, options); restoreResult = await this.performDatabaseRestore(localBackupPath, manifest, options);
// Database-only restore: the existing files stay in place, so there
// is nothing to race.
await this.requeueFaceScans();
break; break;
case 'files': case 'files':
restoreResult = await this.performFilesRestore(localBackupPath, manifest, options); restoreResult = await this.performFilesRestore(localBackupPath, manifest, options);
@@ -836,6 +842,45 @@ class RestoreService {
/** /**
* Perform full restore (database + files) * Perform full restore (database + files)
*/ */
/**
* Requeue face detection after a restore (#1074).
*
* Face data is deliberately excluded from backups — it is derived, and
* biometric data should not travel in an archive. But photos.face_status
* DOES restore, so without this the restored install claims every photo is
* scanned while photo_faces is empty, and the worker never picks them up
* because it only claims 'pending'. The gallery shows a finished scan and
* no people, forever, with nothing to indicate why.
*
* MUST run after the FILES are restored, not merely after the database.
* The face worker is live throughout a restore; queued earlier it races the
* file copy and either scans the previous instance's originals or marks
* photos failed for files that are not there yet — and nothing re-queues
* them afterwards.
*
* Only touches rows that had been scanned; NULL stays NULL, so this never
* switches the feature on for anyone.
*/
async requeueFaceScans() {
try {
const { db: restoredDb } = require('../database/db');
const requeued = await restoredDb('photos')
.whereNotNull('face_status')
.update({
face_status: 'pending',
face_count: null,
face_started_at: null,
face_error: null,
});
if (requeued > 0) {
this.log('info', `Requeued ${requeued} photo(s) for face detection after restore`);
}
} catch (err) {
// Pre-migration-177 backups have no such column; not an error.
this.log('info', `Face state reset skipped: ${err.message}`);
}
}
async performFullRestore(backupPath, manifest, options) { async performFullRestore(backupPath, manifest, options) {
const result = { const result = {
databaseRestored: false, databaseRestored: false,
@@ -1166,6 +1211,7 @@ END $$;`
await reinitPool(); await reinitPool();
this.log('info', 'Knex pool re-initialized'); this.log('info', 'Knex pool re-initialized');
// NOTE: we deliberately do NOT call `db.migrate.latest()` here. // NOTE: we deliberately do NOT call `db.migrate.latest()` here.
// //
// The picpeak migrations directory contains `helpers.js` (a // The picpeak migrations directory contains `helpers.js` (a
+37
View File
@@ -102,6 +102,13 @@ services:
- PICPEAK_RELEASE_CHANNEL=${PICPEAK_CHANNEL:-stable} - PICPEAK_RELEASE_CHANNEL=${PICPEAK_CHANNEL:-stable}
# Watch-folder auto-import: max photos processed in parallel (default 2). # Watch-folder auto-import: max photos processed in parallel (default 2).
- FILE_WATCHER_CONCURRENCY=${FILE_WATCHER_CONCURRENCY:-2} - FILE_WATCHER_CONCURRENCY=${FILE_WATCHER_CONCURRENCY:-2}
# Face recognition (#1074). Defaults to the sidecar's compose service
# name; nothing touches it until the `faces` feature flag is enabled in
# admin settings, so installs without the picpeak-ml container are
# unaffected. Start the sidecar with `--profile faces`.
- FACE_ML_URL=${FACE_ML_URL:-http://picpeak-ml:8000}
- FACE_ML_TOKEN=${FACE_ML_TOKEN:-}
- FACE_PROCESSOR_CONCURRENCY=${FACE_PROCESSOR_CONCURRENCY:-}
volumes: volumes:
- ${APP_STORAGE}:/app/storage - ${APP_STORAGE}:/app/storage
- ${LOGS}:/app/logs - ${LOGS}:/app/logs
@@ -140,6 +147,36 @@ services:
timeout: 10s timeout: 10s
retries: 3 retries: 3
# Optional face-detection sidecar (#1074). Gated behind the `faces` profile:
# `docker compose --profile faces up -d`. Nothing depends on it, and the
# backend never calls it while the `faces` feature flag is off.
#
# The service name is `picpeak-ml` because it doubles as the hostname in
# FACE_ML_URL's default. Renaming it breaks that default for every install
# that never set the variable.
picpeak-ml:
image: ghcr.io/picpeak/picpeak/ml:${PICPEAK_CHANNEL:-stable}
container_name: picpeak-ml
profiles:
- faces
environment:
# The container refuses to start without this rather than serving
# anonymously — it must match the backend's FACE_ML_TOKEN.
- FACE_ML_TOKEN=${FACE_ML_TOKEN:-}
- FACE_ORT_THREADS=${FACE_ORT_THREADS:-1}
- TZ=${TZ:-UTC}
# No volumes and no published ports: stateless, and reachable only from
# the backend on picpeak-network.
networks:
- picpeak-network
restart: unless-stopped
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=4).status == 200 else 1)"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
frontend: frontend:
# Use pre-built image from GitHub Container Registry # Use pre-built image from GitHub Container Registry
# Uses same channel as backend for consistency # Uses same channel as backend for consistency
+52
View File
@@ -73,6 +73,14 @@ services:
- STORAGE_PATH=/app/storage - STORAGE_PATH=/app/storage
# Watch-folder auto-import: max photos processed in parallel (default 2). # Watch-folder auto-import: max photos processed in parallel (default 2).
- FILE_WATCHER_CONCURRENCY=${FILE_WATCHER_CONCURRENCY:-2} - FILE_WATCHER_CONCURRENCY=${FILE_WATCHER_CONCURRENCY:-2}
# Face recognition (#1074). The URL defaults to the sidecar's compose
# service name, so the common case needs no configuration. None of this
# is touched until the `faces` feature flag is enabled in admin
# settings — an install without the picpeak-ml container never attempts
# a connection.
- FACE_ML_URL=${FACE_ML_URL:-http://picpeak-ml:8000}
- FACE_ML_TOKEN=${FACE_ML_TOKEN:-}
- FACE_PROCESSOR_CONCURRENCY=${FACE_PROCESSOR_CONCURRENCY:-}
# No `user:` directive — as of #484, the container starts as root, # No `user:` directive — as of #484, the container starts as root,
# chowns the bind mounts to nodejs (UID 1001), then drops privileges # chowns the bind mounts to nodejs (UID 1001), then drops privileges
# via su-exec. PUID/PGID env vars are no longer read; if you need # via su-exec. PUID/PGID env vars are no longer read; if you need
@@ -167,6 +175,50 @@ services:
networks: networks:
- picpeak-network - picpeak-network
# Optional face-detection sidecar (#1074). Gated behind the `faces` profile
# so a plain `docker compose up -d` does NOT start it — opt in with
# `docker compose --profile faces up -d`. Nothing depends on it: the backend
# only ever calls it when the `faces` feature flag is on, so an install that
# skips this service behaves exactly as it did before the feature existed.
#
# The service name is `picpeak-ml` (not `ml`) because it doubles as the
# hostname in FACE_ML_URL's default, `http://picpeak-ml:8000`. Renaming this
# service silently breaks that default for every install that never set the
# variable.
#
# Requires FACENET_ONNX_URL / FACENET_ONNX_SHA256 at build time — see
# ml/README.md. The image publishes no host port and mounts no volumes; it
# is reachable only from the backend on picpeak-network.
picpeak-ml:
build:
context: ./ml
dockerfile: Dockerfile
args:
# Defaults live in ml/Dockerfile and point at the canonical published
# model. These pass an override through from .env when set; an empty
# value here would BLANK the Dockerfile default and fail the build,
# so the fallbacks repeat it deliberately.
- FACENET_ONNX_URL=${FACENET_ONNX_URL:-https://github.com/PicPeak/picpeak/releases/download/ml-models-v1/facenet512.onnx}
- FACENET_ONNX_SHA256=${FACENET_ONNX_SHA256:-a1c06dcb79dc17a42af01d5bcbce4822caa148b9c24bf7eb8b8e556b4fd0d5db}
container_name: picpeak-ml
restart: unless-stopped
profiles:
- faces
environment:
# Shared secret with the backend. The container refuses to start
# without it rather than serving anonymously.
- FACE_ML_TOKEN=${FACE_ML_TOKEN:-}
- FACE_ORT_THREADS=${FACE_ORT_THREADS:-1}
- TZ=${TZ:-UTC}
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=4).status == 200 else 1)"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
networks:
- picpeak-network
frontend: frontend:
build: build:
context: ./frontend context: ./frontend
+90
View File
@@ -0,0 +1,90 @@
"""Audit i18n coverage for the face-recognition feature (#1074).
Extracts every t('key', ...) used by the face components and reports which
are missing from each locale file. A key that resolves only via its inline
`defaultValue` renders ENGLISH to a German user which is exactly the gap
this looks for, and which nothing else in the toolchain would flag.
"""
import json
import re
import sys
from pathlib import Path
ROOT = Path('/Users/paul/Development/picpeak/frontend/src')
FILES = [
'components/gallery/PeopleStrip.tsx',
'components/gallery/PeopleSheet.tsx',
'components/gallery/GalleryView.tsx',
'components/gallery/PhotoLightbox.tsx',
'components/admin/FaceRecognitionCard.tsx',
'components/admin/PeopleManagerModal.tsx',
'features/settings/tabs/FeaturesTab.tsx',
]
# Only keys belonging to this feature (plus shared keys the new components use).
RELEVANT = re.compile(r'^(gallery\.people\.|admin\.people\.|admin\.faces\.|settings\.features\.faces\.|common\.)')
KEY_RE = re.compile(r"\bt\(\s*'([a-zA-Z0-9_.]+)'")
def load(lang):
with open(ROOT / 'i18n' / 'locales' / f'{lang}.json') as fh:
return json.load(fh)
def has(data, dotted):
node = data
for part in dotted.split('.'):
if not isinstance(node, dict) or part not in node:
return False
node = node[part]
return isinstance(node, str)
used = {}
for rel in FILES:
path = ROOT / rel
if not path.exists():
print(f'!! missing file {rel}')
continue
for key in KEY_RE.findall(path.read_text()):
if RELEVANT.match(key):
used.setdefault(key, set()).add(rel.split('/')[-1])
print(f'{len(used)} face-related keys in use\n')
exit_code = 0
for lang in ('en', 'de'):
data = load(lang)
missing = sorted(k for k in used if not has(data, k))
status = 'COMPLETE' if not missing else f'{len(missing)} MISSING'
print(f'--- {lang.upper()}: {status} ---')
for k in missing:
print(f' {k} ({", ".join(sorted(used[k]))})')
if missing:
exit_code = 1
print()
# Also flag DE values that are byte-identical to EN — usually an untranslated
# copy-paste rather than a word that genuinely matches in both languages.
en, de = load('en'), load('de')
def get(data, dotted):
node = data
for part in dotted.split('.'):
node = node[part]
return node
same = []
for k in sorted(used):
if has(en, k) and has(de, k) and get(en, k) == get(de, k):
same.append((k, get(en, k)))
if same:
print(f'--- DE identical to EN ({len(same)}) — check each is intentional ---')
for k, v in same:
print(f' {k} = {v!r}')
sys.exit(exit_code)
@@ -0,0 +1,357 @@
/**
* <FaceRecognitionCard>
*
* Per-event "People in this gallery" controls (#1074). Only rendered when the
* `faces` feature flag is on see OverviewTab.
*
* Two toggles, deliberately separate:
* - Detect people: the actual processing switch. Off by default.
* - Show to guests: whether the people bar reaches the gallery. On by
* default once detection is on, but a photographer may want clustering
* as a private sorting tool without changing what clients see.
*
* Face embeddings are biometric data (GDPR Art. 9 in the EU, where much of
* this user base operates). The consent paragraph is not boilerplate we
* provide the switch, the photographer provides the lawful basis so it
* renders next to the toggle rather than behind a "learn more".
*/
import React, { useEffect, useState } from 'react';
import { useQuery } from '@tanstack/react-query';
import { useTranslation } from 'react-i18next';
import { toast } from 'react-toastify';
import { Users, RefreshCw, Trash2, AlertTriangle, ShieldCheck, SlidersHorizontal } from 'lucide-react';
import { PeopleManagerModal } from './PeopleManagerModal';
import { Button, Card, Loading } from '../common';
import { api } from '../../config/api';
interface FacesPayload {
enabled: boolean;
visible_to_guests: boolean;
last_scan_at: string | null;
status: {
scanned: number;
total: number;
pending: number;
failed: number;
people: number;
// What guests actually see: the minimum-cluster-size floor and the
// hidden/ignored flags applied. Usually lower than `people`.
people_visible_to_guests?: number;
in_progress: boolean;
};
}
interface FaceRecognitionCardProps {
eventId: number;
isArchived?: boolean;
}
export const FaceRecognitionCard: React.FC<FaceRecognitionCardProps> = ({ eventId, isArchived }) => {
const { t } = useTranslation();
const [saving, setSaving] = useState(false);
const [managerOpen, setManagerOpen] = useState(false);
const [autoCategories, setAutoCategories] = useState(false);
const { data, isLoading, refetch } = useQuery<FacesPayload>({
queryKey: ['admin-event-faces', eventId],
queryFn: async () => (await api.get(`/admin/events/${eventId}/faces`)).data,
// Poll while a scan is running so the status line advances without a
// manual refresh; stop entirely once it settles.
refetchInterval: (query) => (query.state.data?.status?.in_progress ? 5000 : false),
});
useEffect(() => {
if (!data?.enabled) return;
api.get('/admin/events/faces/auto-categories')
.then((r) => setAutoCategories(!!r.data?.enabled))
.catch(() => { /* pre-migration or no permission — leave off */ });
}, [data?.enabled]);
const patch = async (body: Record<string, boolean>) => {
setSaving(true);
try {
const { data: result } = await api.patch(`/admin/events/${eventId}/faces`, body);
await refetch();
if (result?.queued) {
toast.success(t('admin.faces.queued', {
count: result.queued,
defaultValue: `Scanning ${result.queued} photos for people…`,
}));
} else {
toast.success(t('common.saved', { defaultValue: 'Saved' }));
}
} catch (err: any) {
toast.error(err?.response?.data?.error || t('common.saveFailed', { defaultValue: 'Save failed' }));
} finally {
setSaving(false);
}
};
const action = async (path: string, successKey: string, fallback: string) => {
setSaving(true);
try {
await api.post(`/admin/events/${eventId}/faces/${path}`);
await refetch();
toast.success(t(successKey, { defaultValue: fallback }));
} catch (err: any) {
toast.error(err?.response?.data?.error || fallback);
} finally {
setSaving(false);
}
};
const purge = async () => {
// Irreversible and covers biometric data — a plain confirm is the least
// this deserves.
const ok = window.confirm(t('admin.faces.confirmDelete', {
defaultValue: 'Delete all detected people and face data for this gallery? This cannot be undone. Photos are not affected.',
}));
if (!ok) return;
setSaving(true);
try {
await api.delete(`/admin/events/${eventId}/faces`);
await refetch();
toast.success(t('admin.faces.deleted', { defaultValue: 'Face data deleted' }));
} catch (err: any) {
toast.error(err?.response?.data?.error || 'Delete failed');
} finally {
setSaving(false);
}
};
if (isLoading) {
return <Card><Loading /></Card>;
}
if (!data) return null;
const { status } = data;
return (
<Card>
<div className="flex items-start gap-3 mb-4">
<Users className="text-neutral-400 mt-0.5" size={20} />
<div>
<h3 className="text-lg font-medium text-neutral-900">
{t('admin.faces.title', { defaultValue: 'People in this gallery' })}
</h3>
<p className="text-sm text-neutral-500">
{t('admin.faces.subtitle', {
defaultValue: 'Group photos by the people in them, so guests can find and download their own.',
})}
</p>
</div>
</div>
{/* Consent obligation. Stated plainly and up front, because by the time
someone has switched this on they have already processed the data. */}
<div className="flex gap-2 p-3 mb-4 rounded-lg bg-amber-50 border border-amber-100 text-sm text-amber-900">
<ShieldCheck size={16} className="flex-shrink-0 mt-0.5" />
<p>
{t('admin.faces.consentNotice', {
defaultValue:
'Detected faces are personal data, and in the EU they count as a special category. You are the controller for this gallery: make sure you have a lawful basis for the people in these photos before switching this on. Nothing leaves your server — detection runs in your own container.',
})}
</p>
</div>
<label className="flex items-start gap-3 py-2 cursor-pointer">
<input
type="checkbox"
checked={data.enabled}
disabled={saving || isArchived}
onChange={(e) => patch({ enabled: e.target.checked })}
className="mt-1 rounded border-neutral-300 text-primary-600 focus:ring-primary-500"
/>
<span>
<span className="block text-sm font-medium text-neutral-800">
{t('admin.faces.enable', { defaultValue: 'Detect people in this gallery' })}
</span>
<span className="block text-xs text-neutral-500">
{t('admin.faces.enableHint', {
defaultValue: 'Existing photos are scanned in the background. Faces and their numeric signatures are stored in your database; they are never included in backups or exports.',
})}
</span>
</span>
</label>
{data.enabled && (
<label className="flex items-start gap-3 py-2 cursor-pointer">
<input
type="checkbox"
checked={data.visible_to_guests}
disabled={saving || isArchived}
onChange={(e) => patch({ visible_to_guests: e.target.checked })}
className="mt-1 rounded border-neutral-300 text-primary-600 focus:ring-primary-500"
/>
<span>
<span className="block text-sm font-medium text-neutral-800">
{t('admin.faces.visible', { defaultValue: 'Show the people bar to guests' })}
</span>
<span className="block text-xs text-neutral-500">
{t('admin.faces.visibleHint', {
defaultValue: 'Off means you get the grouping as a private tool and guests see an unchanged gallery.',
})}
</span>
</span>
</label>
)}
{/* The preview-tier cost. Scanning generates the lightbox preview for
every photo that lacks one, which is real CPU and real disk an
admin deserves to know that before starting a 2,000-photo backfill
rather than discovering it in their storage graph. */}
{data.enabled && (
<div className="flex gap-2 p-3 mt-3 rounded-lg bg-neutral-50 text-xs text-neutral-600">
<AlertTriangle size={14} className="flex-shrink-0 mt-0.5 text-neutral-400" />
<p>
{t('admin.faces.previewNotice', {
defaultValue: 'Scanning works on the preview-sized copy of each photo. Galleries that have not generated previews yet will create them during the first scan, which uses additional CPU and disk space.',
})}
</p>
</div>
)}
{/* Auto-categories (#1074 phase 3). Global, not per-event, which is why
it sits apart from the two toggles above. Without a control here the
rule engine had no way to be switched on at all. */}
{data.enabled && (
<label className="flex items-start gap-3 py-2 mt-2 pt-3 border-t border-neutral-100 cursor-pointer">
<input
type="checkbox"
checked={autoCategories}
disabled={saving}
onChange={async (e) => {
const next = e.target.checked;
setSaving(true);
try {
await api.put('/admin/events/faces/auto-categories', { enabled: next });
setAutoCategories(next);
toast.success(t('common.saved', { defaultValue: 'Saved' }));
} catch (err: any) {
toast.error(err?.response?.data?.error || t('common.saveFailed', { defaultValue: 'Save failed' }));
} finally {
setSaving(false);
}
}}
className="mt-1 rounded border-neutral-300 text-primary-600 focus:ring-primary-500"
/>
<span>
<span className="block text-sm font-medium text-neutral-800">
{t('admin.faces.autoCategories', { defaultValue: 'Sort photos into categories automatically' })}
</span>
<span className="block text-xs text-neutral-500">
{t('admin.faces.autoCategoriesHint', {
defaultValue: 'Uses the number of faces to file photos as Details, Portraits, Small groups or Groups. Applies to every gallery, only ever fills an empty category, and never changes one you set yourself.',
})}
</span>
</span>
</label>
)}
{data.enabled && (
<>
<div className="mt-4 pt-4 border-t border-neutral-100 text-sm text-neutral-600">
{status.in_progress ? (
<p className="flex items-center gap-2">
<RefreshCw size={14} className="animate-spin text-primary-500" />
{t('admin.faces.scanning', {
scanned: status.scanned,
total: status.total,
defaultValue: `Scanning… ${status.scanned} of ${status.total} photos`,
})}
</p>
) : (
<p>
{t('admin.faces.status', {
scanned: status.scanned,
total: status.total,
people: status.people,
defaultValue: `${status.scanned} / ${status.total} photos scanned · ${status.people} people`,
})}
{/* The gallery shows fewer: one-off appearances stay out of
the strip. Without both numbers an admin sees the settings
page and their own gallery disagree, with no explanation. */}
{typeof status.people_visible_to_guests === 'number'
&& status.people_visible_to_guests !== status.people && (
<span className="text-neutral-400">
{' '}
{t('admin.faces.visibleToGuests', {
count: status.people_visible_to_guests,
defaultValue: `(${status.people_visible_to_guests} shown to guests)`,
})}
</span>
)}
{status.failed > 0 && (
<span className="text-amber-600">
{' · '}
{t('admin.faces.failed', {
count: status.failed,
defaultValue: `${status.failed} failed`,
})}
</span>
)}
</p>
)}
</div>
<div className="flex flex-wrap gap-2 mt-4">
<Button
variant="outline"
size="sm"
disabled={saving}
onClick={() => setManagerOpen(true)}
leftIcon={<SlidersHorizontal size={14} />}
>
{t('admin.faces.manage', { defaultValue: 'Manage people' })}
</Button>
<Button
variant="outline"
size="sm"
disabled={saving || isArchived}
onClick={() => action('rescan', 'admin.faces.rescanQueued', 'Re-scan queued')}
leftIcon={<RefreshCw size={14} />}
>
{t('admin.faces.rescan', { defaultValue: 'Re-scan' })}
</Button>
{/* Cheap re-derives people from data we already have, with no
sidecar call. The button to reach for after changing the
match threshold. */}
<Button
variant="outline"
size="sm"
disabled={saving || isArchived}
onClick={() => action('recluster', 'admin.faces.reclustered', 'People regrouped')}
leftIcon={<Users size={14} />}
>
{t('admin.faces.recluster', { defaultValue: 'Re-group people' })}
</Button>
<Button
variant="outline"
size="sm"
disabled={saving}
onClick={purge}
leftIcon={<Trash2 size={14} />}
className="text-red-600 border-red-200 hover:bg-red-50"
>
{t('admin.faces.delete', { defaultValue: 'Delete all face data' })}
</Button>
</div>
<PeopleManagerModal
eventId={eventId}
open={managerOpen}
onClose={() => setManagerOpen(false)}
onChanged={() => refetch()}
/>
</>
)}
</Card>
);
};
export default FaceRecognitionCard;
@@ -0,0 +1,436 @@
/**
* <PeopleManagerModal> manage the people detected in one gallery (#1074).
*
* Merge and split are the two operations that make automatic clustering
* survive contact with reality. A well-behaved wedding gallery still produces
* "Anna in daylight" and "Anna at the party" as separate people, and the
* clustering deliberately errs toward splitting rather than merging (a
* duplicate entry is an annoyance; a wrong merge puts a stranger into
* someone's download). That trade only works if merging is easy, which is
* what this screen is for.
*
* Every action here hits an endpoint that already re-checks event ownership
* and that the ids belong to this gallery the UI is a convenience, never
* the control.
*/
import React, { useMemo, useState } from 'react';
import { useQuery } from '@tanstack/react-query';
import { useTranslation } from 'react-i18next';
import { toast } from 'react-toastify';
import { X, Check, Merge, Scissors, EyeOff, Ban, Loader2 } from 'lucide-react';
import { Button, Loading } from '../common';
import { api } from '../../config/api';
import { faceCropStyle } from '../gallery/faceCrop';
interface AdminPerson {
id: number;
label: string | null;
face_count: number;
total_face_count?: number;
is_hidden?: boolean;
is_ignored?: boolean;
cover: {
face_id: number;
photo_id: number;
bbox: [number, number, number, number];
photo_width: number | null;
photo_height: number | null;
} | null;
}
interface PersonFace {
id: number;
photo_id: number;
bbox: [number, number, number, number];
photo_width: number | null;
photo_height: number | null;
score: number | null;
blur: number | null;
}
interface PeopleManagerModalProps {
eventId: number;
open: boolean;
onClose: () => void;
onChanged?: () => void;
}
/**
* Square crop of a face, served by the admin thumbnail route (which applies
* its own auth + event-ownership checks).
*
* A plain <img>, deliberately NOT <AuthenticatedImage>. That component
* attaches whatever gallery token it finds in session storage, and an admin
* who has also opened one of their own galleries in the same browser then
* sends a `type: "gallery"` bearer token to an admin route which rejects
* it as insufficient permissions, so every thumbnail 403s. The admin routes
* authenticate from the httpOnly admin_token cookie, which a same-origin
* <img> sends on its own.
*/
const FaceThumb: React.FC<{
eventId: number;
photoId: number;
bbox?: [number, number, number, number] | null;
photoWidth?: number | null;
photoHeight?: number | null;
size?: number;
dim?: boolean;
}> = ({ eventId, photoId, bbox, photoWidth, photoHeight, size = 64, dim }) => {
// Crop to the face rather than showing the centred thumbnail. On a group
// photo the uncropped version shows whoever stands in the middle, so two
// different people whose cover is the same photo looked IDENTICAL here —
// in the manager whose whole job is telling faces apart.
//
// COORDINATE SPACE: the bbox is in ORIGINAL image pixels, so it must be
// scaled against the ORIGINAL dimensions (now supplied by the API), never
// against the thumbnail's own natural size. Mixing the two renders the
// wrong region entirely — the box ends up a fraction of its true size and
// offset toward the top-left.
const style = faceCropStyle(bbox ? { bbox } : null, photoWidth, photoHeight, size);
return (
<span
className="relative block rounded-full overflow-hidden bg-neutral-100 flex-shrink-0"
style={{ width: size, height: size, opacity: dim ? 0.4 : 1 }}
>
<img
src={`/api/admin/photos/${eventId}/thumbnail/${photoId}`}
alt=""
loading="lazy"
style={style || { width: '100%', height: '100%', objectFit: 'cover' }}
/>
</span>
);
};
export const PeopleManagerModal: React.FC<PeopleManagerModalProps> = ({
eventId, open, onClose, onChanged,
}) => {
const { t } = useTranslation();
const [selected, setSelected] = useState<number[]>([]);
const [renaming, setRenaming] = useState<number | null>(null);
const [draftLabel, setDraftLabel] = useState('');
const [splitting, setSplitting] = useState<AdminPerson | null>(null);
const [splitFaceIds, setSplitFaceIds] = useState<number[]>([]);
const [busy, setBusy] = useState(false);
const { data, isLoading, refetch } = useQuery<{ people: AdminPerson[] }>({
queryKey: ['admin-event-people', eventId],
queryFn: async () => (await api.get(`/admin/events/${eventId}/people`)).data,
enabled: open,
});
const { data: faceData, isLoading: facesLoading } = useQuery<{ faces: PersonFace[] }>({
queryKey: ['admin-person-faces', eventId, splitting?.id],
queryFn: async () =>
(await api.get(`/admin/events/${eventId}/people/${splitting!.id}/faces`)).data,
enabled: !!splitting,
});
const people = useMemo(() => data?.people || [], [data]);
const after = async (message: string) => {
await refetch();
onChanged?.();
setSelected([]);
toast.success(message);
};
const run = async (fn: () => Promise<void>, successMessage: string) => {
setBusy(true);
try {
await fn();
await after(successMessage);
} catch (err: any) {
toast.error(err?.response?.data?.error || t('common.saveFailed', { defaultValue: 'Failed' }));
} finally {
setBusy(false);
}
};
const toggleSelect = (id: number) => {
setSelected((prev) => (prev.includes(id) ? prev.filter((x) => x !== id) : [...prev, id]));
};
const saveLabel = (person: AdminPerson) => {
const label = draftLabel.trim();
setRenaming(null);
if (label === (person.label || '')) return;
run(
async () => {
await api.patch(`/admin/events/${eventId}/people/${person.id}`, { label: label || null });
},
t('admin.people.renamed', { defaultValue: 'Name saved' })
);
};
const doMerge = () => {
// The first selected person is the target — it keeps its name, which is
// almost always the one the photographer already bothered to type.
const [target, ...sources] = selected;
run(
async () => {
await api.post(`/admin/events/${eventId}/people/merge`, {
source_ids: sources, target_id: target,
});
},
t('admin.people.merged', { count: sources.length, defaultValue: 'People merged' })
);
};
const doSplit = () => {
if (!splitting || !splitFaceIds.length) return;
const personId = splitting.id;
const ids = splitFaceIds;
setSplitting(null);
setSplitFaceIds([]);
run(
async () => {
await api.post(`/admin/events/${eventId}/people/${personId}/split`, { face_ids: ids });
},
t('admin.people.split', { defaultValue: 'Split into a new person' })
);
};
const setFlag = (person: AdminPerson, field: 'is_hidden' | 'is_ignored', value: boolean) =>
run(
async () => {
await api.patch(`/admin/events/${eventId}/people/${person.id}`, { [field]: value });
},
t('admin.people.updated', { defaultValue: 'Updated' })
);
if (!open) return null;
return (
<div className="fixed inset-0 z-50 flex items-center justify-center p-4">
<div className="absolute inset-0 bg-black/50" onClick={onClose} aria-hidden="true" />
<div
role="dialog"
aria-modal="true"
className="relative bg-white text-neutral-900 rounded-xl shadow-xl w-full max-w-4xl max-h-[88vh] flex flex-col"
>
<div className="flex items-center justify-between px-5 py-4 border-b border-neutral-100">
<div>
<h2 className="text-lg font-medium text-neutral-900">
{t('admin.people.title', { defaultValue: 'People in this gallery' })}
</h2>
<p className="text-sm text-neutral-500">
{t('admin.people.subtitle', {
defaultValue: 'Rename, merge people who were split apart, or hide someone from guests.',
})}
</p>
</div>
<button type="button" onClick={onClose} className="p-2 -m-2 text-neutral-400 hover:text-neutral-600">
<X size={20} />
</button>
</div>
{/* --- split picker ------------------------------------------------ */}
{splitting ? (
<>
<div className="px-5 py-3 bg-amber-50 border-b border-amber-100 text-sm text-amber-900">
{t('admin.people.splitHelp', {
defaultValue: 'Pick the photos that are NOT this person. They become a new entry, and everything else stays.',
})}
</div>
<div className="flex-1 overflow-y-auto p-5">
{facesLoading ? <Loading /> : (
<div className="grid grid-cols-4 sm:grid-cols-6 gap-3">
{(faceData?.faces || []).map((face) => {
const picked = splitFaceIds.includes(face.id);
return (
<button
key={face.id}
type="button"
onClick={() => setSplitFaceIds((p) =>
p.includes(face.id) ? p.filter((x) => x !== face.id) : [...p, face.id])}
className={`relative rounded-lg overflow-hidden border-2 transition-colors ${
picked ? 'border-primary-600' : 'border-transparent hover:border-neutral-300'
}`}
>
<FaceThumb
eventId={eventId}
photoId={face.photo_id}
bbox={face.bbox}
photoWidth={face.photo_width}
photoHeight={face.photo_height}
size={88}
/>
{picked && (
<span className="absolute top-1 right-1 bg-primary-600 text-white rounded-full p-0.5">
<Check size={12} />
</span>
)}
</button>
);
})}
</div>
)}
</div>
<div className="flex items-center justify-between gap-2 px-5 py-4 border-t border-neutral-100">
<span className="text-sm text-neutral-500">
{t('admin.people.splitSelected', {
count: splitFaceIds.length,
defaultValue: `${splitFaceIds.length} selected`,
})}
</span>
<div className="flex gap-2">
<Button variant="outline" size="sm" onClick={() => { setSplitting(null); setSplitFaceIds([]); }}>
{t('common.cancel', { defaultValue: 'Cancel' })}
</Button>
<Button variant="primary" size="sm" disabled={!splitFaceIds.length || busy} onClick={doSplit}>
{t('admin.people.doSplit', { defaultValue: 'Split out' })}
</Button>
</div>
</div>
</>
) : (
<>
{/* --- people grid --------------------------------------------- */}
<div className="flex-1 overflow-y-auto p-5">
{isLoading ? <Loading /> : people.length === 0 ? (
<p className="text-sm text-neutral-500 text-center py-10">
{t('admin.people.empty', { defaultValue: 'No people detected yet.' })}
</p>
) : (
<div className="space-y-1">
{people.map((person) => {
const isSelected = selected.includes(person.id);
return (
<div
key={person.id}
className={`flex items-center gap-3 p-2 rounded-lg border transition-colors ${
isSelected ? 'border-primary-400 bg-primary-50' : 'border-transparent hover:bg-neutral-50'
}`}
>
<button
type="button"
onClick={() => toggleSelect(person.id)}
aria-pressed={isSelected}
aria-label={t('admin.people.select', { defaultValue: 'Select for merging' })}
className="flex-shrink-0"
>
<FaceThumb
eventId={eventId}
photoId={person.cover?.photo_id ?? 0}
bbox={person.cover?.bbox}
photoWidth={person.cover?.photo_width}
photoHeight={person.cover?.photo_height}
dim={person.is_ignored || person.is_hidden}
/>
</button>
<div className="flex-1 min-w-0">
{renaming === person.id ? (
<input
autoFocus
value={draftLabel}
onChange={(e) => setDraftLabel(e.target.value)}
onBlur={() => saveLabel(person)}
onKeyDown={(e) => {
if (e.key === 'Enter') saveLabel(person);
if (e.key === 'Escape') setRenaming(null);
}}
placeholder={t('admin.people.namePlaceholder', { defaultValue: 'Add a name' })}
className="w-full max-w-xs px-2 py-1 text-sm border border-neutral-300 rounded focus:outline-none focus:ring-2 focus:ring-primary-500"
/>
) : (
<button
type="button"
onClick={() => { setRenaming(person.id); setDraftLabel(person.label || ''); }}
className="text-sm text-left text-neutral-900 hover:underline"
>
{person.label || (
<span className="text-neutral-400 italic">
{t('admin.people.unnamed', { defaultValue: 'Add a name' })}
</span>
)}
</button>
)}
<p className="text-xs text-neutral-500">
{t('admin.people.photoCount', {
count: person.total_face_count ?? person.face_count,
defaultValue: `${person.total_face_count ?? person.face_count} photos`,
})}
{person.is_hidden && ` · ${t('admin.people.hidden', { defaultValue: 'hidden from guests' })}`}
{person.is_ignored && ` · ${t('admin.people.ignored', { defaultValue: 'ignored' })}`}
</p>
</div>
<div className="flex items-center gap-1 flex-shrink-0">
<button
type="button"
disabled={busy}
title={t('admin.people.splitAction', { defaultValue: 'Split out photos that are someone else' })}
onClick={() => { setSplitting(person); setSplitFaceIds([]); }}
className="p-2 text-neutral-400 hover:text-neutral-700 rounded"
>
<Scissors size={16} />
</button>
<button
type="button"
disabled={busy}
title={t('admin.people.hideAction', { defaultValue: 'Hide from guests' })}
onClick={() => setFlag(person, 'is_hidden', !person.is_hidden)}
className={`p-2 rounded ${person.is_hidden ? 'text-primary-600' : 'text-neutral-400 hover:text-neutral-700'}`}
>
<EyeOff size={16} />
</button>
<button
type="button"
disabled={busy}
title={t('admin.people.ignoreAction', { defaultValue: 'Not a real person — ignore' })}
onClick={() => setFlag(person, 'is_ignored', !person.is_ignored)}
className={`p-2 rounded ${person.is_ignored ? 'text-red-600' : 'text-neutral-400 hover:text-neutral-700'}`}
>
<Ban size={16} />
</button>
</div>
</div>
);
})}
</div>
)}
</div>
{/* Merge only becomes available at two, and the wording names the
target explicitly so nobody has to guess which name survives. */}
<div className="flex items-center justify-between gap-3 px-5 py-4 border-t border-neutral-100">
<span className="text-sm text-neutral-500">
{selected.length > 0
? t('admin.people.selectedCount', {
count: selected.length,
defaultValue: `${selected.length} selected`,
})
: t('admin.people.mergeHint', {
defaultValue: 'Tap two or more faces to merge them into one person.',
})}
</span>
<div className="flex gap-2">
{selected.length > 0 && (
<Button variant="outline" size="sm" onClick={() => setSelected([])}>
{t('common.clear', { defaultValue: 'Clear' })}
</Button>
)}
<Button
variant="primary"
size="sm"
disabled={selected.length < 2 || busy}
onClick={doMerge}
leftIcon={busy ? <Loader2 className="w-4 h-4 animate-spin" /> : <Merge className="w-4 h-4" />}
>
{t('admin.people.merge', { defaultValue: 'Merge' })}
</Button>
</div>
</div>
</>
)}
</div>
</div>
);
};
export default PeopleManagerModal;
@@ -20,6 +20,8 @@ interface SystemVersion {
node: string; node: string;
environment: string; environment: string;
channel?: 'stable' | 'beta'; channel?: 'stable' | 'beta';
// True on the all-in-one image; some features are unavailable there.
single_container?: boolean;
} }
interface UpdateInfo { interface UpdateInfo {
+241 -2
View File
@@ -17,12 +17,14 @@ import { PhotoFilterBar } from './PhotoFilterBar';
import { UserPhotoUpload } from './UserPhotoUpload'; import { UserPhotoUpload } from './UserPhotoUpload';
import { GuestNamePromptModal } from './GuestNamePromptModal'; import { GuestNamePromptModal } from './GuestNamePromptModal';
import { GuestRecoveryModal } from './GuestRecoveryModal'; import { GuestRecoveryModal } from './GuestRecoveryModal';
import { PeopleStrip } from './PeopleStrip';
import { PeopleSheet } from './PeopleSheet';
import { GuestIdentityProvider } from '../../contexts/GuestIdentityContext'; import { GuestIdentityProvider } from '../../contexts/GuestIdentityContext';
import type { FilterType, FeedbackFilterType } from './GalleryFilter'; import type { FilterType, FeedbackFilterType } from './GalleryFilter';
import { analyticsService } from '../../services/analytics.service'; import { analyticsService } from '../../services/analytics.service';
import { useDevToolsProtection } from '../../hooks/useDevToolsProtection'; import { useDevToolsProtection } from '../../hooks/useDevToolsProtection';
import { api } from '../../config/api'; import { api } from '../../config/api';
import { Upload, Menu, Eye, EyeOff, Shield } from 'lucide-react'; import { Upload, Menu, Eye, EyeOff, Shield, X, Download } from 'lucide-react';
import { galleryService } from '../../services/gallery.service'; import { galleryService } from '../../services/gallery.service';
import { feedbackService } from '../../services/feedback.service'; import { feedbackService } from '../../services/feedback.service';
import { useWatermarkSettings } from '../../hooks/useWatermarkSettings'; import { useWatermarkSettings } from '../../hooks/useWatermarkSettings';
@@ -104,6 +106,41 @@ export const GalleryView: React.FC<GalleryViewProps> = ({ slug, event }) => {
// Multi-select feedback filters (#889): OR-combined; empty = "All". // Multi-select feedback filters (#889): OR-combined; empty = "All".
// Clicking a filter toggles it, clicking "All" clears the set. // Clicking a filter toggles it, clicking "All" clears the set.
const [activeFilters, setActiveFilters] = useState<FeedbackFilterType[]>([]); const [activeFilters, setActiveFilters] = useState<FeedbackFilterType[]>([]);
// People filter (#1074). Multi-select, AND by default — see the filter
// block below. `peopleMatchAny` only becomes reachable once a second
// person is picked, since the toggle is meaningless for one.
const [selectedPersonIds, setSelectedPersonIds] = useState<number[]>([]);
const [peopleMatchAny, setPeopleMatchAny] = useState(false);
const [showPeopleSheet, setShowPeopleSheet] = useState(false);
// Dismissal is per gallery: a guest who hides the bar in one gallery has
// said nothing about the next one.
const [peopleCollapsed, setPeopleCollapsed] = useState<boolean>(() => {
try {
return localStorage.getItem(`picpeak_people_collapsed_${slug}`) === '1';
} catch {
return false;
}
});
const handlePeopleCollapsedChange = (collapsed: boolean) => {
setPeopleCollapsed(collapsed);
try {
localStorage.setItem(`picpeak_people_collapsed_${slug}`, collapsed ? '1' : '0');
} catch {
// Private-mode Safari throws on setItem; the in-memory state still works.
}
};
const togglePerson = (personId: number) => {
setSelectedPersonIds((prev) => {
const next = prev.includes(personId)
? prev.filter((id) => id !== personId)
: [...prev, personId];
// Dropping back below two people makes the any/all toggle meaningless;
// reset it so it doesn't silently persist into the next selection.
if (next.length < 2) setPeopleMatchAny(false);
return next;
});
};
const handleFilterChange = (filter: FilterType) => { const handleFilterChange = (filter: FilterType) => {
if (filter === 'all') { if (filter === 'all') {
setActiveFilters([]); setActiveFilters([]);
@@ -263,6 +300,41 @@ export const GalleryView: React.FC<GalleryViewProps> = ({ slug, event }) => {
enabled: !!event.id, enabled: !!event.id,
}); });
// People in this gallery (#1074).
//
// Gated on people_enabled so an install without the feature never fires the
// request at all. Polls only while a backfill is running — a finished
// gallery has a stable people list, and polling it forever would be a
// request per guest per interval for no new information.
// From the /photos payload, not the prop: the prop's event shape comes
// from /info, which does not carry this flag.
const peopleEnabled = data?.event?.people_enabled === true;
const { data: peopleData } = useQuery({
queryKey: ['gallery-people', slug],
queryFn: () => galleryService.getPeople(slug),
enabled: peopleEnabled,
refetchInterval: (query) => (query.state.data?.scan?.in_progress ? 5000 : false),
staleTime: 30_000,
});
const people = peopleData?.people || [];
// The strip comes from /people, but FILTERING uses photo.person_ids, which
// rides on the one-shot /photos response. During a backfill those drift
// apart: new faces appear in the strip while the photo memberships behind
// them are still the set fetched on page load, so tapping a person yields
// zero or a partial result until a manual reload — including after the scan
// has finished.
//
// Refetch the photos whenever the scan's progress changes, and once more on
// the transition to finished.
const scanProgress = peopleData?.scan
? `${peopleData.scan.in_progress}:${peopleData.scan.scanned}`
: null;
useEffect(() => {
if (!peopleEnabled || !scanProgress) return;
queryClient.invalidateQueries({ queryKey: ['gallery-photos', slug] });
}, [scanProgress, peopleEnabled, slug, queryClient]);
// Update feedbackEnabled when settings change // Update feedbackEnabled when settings change
useEffect(() => { useEffect(() => {
if (feedbackSettings) { if (feedbackSettings) {
@@ -552,6 +624,21 @@ export const GalleryView: React.FC<GalleryViewProps> = ({ slug, event }) => {
photos = photos.filter(photo => activeFilters.some(filter => matchers[filter](photo))); photos = photos.filter(photo => activeFilters.some(filter => matchers[filter](photo)));
} }
// Apply people filter (#1074). Composes with every filter above rather
// than replacing them, so "photos of Anna that I liked" works.
//
// Two people selected means AND by default ("photos with both Anna and
// Ben") — that is what someone picking a second face is almost always
// asking for. `peopleMatchAny` flips it to OR for the couple-shots case.
if (selectedPersonIds.length > 0) {
photos = photos.filter(photo => {
const ids = photo.person_ids || [];
return peopleMatchAny
? selectedPersonIds.some(id => ids.includes(id))
: selectedPersonIds.every(id => ids.includes(id));
});
}
// Apply sorting // Apply sorting
// Each comparator defaults to its natural order (desc for dates/size/rating, asc for name). // Each comparator defaults to its natural order (desc for dates/size/rating, asc for name).
// The flip multiplier reverses that when sortDesc differs from the natural order. // The flip multiplier reverses that when sortDesc differs from the natural order.
@@ -592,7 +679,7 @@ export const GalleryView: React.FC<GalleryViewProps> = ({ slug, event }) => {
} }
return photos; return photos;
}, [data?.photos, selectedCategoryId, searchTerm, sortBy, sortDesc, watermarkEnabled, slug, activeFilters, mediaFilter, isGuestIdentityMode, myFeedbackPhotoIds]); }, [data?.photos, selectedCategoryId, searchTerm, sortBy, sortDesc, watermarkEnabled, slug, activeFilters, mediaFilter, isGuestIdentityMode, myFeedbackPhotoIds, selectedPersonIds, peopleMatchAny]);
// Counts shown in the filter chips ("Liked (N)", etc.). In guest // Counts shown in the filter chips ("Liked (N)", etc.). In guest
// mode these need to mirror the per-guest filter behaviour above — // mode these need to mirror the per-guest filter behaviour above —
@@ -682,6 +769,40 @@ export const GalleryView: React.FC<GalleryViewProps> = ({ slug, event }) => {
setIsSelectionMode(false); setIsSelectionMode(false);
}; };
// "Download these N" (#1074) — the payoff of the people filter.
//
// Deliberately NO new endpoint or person_id selector: the filtered photo
// ids go through the same path as a manual selection, and the server
// re-applies the access level and per-category permissions on the way
// through. One less thing to authorize.
//
// Photos in a category with downloads disabled (#640) are excluded HERE as
// well as server-side, so the number on the button is the number the guest
// actually receives rather than an optimistic one.
const peopleDownloadableIds = useMemo(() => {
if (selectedPersonIds.length === 0) return [];
return filteredPhotos
.filter((photo) => photo.category_allow_downloads !== false)
.map((photo) => photo.id);
}, [filteredPhotos, selectedPersonIds]);
const handleDownloadPeopleFiltered = async () => {
if (!allowDownloads || peopleDownloadableIds.length === 0) return;
// Same resolution-picker behaviour as every other multi-photo download.
if (downloadChoices.length > 1) {
setResolutionPickerIds(peopleDownloadableIds);
return;
}
analyticsService.trackGalleryEvent('bulk_download', {
gallery: slug,
photo_count: peopleDownloadableIds.length,
});
await galleryService.downloadSelectedPhotos(slug, peopleDownloadableIds);
};
// Calculate photo counts per category // Calculate photo counts per category
const photoCounts = useMemo(() => { const photoCounts = useMemo(() => {
if (!data?.photos) return {}; if (!data?.photos) return {};
@@ -847,6 +968,8 @@ export const GalleryView: React.FC<GalleryViewProps> = ({ slug, event }) => {
<PhotoGridWithLayouts <PhotoGridWithLayouts
photos={filteredPhotos} photos={filteredPhotos}
slug={slug} slug={slug}
people={peopleEnabled ? people : undefined}
onSelectPerson={togglePerson}
categoryId={selectedCategoryId} categoryId={selectedCategoryId}
onFeedbackChange={() => { onFeedbackChange={() => {
refetch(); refetch();
@@ -1118,6 +1241,107 @@ export const GalleryView: React.FC<GalleryViewProps> = ({ slug, event }) => {
</div> </div>
) : null} ) : null}
{/* People in this gallery (#1074). Sits between the filter bar and
the grid. Renders nothing at all unless the photographer enabled
detection AND left it visible to guests people_enabled carries
both decisions plus the global feature flag. */}
{peopleEnabled && (
<div className="mt-4">
<PeopleStrip
people={people}
photos={data.photos}
slug={slug}
selectedPersonIds={selectedPersonIds}
onToggle={togglePerson}
onShowAll={() => setShowPeopleSheet(true)}
scan={peopleData?.scan}
collapsed={peopleCollapsed}
onCollapsedChange={handlePeopleCollapsedChange}
/>
{/* Active people filter. The chip row is the single place the
current selection is stated, so "why am I seeing 97 photos"
is always answerable at a glance. */}
{selectedPersonIds.length > 0 && (
<div
className="flex flex-wrap items-center gap-2 py-2 border-t"
style={{ borderColor: 'var(--color-surface-border)' }}
>
{selectedPersonIds.map((id) => {
const person = people.find((p) => p.id === id);
if (!person) return null;
return (
<button
key={id}
type="button"
onClick={() => togglePerson(id)}
className="inline-flex items-center gap-1.5 px-3 py-1 rounded-full bg-primary-50 text-primary-700 text-sm hover:bg-primary-100"
>
{person.label || t('gallery.people.unnamedCount', {
count: person.face_count,
defaultValue: `${person.face_count} photos`,
})}
<X size={14} />
</button>
);
})}
{/* Only meaningful with two or more people picked. */}
{selectedPersonIds.length > 1 && (
<button
type="button"
onClick={() => setPeopleMatchAny((v) => !v)}
className="px-2.5 py-1 rounded-full border text-xs"
style={{
color: 'var(--color-text)',
borderColor: 'var(--color-surface-border)',
}}
>
{peopleMatchAny
? t('gallery.people.matchAny', { defaultValue: 'Either person' })
: t('gallery.people.matchAll', { defaultValue: 'Both people' })}
</button>
)}
{/* ml-auto only once there's room for it at 390px the count
and Clear were pushed against the right edge and clipped. */}
<span className="text-sm sm:ml-auto" style={{ color: 'var(--color-muted-text)' }}>
{t('gallery.people.matchCount', {
count: filteredPhotos.length,
total: totalCount,
defaultValue: `${filteredPhotos.length} of ${totalCount} photos`,
})}
</span>
{/* Hidden entirely when downloads are off for the gallery,
rather than shown-and-failing. */}
{allowDownloads && peopleDownloadableIds.length > 0 && (
<Button
variant="primary"
size="sm"
onClick={handleDownloadPeopleFiltered}
leftIcon={<Download className="w-4 h-4" />}
>
{t('gallery.people.downloadThese', {
count: peopleDownloadableIds.length,
defaultValue: `Download these ${peopleDownloadableIds.length}`,
})}
</Button>
)}
<button
type="button"
onClick={() => { setSelectedPersonIds([]); setPeopleMatchAny(false); }}
className="text-sm underline"
style={{ color: 'var(--color-muted-text)' }}
>
{t('gallery.people.clear', { defaultValue: 'Clear' })}
</button>
</div>
)}
</div>
)}
{/* Photo Grid when the hero header sits directly under the filter {/* Photo Grid when the hero header sits directly under the filter
bar, double the wrapper margin (mt-12) so the hero's decorative bar, double the wrapper margin (mt-12) so the hero's decorative
`-mt-6` bleed leaves a visible gap instead of gluing the filter `-mt-6` bleed leaves a visible gap instead of gluing the filter
@@ -1126,6 +1350,8 @@ export const GalleryView: React.FC<GalleryViewProps> = ({ slug, event }) => {
<PhotoGridWithLayouts <PhotoGridWithLayouts
photos={filteredPhotos} photos={filteredPhotos}
slug={slug} slug={slug}
people={peopleEnabled ? people : undefined}
onSelectPerson={togglePerson}
categoryId={selectedCategoryId} categoryId={selectedCategoryId}
onFeedbackChange={() => { onFeedbackChange={() => {
refetch(); refetch();
@@ -1201,6 +1427,19 @@ export const GalleryView: React.FC<GalleryViewProps> = ({ slug, event }) => {
}} }}
/> />
)} )}
{/* "Show all" people (#1074) — a bottom sheet on mobile. */}
{peopleEnabled && (
<PeopleSheet
open={showPeopleSheet}
onClose={() => setShowPeopleSheet(false)}
people={people}
photos={data?.photos || []}
slug={slug}
selectedPersonIds={selectedPersonIds}
onToggle={togglePerson}
/>
)}
</GalleryLayout> </GalleryLayout>
</> </>
</GuestIdentityProvider> </GuestIdentityProvider>
@@ -0,0 +1,172 @@
import React, { useMemo, useState } from 'react';
import { Search, X, Info } from 'lucide-react';
import { useTranslation } from 'react-i18next';
import { AuthenticatedImage } from '../common/AuthenticatedImage';
import type { GalleryPerson, Photo } from '../../types';
import { faceCropStyle } from './faceCrop';
/**
* "Show all" people (#1074).
*
* A bottom sheet on mobile, a centred panel on desktop. The footnote is not
* decoration "where does this data go?" is the first question every guest
* has, and answering it inline is cheaper than losing their trust.
*/
interface PeopleSheetProps {
open: boolean;
onClose: () => void;
people: GalleryPerson[];
photos: Photo[];
slug: string;
selectedPersonIds: number[];
onToggle: (personId: number) => void;
}
export const PeopleSheet: React.FC<PeopleSheetProps> = ({
open, onClose, people, photos, slug, selectedPersonIds, onToggle,
}) => {
const { t } = useTranslation();
const [query, setQuery] = useState('');
const photoById = useMemo(() => {
const map = new Map<number, Photo>();
for (const photo of photos) map.set(photo.id, photo);
return map;
}, [photos]);
const filtered = useMemo(() => {
const q = query.trim().toLowerCase();
if (!q) return people;
// Only named people are searchable — there is nothing to match an
// unnamed cluster against, and pretending otherwise (matching the count,
// say) would be a puzzle rather than a feature.
return people.filter((p) => p.label?.toLowerCase().includes(q));
}, [people, query]);
if (!open) return null;
return (
<div className="fixed inset-0 z-50 flex items-end sm:items-center sm:justify-center">
<div
className="absolute inset-0 bg-black/40"
onClick={onClose}
aria-hidden="true"
/>
<div
role="dialog"
aria-modal="true"
aria-label={t('gallery.people.title', { defaultValue: 'People in this gallery' })}
className="relative w-full sm:max-w-2xl bg-white rounded-t-2xl sm:rounded-2xl shadow-xl max-h-[85vh] flex flex-col"
>
<div className="flex items-center justify-between px-4 pt-4 pb-3 border-b border-neutral-100">
<h2 className="text-base font-medium text-neutral-900">
{t('gallery.people.title', { defaultValue: 'People in this gallery' })}
</h2>
<button
type="button"
onClick={onClose}
aria-label={t('common.close', { defaultValue: 'Close' })}
className="p-2 -m-2 text-neutral-400 hover:text-neutral-600"
>
<X size={20} />
</button>
</div>
{people.some((p) => p.label) && (
<div className="px-4 pt-3">
<div className="relative">
<Search size={16} className="absolute left-3 top-1/2 -translate-y-1/2 text-neutral-400" />
<input
type="search"
value={query}
onChange={(e) => setQuery(e.target.value)}
placeholder={t('gallery.people.searchPlaceholder', { defaultValue: 'Find a person' })}
className="w-full pl-9 pr-3 py-2 text-sm border border-neutral-200 rounded-lg focus:outline-none focus:ring-2 focus:ring-primary-500 focus:border-transparent"
/>
</div>
</div>
)}
<div className="flex-1 overflow-y-auto px-4 py-4">
<div className="grid grid-cols-3 sm:grid-cols-5 gap-4">
{filtered.map((person) => {
const photo = person.cover ? photoById.get(person.cover.photo_id) : undefined;
const selected = selectedPersonIds.includes(person.id);
return (
<button
key={person.id}
type="button"
onClick={() => onToggle(person.id)}
aria-pressed={selected}
className="flex flex-col items-center gap-1.5 group focus:outline-none"
>
<span
className={[
'relative block w-16 h-16 rounded-full overflow-hidden bg-neutral-100 transition-all',
selected
? 'ring-[3px] ring-offset-2 ring-primary-600'
: 'ring-1 ring-neutral-200 group-hover:ring-neutral-400',
].join(' ')}
>
{photo && (
<AuthenticatedImage
src={photo.thumbnail_url || photo.url}
alt=""
isGallery
slug={slug}
photoId={photo.id}
requiresToken={photo.requires_token}
secureUrlTemplate={photo.secure_url_template}
// Crop to the face, exactly as the strip does. Without
// this a group photo shows whoever is centred — often
// not the person being labelled, and identical for two
// people whose cover is the same photo.
style={faceCropStyle(person.cover, photo.width, photo.height, 64)
|| { width: '100%', height: '100%', objectFit: 'cover' }}
/>
)}
</span>
<span className="w-full text-center leading-tight">
<span className={`block truncate text-xs ${person.label ? 'font-medium text-neutral-800' : 'text-neutral-500'}`}>
{person.label || t('gallery.people.unnamedCount', {
count: person.face_count,
defaultValue: `${person.face_count} photos`,
})}
</span>
{person.label && (
<span className="block text-[11px] text-neutral-400">{person.face_count}</span>
)}
</span>
</button>
);
})}
</div>
{filtered.length === 0 && (
<p className="text-sm text-neutral-500 text-center py-8">
{t('gallery.people.noMatches', { defaultValue: 'No one matches that name.' })}
</p>
)}
</div>
{/* The answer to the first question every guest has. Deliberately in
plain language this copy never says "biometric" or
"recognition", because those words describe our implementation,
not the guest's experience. */}
<div className="px-4 py-3 border-t border-neutral-100 flex gap-2 text-xs text-neutral-500">
<Info size={14} className="flex-shrink-0 mt-0.5" />
<p>
{t('gallery.people.privacyNote', {
defaultValue: 'People are detected automatically inside this gallery. Nothing is sent to any external service.',
})}
</p>
</div>
</div>
</div>
);
};
export default PeopleSheet;
@@ -0,0 +1,246 @@
import React, { useMemo, useState } from 'react';
import { ChevronRight, Users, X } from 'lucide-react';
import { useTranslation } from 'react-i18next';
import { AuthenticatedImage } from '../common/AuthenticatedImage';
import type { GalleryPerson, Photo } from '../../types';
import { faceCropStyle } from './faceCrop';
/**
* "People in this gallery" (#1074).
*
* The design goal is that a guest on a phone finds themselves in two taps and
* never learns the words "face recognition". So: circular crops, first names
* where the photographer supplied them, a photo count where they didn't, and
* no vocabulary from the implementation anywhere in the UI.
*
* Unnamed people show ONLY a count. Never "Person 7" a number is honest
* about what the system knows; an invented name is not.
*/
interface PersonAvatarProps {
person: GalleryPerson;
photo: Photo | undefined;
slug: string;
size: number;
selected: boolean;
onClick: () => void;
}
/**
* A face crop, produced by scaling the source photo inside a round window so
* the stored bbox lands centred.
*
* The bbox is in ORIGINAL image pixels but the thumbnail is served at some
* other size, so everything is done in RATIOS of the source dimensions
* which survives whatever rendition the browser actually gets. Without the
* photo's width/height we can't compute those ratios, so the component falls
* back to an un-cropped thumbnail rather than rendering a wrongly-offset crop.
*/
const PersonAvatar: React.FC<PersonAvatarProps> = ({
person, photo, slug, size, selected, onClick,
}) => {
const { t } = useTranslation();
const cropStyle = useMemo(
() => faceCropStyle(person.cover, photo?.width, photo?.height, size),
[person.cover, photo?.width, photo?.height, size],
);
const label = person.label
|| t('gallery.people.unnamedCount', { count: person.face_count, defaultValue: `${person.face_count} photos` });
return (
<button
type="button"
onClick={onClick}
aria-pressed={selected}
aria-label={person.label
? t('gallery.people.filterBy', { name: person.label, defaultValue: `Show photos of ${person.label}` })
: t('gallery.people.filterByUnnamed', { defaultValue: 'Show photos of this person' })}
className="flex flex-col items-center gap-1.5 flex-shrink-0 group focus:outline-none"
style={{ width: `${size + 8}px` }}
>
<span
className={[
'relative block rounded-full overflow-hidden bg-neutral-100 transition-all',
selected
? 'ring-[3px] ring-offset-2 ring-primary-600'
: 'ring-1 ring-neutral-200 group-hover:ring-neutral-400 group-focus-visible:ring-primary-500',
].join(' ')}
style={{ width: `${size}px`, height: `${size}px` }}
>
{photo && person.cover ? (
<AuthenticatedImage
src={photo.thumbnail_url || photo.url}
alt=""
isGallery
slug={slug}
photoId={photo.id}
requiresToken={photo.requires_token}
secureUrlTemplate={photo.secure_url_template}
style={cropStyle || { width: '100%', height: '100%', objectFit: 'cover' }}
/>
) : (
<span className="flex items-center justify-center w-full h-full text-neutral-400">
<Users size={size / 2.5} />
</span>
)}
</span>
<span className="w-full text-center leading-tight">
{/* Colours come from the gallery theme tokens, not fixed neutrals:
galleries can be dark, and hardcoded `text-neutral-800` renders
a named person's label almost invisibly against one. */}
<span
className="block truncate text-xs"
style={{
color: selected ? 'var(--color-accent)' : 'var(--color-text)',
fontWeight: person.label ? 500 : 400,
opacity: person.label ? 1 : 0.75,
}}
>
{label}
</span>
{person.label && (
<span className="block text-[11px]" style={{ color: 'var(--color-muted-text)' }}>
{person.face_count}
</span>
)}
</span>
</button>
);
};
interface PeopleStripProps {
people: GalleryPerson[];
photos: Photo[];
slug: string;
selectedPersonIds: number[];
onToggle: (personId: number) => void;
onShowAll: () => void;
scan?: { in_progress: boolean; scanned: number; total: number };
/** Persisted per-slug so a guest who dismisses it stays dismissed. */
collapsed: boolean;
onCollapsedChange: (collapsed: boolean) => void;
/** Avatars shown inline; the rest live behind "Show all". */
maxInline?: number;
}
export const PeopleStrip: React.FC<PeopleStripProps> = ({
people, photos, slug, selectedPersonIds, onToggle, onShowAll,
scan, collapsed, onCollapsedChange, maxInline = 12,
}) => {
const { t } = useTranslation();
const [avatarSize] = useState(() => (typeof window !== 'undefined' && window.innerWidth < 640 ? 56 : 64));
const photoById = useMemo(() => {
const map = new Map<number, Photo>();
for (const photo of photos) map.set(photo.id, photo);
return map;
}, [photos]);
// Fewer than two people isn't a "people in this gallery" feature, it's a
// single face taking up a row of the screen. Don't render at all.
if (people.length < 2 && !scan?.in_progress) return null;
if (collapsed) {
return (
<div className="flex items-center justify-between px-1 py-2 text-sm">
<span style={{ color: 'var(--color-muted-text)' }}>
{t('gallery.people.collapsedSummary', {
count: people.length,
defaultValue: `${people.length} people found`,
})}
</span>
<button
type="button"
onClick={() => onCollapsedChange(false)}
className="text-primary-600 hover:text-primary-700 font-medium"
>
{t('gallery.people.show', { defaultValue: 'Show' })}
</button>
</div>
);
}
const inline = people.slice(0, maxInline);
const hasMore = people.length > inline.length;
return (
<div className="py-3">
<div className="flex items-center justify-between mb-2 px-1">
<h3 className="text-sm font-medium" style={{ color: 'var(--color-text)' }}>
{t('gallery.people.title', { defaultValue: 'People in this gallery' })}
</h3>
<div className="flex items-center gap-3">
{hasMore && (
<button
type="button"
onClick={onShowAll}
className="flex items-center gap-0.5 text-sm text-primary-600 hover:text-primary-700"
>
{t('gallery.people.showAll', {
count: people.length,
defaultValue: `Show all ${people.length}`,
})}
<ChevronRight size={16} />
</button>
)}
<button
type="button"
onClick={() => onCollapsedChange(true)}
aria-label={t('gallery.people.dismiss', { defaultValue: 'Hide the people bar' })}
className="p-1 -m-1 text-neutral-400 hover:text-neutral-600"
>
<X size={16} />
</button>
</div>
</div>
{/* Backfill progress. The strip appears as soon as the first clusters
exist rather than blocking the gallery behind a spinner a guest
who arrives mid-scan gets a working gallery and a growing strip. */}
{scan?.in_progress && (
<div className="px-1 pb-2">
<div className="flex items-center justify-between text-xs text-neutral-500 mb-1">
<span style={{ color: 'var(--color-muted-text)' }}>
{t('gallery.people.scanning', {
scanned: scan.scanned,
total: scan.total,
defaultValue: `Finding people… ${scan.scanned}/${scan.total} photos`,
})}
</span>
</div>
<div className="h-0.5 bg-neutral-200 rounded-full overflow-hidden">
<div
className="h-full bg-primary-500 transition-all duration-500"
style={{ width: `${scan.total ? Math.round((scan.scanned / scan.total) * 100) : 0}%` }}
/>
</div>
</div>
)}
<div
className="flex gap-3 overflow-x-auto pb-1 px-1 snap-x scrollbar-thin"
style={{ scrollbarWidth: 'thin', WebkitOverflowScrolling: 'touch' }}
>
{inline.map((person) => (
<div key={person.id} className="snap-start">
<PersonAvatar
person={person}
photo={person.cover ? photoById.get(person.cover.photo_id) : undefined}
slug={slug}
size={avatarSize}
selected={selectedPersonIds.includes(person.id)}
onClick={() => onToggle(person.id)}
/>
</div>
))}
</div>
</div>
);
};
export default PeopleStrip;
@@ -3,7 +3,7 @@ import { Package } from 'lucide-react';
import { toast as toastify } from 'react-toastify'; import { toast as toastify } from 'react-toastify';
import { useTranslation } from 'react-i18next'; import { useTranslation } from 'react-i18next';
import type { Photo, DownloadResolutionChoice } from '../../types'; import type { Photo, DownloadResolutionChoice, GalleryPerson } from '../../types';
import { useDownloadPhoto } from '../../hooks/useGallery'; import { useDownloadPhoto } from '../../hooks/useGallery';
import { PhotoLightbox } from './PhotoLightbox'; import { PhotoLightbox } from './PhotoLightbox';
import { DownloadResolutionModal } from './DownloadResolutionModal'; import { DownloadResolutionModal } from './DownloadResolutionModal';
@@ -79,6 +79,10 @@ interface PhotoGridWithLayoutsProps {
// Mirror of the admin original-filename toggle (#508). When true, the // Mirror of the admin original-filename toggle (#508). When true, the
// lightbox bottom toolbar surfaces each photo's original camera name. // lightbox bottom toolbar surfaces each photo's original camera name.
showOriginalFilename?: boolean; showOriginalFilename?: boolean;
// People in this gallery (#1074) — forwarded to the lightbox so it can
// show "In this photo: …". Undefined when the feature is off.
people?: GalleryPerson[];
onSelectPerson?: (personId: number) => void;
} }
export const PhotoGridWithLayouts: React.FC<PhotoGridWithLayoutsProps> = ({ export const PhotoGridWithLayouts: React.FC<PhotoGridWithLayoutsProps> = ({
@@ -117,6 +121,8 @@ export const PhotoGridWithLayouts: React.FC<PhotoGridWithLayoutsProps> = ({
isClient = false, isClient = false,
onToggleVisibility, onToggleVisibility,
showOriginalFilename = false, showOriginalFilename = false,
people,
onSelectPerson,
}) => { }) => {
const { t } = useTranslation(); const { t } = useTranslation();
const { theme } = useTheme(); const { theme } = useTheme();
@@ -232,6 +238,12 @@ export const PhotoGridWithLayouts: React.FC<PhotoGridWithLayoutsProps> = ({
const layoutProps = { const layoutProps = {
photos, photos,
slug, slug,
// Face data (#1074) must reach the full-page layouts too — they render
// their OWN lightbox rather than the one below, so without this the
// "In this photo" chips silently vanish on gallery-premium and
// gallery-story even when the feature is fully enabled.
people,
onSelectPerson,
// Full-page layouts own their bulk-download control, so the resolution // Full-page layouts own their bulk-download control, so the resolution
// picker has to reach them too (#858) — otherwise premium/story galleries // picker has to reach them too (#858) — otherwise premium/story galleries
// silently skip the choice the admin enabled. // silently skip the choice the admin enabled.
@@ -407,6 +419,8 @@ export const PhotoGridWithLayouts: React.FC<PhotoGridWithLayoutsProps> = ({
initialShowFeedback={openFeedbackInitially} initialShowFeedback={openFeedbackInitially}
onFeedbackChange={onFeedbackChange} onFeedbackChange={onFeedbackChange}
showOriginalFilename={showOriginalFilename} showOriginalFilename={showOriginalFilename}
people={people}
onSelectPerson={onSelectPerson}
/> />
)} )}
@@ -1,7 +1,8 @@
import React, { useState, useEffect, useLayoutEffect, useRef } from 'react'; import React, { useState, useEffect, useLayoutEffect, useMemo, useRef } from 'react';
import { useTranslation } from 'react-i18next';
import { useDevToolsProtection } from '../../hooks/useDevToolsProtection'; import { useDevToolsProtection } from '../../hooks/useDevToolsProtection';
import { X, ChevronLeft, ChevronRight, Download, ZoomIn, ZoomOut, Minimize2, MessageSquare, Heart, Star } from 'lucide-react'; import { X, ChevronLeft, ChevronRight, Download, ZoomIn, ZoomOut, Minimize2, MessageSquare, Heart, Star } from 'lucide-react';
import type { Photo } from '../../types'; import type { Photo, GalleryPerson } from '../../types';
import { useSavePhotoToDevice } from '../../hooks/useGallery'; import { useSavePhotoToDevice } from '../../hooks/useGallery';
import { AuthenticatedImage } from '../common'; import { AuthenticatedImage } from '../common';
import { PhotoFeedback } from './PhotoFeedback'; import { PhotoFeedback } from './PhotoFeedback';
@@ -31,6 +32,14 @@ interface PhotoLightboxProps {
// back to source files (#508). Tied to the admin-side toggle that // back to source files (#508). Tied to the admin-side toggle that
// also drives original-filename downloads (#493). // also drives original-filename downloads (#493).
showOriginalFilename?: boolean; showOriginalFilename?: boolean;
// People in this gallery (#1074). Passed only when the feature is on for
// the event AND visible to this viewer — an empty list means "scanned,
// nobody found", which the chips row handles by not rendering.
people?: GalleryPerson[];
// Applying a person filter closes the lightbox and filters the grid
// behind it, so the guest lands on the result rather than paging through
// the old set.
onSelectPerson?: (personId: number) => void;
} }
export const PhotoLightbox: React.FC<PhotoLightboxProps> = ({ export const PhotoLightbox: React.FC<PhotoLightboxProps> = ({
@@ -48,6 +57,8 @@ export const PhotoLightbox: React.FC<PhotoLightboxProps> = ({
disableRightClick = false, disableRightClick = false,
enableDevtoolsProtection = false, enableDevtoolsProtection = false,
showOriginalFilename = false, showOriginalFilename = false,
people,
onSelectPerson,
}) => { }) => {
const [currentIndex, setCurrentIndex] = useState(initialIndex); const [currentIndex, setCurrentIndex] = useState(initialIndex);
const [zoom, setZoom] = useState(1); const [zoom, setZoom] = useState(1);
@@ -153,6 +164,17 @@ export const PhotoLightbox: React.FC<PhotoLightboxProps> = ({
// leaving currentIndex past the end. The effect below re-syncs the // leaving currentIndex past the end. The effect below re-syncs the
// index (or closes the lightbox when nothing is left). // index (or closes the lightbox when nothing is left).
const currentPhoto = photos[currentIndex] ?? photos[photos.length - 1]; const currentPhoto = photos[currentIndex] ?? photos[photos.length - 1];
const { t } = useTranslation();
// People detected in the open photo (#1074). Resolved against the list the
// server returned rather than the raw ids, so a person the photographer
// hid or ignored has no entry to match and simply never appears.
const peopleInPhoto = useMemo<GalleryPerson[]>(() => {
const ids = currentPhoto?.person_ids;
if (!ids?.length || !people?.length) return [];
return people.filter((person) => ids.includes(person.id));
}, [currentPhoto?.person_ids, people]);
// Per-category download permission (#640). AND'd with the event-level // Per-category download permission (#640). AND'd with the event-level
// allowDownloads — disabling at either level hides the download button. // allowDownloads — disabling at either level hides the download button.
// Defaults true for uncategorised photos and pre-migration-135 categories. // Defaults true for uncategorised photos and pre-migration-135 categories.
@@ -782,6 +804,37 @@ export const PhotoLightbox: React.FC<PhotoLightboxProps> = ({
{currentPhoto.original_filename || currentPhoto.filename} {currentPhoto.original_filename || currentPhoto.filename}
</p> </p>
)} )}
{/* People in this photo (#1074). The second way into the face
filter: a guest looking at a photo of themselves can act on
it without scrolling back to the strip.
Only people the server already returned are shown, so hidden
and ignored ones never appear here either. Unnamed people
show their photo count, never an invented name. */}
{peopleInPhoto.length > 0 && (
<div className="flex items-center gap-1.5 flex-wrap mt-1.5">
<span className="text-xs text-white opacity-60">
{t('gallery.people.inThisPhoto', { defaultValue: 'In this photo:' })}
</span>
{peopleInPhoto.map((person) => (
<button
key={person.id}
type="button"
onClick={() => {
onSelectPerson?.(person.id);
onClose();
}}
className="px-2 py-0.5 rounded-full bg-white/15 hover:bg-white/25 text-white text-xs transition-colors"
>
{person.label || t('gallery.people.unnamedCount', {
count: person.face_count,
defaultValue: `${person.face_count} photos`,
})}
</button>
))}
</div>
)}
</div> </div>
<div className="flex items-center gap-1 sm:gap-2 flex-wrap justify-end"> <div className="flex items-center gap-1 sm:gap-2 flex-wrap justify-end">
@@ -0,0 +1,134 @@
import React from 'react';
import { render, screen } from '@testing-library/react';
import { describe, it, expect, vi } from 'vitest';
import { PeopleStrip } from '../PeopleStrip';
import type { GalleryPerson, Photo } from '../../../types';
vi.mock('react-i18next', () => ({
useTranslation: () => ({
t: (_key: string, opts?: any) => opts?.defaultValue ?? _key,
}),
}));
vi.mock('../../common/AuthenticatedImage', () => ({
AuthenticatedImage: (props: any) => <img alt={props.alt} data-testid="avatar-img" />,
}));
function person(id: number, over: Partial<GalleryPerson> = {}): GalleryPerson {
return {
id,
label: null,
face_count: 10,
cover: { face_id: id, photo_id: id, bbox: [10, 10, 50, 50] },
...over,
};
}
const photos: Photo[] = [1, 2, 3].map((id) => ({
id,
filename: `${id}.jpg`,
url: `/p/${id}`,
thumbnail_url: `/t/${id}`,
type: 'individual',
size: 100,
uploaded_at: '2026-01-01T00:00:00Z',
width: 1000,
height: 800,
})) as Photo[];
const baseProps = {
photos,
slug: 'test-gallery',
selectedPersonIds: [],
onToggle: vi.fn(),
onShowAll: vi.fn(),
collapsed: false,
onCollapsedChange: vi.fn(),
};
describe('PeopleStrip (#1074)', () => {
it('renders nothing for fewer than two people', () => {
// One face taking up a whole row is not a "people in this gallery"
// feature, it's clutter.
const { container } = render(<PeopleStrip {...baseProps} people={[person(1)]} />);
expect(container).toBeEmptyDOMElement();
});
it('still renders during a scan even with too few people yet', () => {
render(
<PeopleStrip
{...baseProps}
people={[person(1)]}
scan={{ in_progress: true, scanned: 40, total: 200 }}
/>
);
expect(screen.getByText(/Finding people/)).toBeInTheDocument();
});
it('shows a photo count for unnamed people, never an invented name', () => {
render(<PeopleStrip {...baseProps} people={[person(1, { face_count: 42 }), person(2)]} />);
expect(screen.getByText('42 photos')).toBeInTheDocument();
// The thing we specifically refuse to do.
expect(screen.queryByText(/Person \d/)).not.toBeInTheDocument();
});
it('shows the name and the count when a person has been named', () => {
render(
<PeopleStrip
{...baseProps}
people={[person(1, { label: 'Anna', face_count: 97 }), person(2)]}
/>
);
expect(screen.getByText('Anna')).toBeInTheDocument();
expect(screen.getByText('97')).toBeInTheDocument();
});
it('marks the selected person as pressed for assistive tech', () => {
render(
<PeopleStrip
{...baseProps}
people={[person(1, { label: 'Anna' }), person(2, { label: 'Ben' })]}
selectedPersonIds={[1]}
/>
);
const buttons = screen.getAllByRole('button', { pressed: true });
expect(buttons).toHaveLength(1);
expect(buttons[0]).toHaveAttribute('aria-label', expect.stringContaining('Anna'));
});
it('collapses to a one-line summary when dismissed', () => {
render(
<PeopleStrip
{...baseProps}
people={[person(1), person(2), person(3)]}
collapsed
/>
);
expect(screen.getByText('3 people found')).toBeInTheDocument();
expect(screen.queryByTestId('avatar-img')).not.toBeInTheDocument();
});
it('offers "show all" only when people overflow the inline strip', () => {
const many = Array.from({ length: 14 }, (_, i) => person(i + 1));
const { rerender } = render(<PeopleStrip {...baseProps} people={many} maxInline={12} />);
expect(screen.getByText('Show all 14')).toBeInTheDocument();
rerender(<PeopleStrip {...baseProps} people={many.slice(0, 5)} maxInline={12} />);
expect(screen.queryByText(/Show all/)).not.toBeInTheDocument();
});
it('falls back to an uncropped thumbnail when photo dimensions are unknown', () => {
// Without width/height the bbox ratios can't be computed; rendering a
// wrongly-offset crop would be worse than not cropping.
const noDims = [{ ...photos[0], width: undefined, height: undefined }] as Photo[];
render(
<PeopleStrip
{...baseProps}
photos={noDims}
people={[person(1), person(2)]}
/>
);
expect(screen.getAllByTestId('avatar-img').length).toBeGreaterThan(0);
});
});
@@ -0,0 +1,52 @@
/**
* Shared face-crop geometry (#1074).
*
* Extracted because three surfaces need identical maths and two of them
* originally shipped without it: the "Show all" sheet and the admin People
* manager rendered centred full-photo thumbnails, so on any group photo the
* avatar showed whoever happened to be in the middle often not the person
* it was labelling, and identical for two people from the same photo.
*
* The bbox is in ORIGINAL image pixels; the rendered thumbnail is some other
* size. Everything is therefore done in RATIOS of the source dimensions, which
* survives whatever rendition the browser is actually served.
*/
export interface FaceBox {
bbox: [number, number, number, number];
}
/**
* Style for an <img> inside a square, overflow-hidden, position-relative box
* of `size` px, such that `bbox` lands centred and filling it.
*
* Returns null when the source dimensions are unknown the caller should then
* fall back to an uncropped thumbnail rather than render a wrongly-offset crop.
*/
export function faceCropStyle(
cover: FaceBox | null | undefined,
photoWidth: number | undefined,
photoHeight: number | undefined,
size: number,
): React.CSSProperties | null {
if (!cover || !photoWidth || !photoHeight) return null;
const [bx, by, bw, bh] = cover.bbox;
if (!bw || !bh) return null;
// Detectors crop tight to the face; a little padding reads as a portrait
// rather than a specimen.
const pad = 0.45;
const cx = bx + bw / 2;
const cy = by + bh / 2;
const side = Math.max(bw, bh) * (1 + pad);
const scale = size / side;
return {
width: `${photoWidth * scale}px`,
height: `${photoHeight * scale}px`,
maxWidth: 'none',
position: 'absolute',
left: `${size / 2 - cx * scale}px`,
top: `${size / 2 - cy * scale}px`,
};
}
@@ -1,8 +1,13 @@
import React from 'react'; import React from 'react';
import type { Photo, DownloadResolutionChoice } from '../../../types'; import type { Photo, DownloadResolutionChoice, GalleryPerson } from '../../../types';
export interface BaseGalleryLayoutProps { export interface BaseGalleryLayoutProps {
photos: Photo[]; photos: Photo[];
// People in this gallery (#1074) — forwarded by PhotoGridWithLayouts so
// full-page layouts, which render their OWN lightbox, can still show the
// "In this photo" chips.
people?: GalleryPerson[];
onSelectPerson?: (personId: number) => void;
slug: string; slug: string;
onPhotoClick: (index: number) => void; onPhotoClick: (index: number) => void;
// Optional: open the lightbox with feedback panel visible // Optional: open the lightbox with feedback panel visible
@@ -62,6 +62,9 @@ export const GalleryStoryLayout: React.FC<GalleryStoryLayoutProps> = ({
welcomeMessage, welcomeMessage,
onLogout, onLogout,
showOriginalFilename = false, showOriginalFilename = false,
people,
onSelectPerson,
}) => { }) => {
// These props are passed by parent but we use our own feedback system, so mark as intentionally unused // These props are passed by parent but we use our own feedback system, so mark as intentionally unused
void _onPhotoClick; void _onPhotoClick;
@@ -391,6 +394,11 @@ export const GalleryStoryLayout: React.FC<GalleryStoryLayoutProps> = ({
useCanvasRendering={useCanvasRendering} useCanvasRendering={useCanvasRendering}
onFeedbackChange={onFeedbackChange} onFeedbackChange={onFeedbackChange}
showOriginalFilename={showOriginalFilename} showOriginalFilename={showOriginalFilename}
// #1074: this layout renders its own lightbox, so the people props
// have to be threaded through explicitly or the "In this photo"
// chips silently disappear on the Story theme.
people={people}
onSelectPerson={onSelectPerson}
/> />
)} )}
@@ -26,8 +26,10 @@ import {
Send, Send,
Workflow, Workflow,
} from 'lucide-react'; } from 'lucide-react';
import { useQuery } from '@tanstack/react-query';
import { useTranslation } from 'react-i18next'; import { useTranslation } from 'react-i18next';
import { Button, Card } from '../../../components/common'; import { Button, Card } from '../../../components/common';
import { api } from '../../../config/api';
import { FeatureCard } from '../components/FeatureCard'; import { FeatureCard } from '../components/FeatureCard';
import { SidebarPreview } from '../components/SidebarPreview'; import { SidebarPreview } from '../components/SidebarPreview';
import { useFeatureFlags } from '../../../contexts/FeatureFlagsContext'; import { useFeatureFlags } from '../../../contexts/FeatureFlagsContext';
@@ -48,6 +50,17 @@ const Section: React.FC<SectionProps> = ({ title, children }) => (
); );
export const FeaturesTab: React.FC = () => { export const FeaturesTab: React.FC = () => {
// The all-in-one image cannot run face recognition (#1074) — see
// faceSettings.isSingleContainerImage. Read it from the version endpoint the
// admin UI already calls, so the card can say WHY rather than offering a
// switch that refuses to stay on.
const { data: systemVersion } = useQuery({
queryKey: ['admin-system-version'],
queryFn: async () => (await api.get('/admin/system/version')).data,
staleTime: 5 * 60_000,
});
const isSingleContainer = systemVersion?.single_container === true;
const { t } = useTranslation(); const { t } = useTranslation();
const { staged, setFlag, save, reset, isDirty, isSaving } = useFeatureFlags(); const { staged, setFlag, save, reset, isDirty, isSaving } = useFeatureFlags();
@@ -148,6 +161,33 @@ export const FeaturesTab: React.FC = () => {
enabled={staged.transfers} enabled={staged.transfers}
onToggle={(next) => setFlag('transfers', next)} onToggle={(next) => setFlag('transfers', next)}
/> />
{/* Face recognition (#1074). Requires the optional picpeak-ml
sidecar container with the flag on but no sidecar running,
photos simply stay queued and nothing breaks. Two deliberate
actions are still needed before any face is processed: this
toggle, and the per-event switch on each gallery. */}
<FeatureCard
icon={Users}
title={t('settings.features.faces.title', 'People in galleries')}
description={t(
'settings.features.faces.description',
'Group each gallery\u2019s photos by the people in them, so guests can find themselves in two taps and download just their own photos. Runs entirely on your own server in a separate, optional container \u2014 nothing is sent anywhere. Detected faces are personal data, so this stays off until you enable it per gallery too.',
)}
status="new"
statusLabel={statusLabel('new')}
sidebarHidden
sidebarHiddenLabel={sidebarHiddenLabel}
enabled={staged.faces && !isSingleContainer}
onToggle={(next) => setFlag('faces', next)}
disabled={isSingleContainer}
lockedReason={isSingleContainer
? t(
'settings.features.faces.lockedSingleContainer',
'Not available on the all-in-one image. Face detection needs a separate ML container and competes with image processing for the same CPU and memory, which would slow the whole install down.',
)
: undefined}
/>
</Section> </Section>
{/* Automation the visual workflow engine. Master kill-switch for the {/* Automation the visual workflow engine. Master kill-switch for the
+79 -2
View File
@@ -156,7 +156,10 @@
"show": "Einblenden", "show": "Einblenden",
"poweredBy": "Bereitgestellt von", "poweredBy": "Bereitgestellt von",
"on": "an", "on": "an",
"off": "aus" "off": "aus",
"clear": "Zurücksetzen",
"saved": "Gespeichert",
"saveFailed": "Speichern fehlgeschlagen"
}, },
"upload": { "upload": {
"photoCategory": "Fotokategorie", "photoCategory": "Fotokategorie",
@@ -1061,7 +1064,27 @@
"downloadReady": "Ihr Download ist bereit", "downloadReady": "Ihr Download ist bereit",
"downloadNow": "Herunterladen", "downloadNow": "Herunterladen",
"downloadPrepFailed": "Vorbereitung fehlgeschlagen", "downloadPrepFailed": "Vorbereitung fehlgeschlagen",
"downloadPrepTimeout": "Das dauert länger als erwartet. Bitte erneut versuchen." "downloadPrepTimeout": "Das dauert länger als erwartet. Bitte erneut versuchen.",
"people": {
"title": "Personen in dieser Galerie",
"showAll": "Alle {{count}} anzeigen",
"show": "Anzeigen",
"dismiss": "Personenleiste ausblenden",
"collapsedSummary": "{{count}} Personen gefunden",
"unnamedCount": "{{count}} Fotos",
"filterBy": "Fotos von {{name}} anzeigen",
"filterByUnnamed": "Fotos dieser Person anzeigen",
"scanning": "Personen werden gesucht… {{scanned}}/{{total}} Fotos",
"matchAll": "Beide Personen",
"matchAny": "Eine der Personen",
"matchCount": "{{count}} von {{total}} Fotos",
"clear": "Zurücksetzen",
"searchPlaceholder": "Person suchen",
"noMatches": "Niemand passt zu diesem Namen.",
"privacyNote": "Personen werden automatisch innerhalb dieser Galerie erkannt. Es werden keine Daten an externe Dienste gesendet.",
"inThisPhoto": "Auf diesem Foto:",
"downloadThese": "Diese {{count}} herunterladen"
}
}, },
"categories": { "categories": {
"title": "Fotokategorien", "title": "Fotokategorien",
@@ -2115,6 +2138,11 @@
"title": "Workflows", "title": "Workflows",
"description": "Visuelle Automatisierungen auf einer Canvas erstellen Auslöser, Bedingungen, Verzweigungen, Schleifen und Freigabe-Gates für Admins. Deine Mahnstufen und Buchungsschritte werden zu bearbeitbaren Abläufen. Strikt optional.", "description": "Visuelle Automatisierungen auf einer Canvas erstellen Auslöser, Bedingungen, Verzweigungen, Schleifen und Freigabe-Gates für Admins. Deine Mahnstufen und Buchungsschritte werden zu bearbeitbaren Abläufen. Strikt optional.",
"sidebar": "Workflows" "sidebar": "Workflows"
},
"faces": {
"title": "Personen in Galerien",
"description": "Gruppiert die Fotos jeder Galerie nach den abgebildeten Personen, damit Gäste sich mit zwei Tipps selbst finden und nur ihre eigenen Fotos herunterladen. Läuft vollständig auf Ihrem eigenen Server in einem separaten, optionalen Container — es werden keine Daten übermittelt. Erkannte Gesichter sind personenbezogene Daten; die Funktion bleibt daher aus, bis Sie sie zusätzlich pro Galerie aktivieren.",
"lockedSingleContainer": "Im All-in-One-Image nicht verfügbar. Die Gesichtserkennung benötigt einen separaten ML-Container und konkurriert mit der Bildverarbeitung um dieselbe CPU und denselben Arbeitsspeicher, was die gesamte Installation verlangsamen würde."
} }
}, },
"customerSurface": { "customerSurface": {
@@ -3085,6 +3113,55 @@
"email_template_created": "E-Mail-Vorlage erstellt: {{template_key}}", "email_template_created": "E-Mail-Vorlage erstellt: {{template_key}}",
"event_duplicated": "Event dupliziert aus {{source_event_name}}", "event_duplicated": "Event dupliziert aus {{source_event_name}}",
"whatsapp_config_updated": "WhatsApp-Konfiguration aktualisiert" "whatsapp_config_updated": "WhatsApp-Konfiguration aktualisiert"
},
"people": {
"title": "Personen in dieser Galerie",
"subtitle": "Umbenennen, getrennte Personen zusammenführen oder jemanden vor Gästen verbergen.",
"select": "Zum Zusammenführen auswählen",
"unnamed": "Namen hinzufügen",
"namePlaceholder": "Namen hinzufügen",
"photoCount": "{{count}} Fotos",
"hidden": "für Gäste verborgen",
"ignored": "ignoriert",
"renamed": "Name gespeichert",
"merged": "Personen zusammengeführt",
"split": "Als neue Person abgetrennt",
"updated": "Aktualisiert",
"merge": "Zusammenführen",
"mergeHint": "Tippen Sie zwei oder mehr Gesichter an, um sie zu einer Person zusammenzuführen.",
"selectedCount": "{{count}} ausgewählt",
"splitAction": "Fotos abtrennen, die jemand anderes zeigen",
"hideAction": "Vor Gästen verbergen",
"ignoreAction": "Keine echte Person — ignorieren",
"splitHelp": "Wählen Sie die Fotos aus, die NICHT diese Person zeigen. Sie werden zu einem neuen Eintrag, alles andere bleibt.",
"splitSelected": "{{count}} ausgewählt",
"doSplit": "Abtrennen",
"empty": "Noch keine Personen erkannt."
},
"faces": {
"manage": "Personen verwalten",
"visibleToGuests": "({{count}} für Gäste sichtbar)",
"title": "Personen in dieser Galerie",
"subtitle": "Fotos nach den abgebildeten Personen gruppieren, damit Gäste ihre eigenen finden und herunterladen können.",
"consentNotice": "Erkannte Gesichter sind personenbezogene Daten und gelten in der EU als besondere Kategorie. Sie sind für diese Galerie verantwortlich: Stellen Sie sicher, dass Sie eine Rechtsgrundlage für die abgebildeten Personen haben, bevor Sie dies aktivieren. Nichts verlässt Ihren Server — die Erkennung läuft in Ihrem eigenen Container.",
"enable": "Personen in dieser Galerie erkennen",
"enableHint": "Vorhandene Fotos werden im Hintergrund durchsucht. Gesichter und ihre numerischen Signaturen werden in Ihrer Datenbank gespeichert; sie sind niemals in Backups oder Exporten enthalten.",
"visible": "Personenleiste für Gäste anzeigen",
"visibleHint": "Ausgeschaltet erhalten nur Sie die Gruppierung als privates Werkzeug, und Gäste sehen eine unveränderte Galerie.",
"previewNotice": "Die Erkennung arbeitet mit der Vorschaugröße jedes Fotos. In Galerien ohne vorhandene Vorschaubilder werden diese beim ersten Durchlauf erzeugt, was zusätzliche CPU-Last und Speicherplatz benötigt.",
"scanning": "Wird durchsucht… {{scanned}} von {{total}} Fotos",
"status": "{{scanned}} / {{total}} Fotos durchsucht · {{people}} Personen",
"failed": "{{count}} fehlgeschlagen",
"queued": "{{count}} Fotos werden nach Personen durchsucht…",
"rescan": "Erneut durchsuchen",
"rescanQueued": "Erneute Suche eingeplant",
"recluster": "Personen neu gruppieren",
"reclustered": "Personen neu gruppiert",
"delete": "Alle Gesichtsdaten löschen",
"deleted": "Gesichtsdaten gelöscht",
"confirmDelete": "Alle erkannten Personen und Gesichtsdaten dieser Galerie löschen? Dies kann nicht rückgängig gemacht werden. Die Fotos selbst bleiben unverändert.",
"autoCategories": "Fotos automatisch in Kategorien einsortieren",
"autoCategoriesHint": "Nutzt die Anzahl der Gesichter, um Fotos als Details, Porträts, Kleine Gruppen oder Gruppen abzulegen. Gilt für alle Galerien, füllt ausschließlich leere Kategorien und ändert niemals eine von Ihnen gesetzte."
} }
}, },
"acceptInvitation": { "acceptInvitation": {
+79 -2
View File
@@ -156,7 +156,10 @@
"show": "Show", "show": "Show",
"poweredBy": "Powered by", "poweredBy": "Powered by",
"on": "on", "on": "on",
"off": "off" "off": "off",
"clear": "Clear",
"saved": "Saved",
"saveFailed": "Save failed"
}, },
"upload": { "upload": {
"photoCategory": "Photo Category", "photoCategory": "Photo Category",
@@ -602,7 +605,27 @@
"downloadReady": "Your download is ready", "downloadReady": "Your download is ready",
"downloadNow": "Download", "downloadNow": "Download",
"downloadPrepFailed": "Preparation failed", "downloadPrepFailed": "Preparation failed",
"downloadPrepTimeout": "This is taking longer than expected. Please try again." "downloadPrepTimeout": "This is taking longer than expected. Please try again.",
"people": {
"title": "People in this gallery",
"showAll": "Show all {{count}}",
"show": "Show",
"dismiss": "Hide the people bar",
"collapsedSummary": "{{count}} people found",
"unnamedCount": "{{count}} photos",
"filterBy": "Show photos of {{name}}",
"filterByUnnamed": "Show photos of this person",
"scanning": "Finding people… {{scanned}}/{{total}} photos",
"matchAll": "Both people",
"matchAny": "Either person",
"matchCount": "{{count}} of {{total}} photos",
"clear": "Clear",
"searchPlaceholder": "Find a person",
"noMatches": "No one matches that name.",
"privacyNote": "People are detected automatically inside this gallery. Nothing is sent to any external service.",
"inThisPhoto": "In this photo:",
"downloadThese": "Download these {{count}}"
}
}, },
"categories": { "categories": {
"title": "Photo Categories", "title": "Photo Categories",
@@ -1656,6 +1679,11 @@
"title": "Workflows", "title": "Workflows",
"description": "Build visual automations on a canvas — triggers, conditions, branches, loops and admin approval gates. Your reminder ladder and booking steps become editable flows. Strictly opt-in.", "description": "Build visual automations on a canvas — triggers, conditions, branches, loops and admin approval gates. Your reminder ladder and booking steps become editable flows. Strictly opt-in.",
"sidebar": "Workflows" "sidebar": "Workflows"
},
"faces": {
"title": "People in galleries",
"description": "Group each gallery's photos by the people in them, so guests can find themselves in two taps and download just their own photos. Runs entirely on your own server in a separate, optional container — nothing is sent anywhere. Detected faces are personal data, so this stays off until you enable it per gallery too.",
"lockedSingleContainer": "Not available on the all-in-one image. Face detection needs a separate ML container and competes with image processing for the same CPU and memory, which would slow the whole install down."
} }
}, },
"customerSurface": { "customerSurface": {
@@ -2657,6 +2685,55 @@
"email_template_created": "Email template created: {{template_key}}", "email_template_created": "Email template created: {{template_key}}",
"event_duplicated": "Event duplicated from {{source_event_name}}", "event_duplicated": "Event duplicated from {{source_event_name}}",
"whatsapp_config_updated": "WhatsApp configuration updated" "whatsapp_config_updated": "WhatsApp configuration updated"
},
"people": {
"title": "People in this gallery",
"subtitle": "Rename, merge people who were split apart, or hide someone from guests.",
"select": "Select for merging",
"unnamed": "Add a name",
"namePlaceholder": "Add a name",
"photoCount": "{{count}} photos",
"hidden": "hidden from guests",
"ignored": "ignored",
"renamed": "Name saved",
"merged": "People merged",
"split": "Split into a new person",
"updated": "Updated",
"merge": "Merge",
"mergeHint": "Tap two or more faces to merge them into one person.",
"selectedCount": "{{count}} selected",
"splitAction": "Split out photos that are someone else",
"hideAction": "Hide from guests",
"ignoreAction": "Not a real person — ignore",
"splitHelp": "Pick the photos that are NOT this person. They become a new entry, and everything else stays.",
"splitSelected": "{{count}} selected",
"doSplit": "Split out",
"empty": "No people detected yet."
},
"faces": {
"manage": "Manage people",
"visibleToGuests": "({{count}} shown to guests)",
"title": "People in this gallery",
"subtitle": "Group photos by the people in them, so guests can find and download their own.",
"consentNotice": "Detected faces are personal data, and in the EU they count as a special category. You are the controller for this gallery: make sure you have a lawful basis for the people in these photos before switching this on. Nothing leaves your server — detection runs in your own container.",
"enable": "Detect people in this gallery",
"enableHint": "Existing photos are scanned in the background. Faces and their numeric signatures are stored in your database; they are never included in backups or exports.",
"visible": "Show the people bar to guests",
"visibleHint": "Off means you get the grouping as a private tool and guests see an unchanged gallery.",
"previewNotice": "Scanning works on the preview-sized copy of each photo. Galleries that have not generated previews yet will create them during the first scan, which uses additional CPU and disk space.",
"scanning": "Scanning… {{scanned}} of {{total}} photos",
"status": "{{scanned}} / {{total}} photos scanned · {{people}} people",
"failed": "{{count}} failed",
"queued": "Scanning {{count}} photos for people…",
"rescan": "Re-scan",
"rescanQueued": "Re-scan queued",
"recluster": "Re-group people",
"reclustered": "People regrouped",
"delete": "Delete all face data",
"deleted": "Face data deleted",
"confirmDelete": "Delete all detected people and face data for this gallery? This cannot be undone. Photos are not affected.",
"autoCategories": "Sort photos into categories automatically",
"autoCategoriesHint": "Uses the number of faces to file photos as Details, Portraits, Small groups or Groups. Applies to every gallery, only ever fills an empty category, and never changes one you set yourself."
} }
}, },
"acceptInvitation": { "acceptInvitation": {
@@ -5,6 +5,7 @@ import { PermissionGate } from '../../../components/admin/PermissionGate';
import { EventReminderOverrideCard } from '../../../components/admin/EventReminderOverrideCard'; import { EventReminderOverrideCard } from '../../../components/admin/EventReminderOverrideCard';
import { SlideshowSettingsCard } from '../../../components/admin/SlideshowSettingsCard'; import { SlideshowSettingsCard } from '../../../components/admin/SlideshowSettingsCard';
import { DownloadResolutionCard } from '../../../components/admin/DownloadResolutionCard'; import { DownloadResolutionCard } from '../../../components/admin/DownloadResolutionCard';
import { FaceRecognitionCard } from '../../../components/admin/FaceRecognitionCard';
import { ShortUrlsCard } from '../../../components/admin/ShortUrlsCard'; import { ShortUrlsCard } from '../../../components/admin/ShortUrlsCard';
import { useFeatureFlags } from '../../../contexts/FeatureFlagsContext'; import { useFeatureFlags } from '../../../contexts/FeatureFlagsContext';
import type { AdminPhoto } from '../../../services/photos.service'; import type { AdminPhoto } from '../../../services/photos.service';
@@ -122,6 +123,14 @@ export const OverviewTab: React.FC<OverviewTabProps> = ({
other "what the customer receives" controls. */} other "what the customer receives" controls. */}
<DownloadResolutionCard eventId={event.id} onChanged={() => refetchEvent()} /> <DownloadResolutionCard eventId={event.id} onChanged={() => refetchEvent()} />
{/* People in this gallery (#1074). Gated behind the `faces` feature
flag which is itself gated on the operator running the optional
picpeak-ml sidecar, so this card is invisible on the vast majority
of installs. */}
{flags.faces && (
<FaceRecognitionCard eventId={event.id} isArchived={event.is_archived} />
)}
{/* Live Slideshow ("Diashow") link + live display settings (migrations 138/139). {/* Live Slideshow ("Diashow") link + live display settings (migrations 138/139).
Gated behind the `slideshow` feature flag. */} Gated behind the `slideshow` feature flag. */}
{flags.slideshow && ( {flags.slideshow && (
+11 -1
View File
@@ -75,7 +75,17 @@ export type FeatureKey =
// Workflow / automation engine — admin-configurable visual flows (triggers, // Workflow / automation engine — admin-configurable visual flows (triggers,
// conditions, branches, loops, approval gates) built on a canvas. Strictly // conditions, branches, loops, approval gates) built on a canvas. Strictly
// opt-in; gates the Workflows admin area and the engine runtime. // opt-in; gates the Workflows admin area and the engine runtime.
| 'workflows'; | 'workflows'
// Face recognition — "People in this gallery" (migration 177, #1074).
// Requires the optional picpeak-ml sidecar container. THIS FLAG IS THE
// GATE for the whole feature: the backend's FACE_ML_URL has a working
// default, so the sidecar's presence can't be detected from config alone.
// While this is off, no face UI renders anywhere — no admin panel, no
// people strip, no lightbox chips — and the backend never contacts the
// sidecar. Face embeddings are biometric data (GDPR Art. 9); turning this
// on is only the first of two deliberate actions, since detection is still
// enabled per event.
| 'faces';
export type FeatureFlags = Record<FeatureKey, boolean>; export type FeatureFlags = Record<FeatureKey, boolean>;
+13 -1
View File
@@ -1,7 +1,7 @@
import { api } from '../config/api'; import { api } from '../config/api';
import type { import type {
GalleryInfo, GalleryData, GalleryStats, ResolvedGalleryIdentifier, GalleryInfo, GalleryData, GalleryStats, ResolvedGalleryIdentifier,
DownloadJobStatus, DownloadJobState, DownloadJobStatus, DownloadJobState, GalleryPeopleResponse,
} from '../types'; } from '../types';
import { normalizeRequirePassword } from '../utils/accessControl'; import { normalizeRequirePassword } from '../utils/accessControl';
import { parseContentDispositionFilename } from '../utils/contentDisposition'; import { parseContentDispositionFilename } from '../utils/contentDisposition';
@@ -383,4 +383,16 @@ export const galleryService = {
const response = await api.get<ResolvedGalleryIdentifier>(`/gallery/resolve/${identifier}`); const response = await api.get<ResolvedGalleryIdentifier>(`/gallery/resolve/${identifier}`);
return response.data; return response.data;
}, },
/**
* People detected in this gallery (#1074).
*
* Returns an empty list rather than an error when the feature is off, so a
* guest can't tell "no people here" from "feature disabled". Counts and
* cover faces are scoped server-side to the photos this viewer may see.
*/
async getPeople(slug: string): Promise<GalleryPeopleResponse> {
const response = await api.get<GalleryPeopleResponse>(`/gallery/${slug}/people`);
return response.data;
},
}; };
+42
View File
@@ -113,6 +113,36 @@ export interface GalleryInfo {
login_logo_visible?: boolean; login_logo_visible?: boolean;
} }
/**
* A person clustered within one gallery (#1074).
*
* `face_count` is scoped to the photos THIS viewer can see, which is not the
* same as the number of faces stored for them a guest must never learn how
* many hidden photos someone appears in. `label` is null until the
* photographer names them; the UI shows the count instead of inventing a
* "Person 7", because a number is honest and a fake name is not.
*/
export interface GalleryPerson {
id: number;
label: string | null;
face_count: number;
cover: {
face_id: number;
photo_id: number;
/** [x, y, w, h] in ORIGINAL image pixels — scale by the rendered size. */
bbox: [number, number, number, number];
} | null;
}
export interface GalleryPeopleResponse {
people: GalleryPerson[];
scan: {
in_progress: boolean;
scanned: number;
total: number;
};
}
export interface Photo { export interface Photo {
id: number; id: number;
filename: string; filename: string;
@@ -145,6 +175,13 @@ export interface Photo {
// the lightbox download button when this is false (event-level allow_downloads // the lightbox download button when this is false (event-level allow_downloads
// also has to be true — they AND together). // also has to be true — they AND together).
category_allow_downloads?: boolean; category_allow_downloads?: boolean;
// People detected in this photo (#1074). Always present when the feature
// is on for the event — an empty array means "scanned, nobody found",
// which is different from the feature being off (see
// GalleryInfo.event.people_enabled). Hidden and ignored people are
// filtered out server-side, so this never reveals a person the
// photographer suppressed.
person_ids?: number[];
size: number; size: number;
uploaded_at: string; uploaded_at: string;
captured_at?: string; // EXIF capture date (if available) captured_at?: string; // EXIF capture date (if available)
@@ -248,6 +285,11 @@ export interface GalleryData {
// When true, the lightbox surfaces each photo's `original_filename` // When true, the lightbox surfaces each photo's `original_filename`
// alongside the position counter (#508). // alongside the position counter (#508).
use_original_filenames?: boolean; use_original_filenames?: boolean;
// "People in this gallery" (#1074). False whenever the global `faces`
// flag is off, detection is off for this event, or the photographer
// chose to keep the people strip to themselves. The whole face UI hangs
// off this one boolean.
people_enabled?: boolean;
}; };
categories?: PhotoCategory[]; categories?: PhotoCategory[];
photos: Photo[]; photos: Photo[];
+16
View File
@@ -0,0 +1,16 @@
# The conversion tool and its TensorFlow dependency never belong in the image.
tools/
tests/
README.md
LICENSES.md
__pycache__/
*.pyc
.pytest_cache/
.venv/
venv/
# Locally-produced model artifacts — the image fetches these by pinned URL
# and checksum in the `models` build stage instead.
*.onnx
*.h5
+123
View File
@@ -0,0 +1,123 @@
# picpeak-ml — optional face-detection sidecar (#1074).
#
# Debian slim rather than Alpine: onnxruntime publishes manylinux wheels for
# x86_64 and aarch64 but nothing for musl, so Alpine would mean compiling ORT
# from source on both legs of the multi-arch build. The slim base costs ~40MB
# over Alpine and saves an hour of CI per build.
# ---------------------------------------------------------------------------
# Stage 1 — fetch and verify model weights
# ---------------------------------------------------------------------------
# Weights are baked in, never downloaded at runtime: airgapped installs must
# work, and a model that changes under a running deployment would silently
# invalidate every stored embedding.
#
# Both artifacts are pinned by URL *and* SHA-256. The checksum is the point —
# an immutable-looking URL that starts serving different bytes must fail the
# build rather than quietly reshape the embedding space.
FROM python:3.12-slim AS models
ARG YUNET_URL=https://media.githubusercontent.com/media/opencv/opencv_zoo/f12e12798e8314f7c074a6656816c048dcc95b7a/models/face_detection_yunet/face_detection_yunet_2023mar.onnx
ARG YUNET_SHA256=8f2383e4dd3cfbb4553ea8718107fc0423210dc964f9f4280604804ed2552fa4
# FaceNet-512 as ONNX. deepface distributes this model as Keras .h5 only, so
# the ONNX is produced once by `tools/convert_facenet.py` and published as a
# release asset — converting inside this build would drag TensorFlow (~600MB)
# through both architecture legs to produce a file that is identical either
# way.
#
# Defaults to the canonical published artifact so `docker build ml/` and
# `docker compose --profile faces up` both work with no arguments. Override
# both together to use a different embedder. Blanking either one still fails
# the build loudly (below) rather than silently producing an image with no
# embedder — the checksum is what makes the URL safe to trust, so a URL
# without one is never acceptable.
ARG FACENET_ONNX_URL=https://github.com/PicPeak/picpeak/releases/download/ml-models-v1/facenet512.onnx
ARG FACENET_ONNX_SHA256=a1c06dcb79dc17a42af01d5bcbce4822caa148b9c24bf7eb8b8e556b4fd0d5db
RUN apt-get update \
&& apt-get install -y --no-install-recommends curl ca-certificates \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /models
RUN curl -fsSL -o face_detection_yunet_2023mar.onnx "${YUNET_URL}" \
&& echo "${YUNET_SHA256} face_detection_yunet_2023mar.onnx" | sha256sum -c -
RUN if [ -z "${FACENET_ONNX_URL}" ] || [ -z "${FACENET_ONNX_SHA256}" ]; then \
echo "ERROR: FACENET_ONNX_URL and FACENET_ONNX_SHA256 build args are required." >&2; \
echo " Produce the artifact with ml/tools/convert_facenet.py, publish it," >&2; \
echo " then pass both args. See ml/README.md." >&2; \
exit 1; \
fi \
&& curl -fsSL -o facenet512.onnx "${FACENET_ONNX_URL}" \
&& echo "${FACENET_ONNX_SHA256} facenet512.onnx" | sha256sum -c -
# ---------------------------------------------------------------------------
# Stage 2 — runtime
# ---------------------------------------------------------------------------
FROM python:3.12-slim
ARG BUILD_DATE
ARG VCS_REF
ARG VERSION
LABEL org.opencontainers.image.source="https://github.com/PicPeak/picpeak"
LABEL org.opencontainers.image.description="PicPeak ML sidecar — face detection and embedding"
LABEL org.opencontainers.image.licenses="MIT"
# Busts the apt layer each CI run so the image picks up current Debian
# security updates instead of reusing a stale cached upgrade layer — same
# reasoning as backend/Dockerfile.
ARG CACHEBUST=1
RUN echo "cachebust=${CACHEBUST}" \
&& apt-get update \
&& apt-get upgrade -y \
# libGL and libglib are opencv-python-headless's remaining shared-library
# deps. The headless wheel drops the GUI toolkits but still links libGL.
&& apt-get install -y --no-install-recommends libgl1 libglib2.0-0 \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY requirements.txt .
# pip itself is ~10MB of an image that never installs anything at runtime.
#
# The `|| true` is scoped to the uninstall ONLY. Written as
# `pip install && pip uninstall || true` the shell parses it as
# `(install && uninstall) || true`, so a failed requirements install still
# produces a green layer — and CI would publish an ML image with no FastAPI,
# no uvicorn and no onnxruntime that fails at container start instead of at
# build time.
RUN pip install --no-cache-dir -r requirements.txt \
&& { pip uninstall -y pip setuptools 2>/dev/null || true; }
# Non-root. Nothing in this container writes anything — no volumes, no
# database, no model download — so the whole filesystem can stay read-only to
# the service account.
#
# Created BEFORE the copies so ownership can be set by COPY --chown. A
# `chown -R` afterwards would rewrite every copied file into a fresh layer,
# duplicating the 90MB model and adding ~94MB to the image for nothing.
RUN useradd --system --uid 1001 --create-home picpeak
COPY --from=models --chown=picpeak:picpeak /models /models
COPY --chown=picpeak:picpeak app ./app
USER picpeak
ENV FACE_MODEL_DIR=/models \
PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1
EXPOSE 8000
# Mirrors the backend's healthcheck shape. /health is unauthenticated so this
# needs no secret; it reports liveness only, because a failed model load
# aborts startup and the container never serves at all.
HEALTHCHECK --interval=30s --timeout=5s --start-period=40s --retries=3 \
CMD python -c "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=4).status == 200 else 1)"
# Single worker on purpose: the models are loaded per process, so a second
# worker doubles RSS for a service the backend calls at concurrency 1.
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "1"]
+60
View File
@@ -0,0 +1,60 @@
# Model provenance and licences
PicPeak is run commercially by working photographers. That sets a hard rule
for this image:
> **No non-commercial artifact is ever baked into a PicPeak image.**
Every open face-recognition weight set traces back to a scraped,
research-only dataset — CASIA-WebFace, VGGFace2, MS1M, Glint360K and
WebFace260M all carry academic-use-only agreements, so "trained on clean
data" is not an option that exists. What differs, and what actually binds a
redistributor, is the grant the **distributor** places on the artifact we
copy into this image.
## Shipped in this image
| Artifact | Role | Distributor | Licence |
|---|---|---|---|
| `face_detection_yunet_2023mar.onnx` | detection | [OpenCV Zoo](https://github.com/opencv/opencv_zoo/tree/main/models/face_detection_yunet) | MIT |
| `facenet512.onnx` | embedding | [serengil/deepface](https://github.com/serengil/deepface) | MIT |
Both are redistributable. `facenet512.onnx` is converted from deepface's
published `facenet512_weights.h5` by `tools/convert_facenet.py`; the
conversion changes the container format, not the weights, so the MIT grant
carries over.
Pinned sources, verified by SHA-256 at build time:
- YuNet — `opencv/opencv_zoo` at commit `f12e12798e8314f7c074a6656816c048dcc95b7a`
`8f2383e4dd3cfbb4553ea8718107fc0423210dc964f9f4280604804ed2552fa4`
- FaceNet-512 source weights — `serengil/deepface_models` release `v1.0`
`3f76b5117a9ca574d536af8199e6720089eb4ad3dc7e93534496d88265de864f`
## Deliberately NOT shipped
| Artifact | Why not |
|---|---|
| InsightFace `buffalo_*`, `antelopev2` | **Non-commercial research only.** Commercial use requires a licence from insightface.ai ([model zoo README](https://github.com/deepinsight/insightface/blob/master/model_zoo/README.md), [#2587](https://github.com/deepinsight/insightface/issues/2587)). Available as an opt-in via `FACE_MODEL`, downloaded by the operator who has cleared that licence themselves — never by us. |
| Idiap EdgeFace | CC BY-NC-SA 4.0 — explicitly non-commercial. The best accuracy-per-parameter of the candidates, and unusable for that reason. |
| AdaFace WebFace4M weights | MIT code, non-commercial weights. |
## What that choice cost
Measured on DeepFace's own matched benchmark (LFW, aligned, cosine — the only
comparison worth anything, since every model quotes its own "LFW 99.x%" on
its own pipeline):
| Embedder | with RetinaFace | with YuNet |
|---|---|---|
| **FaceNet-512** | **98.4%** | **97.9%** |
| ArcFace | 96.6% | 96.7% |
| SFace | 92.4% | 91.0% |
Choosing YuNet over RetinaFace costs ~0.5 points. Holding the
commercial-redistribution line costs nothing beyond that — FaceNet-512 is
both the most accurate option in the table *and* MIT. Taking the headline
numbers at face value would have pointed at SFace (advertised 0.9940, actual
9192% matched) and cost seven points, which for clustering is fatal: every
false split invents a duplicate person and every false merge puts a stranger
into someone's "download my photos".
+149
View File
@@ -0,0 +1,149 @@
# picpeak-ml
Optional face-detection sidecar for PicPeak (#1074). Detects faces in one
image and returns a bounding box, five landmarks, quality signals and a
512-d embedding per face.
**Nothing else.** No database, no volumes, no state, no egress, no model
download at runtime. Clustering, person identity, thresholds and every
privacy decision live in the PicPeak backend, where the data already is.
This service forgets each image the moment it answers.
If you don't run this container, the feature does not exist — see
"Turning it on" below.
## API
All endpoints except `/health` require the `X-Face-ML-Token` header. The
service **refuses to start** without `FACE_ML_TOKEN` set, so an accidentally
published port is never a free face-detection API.
| | |
|---|---|
| `GET /health` | `{"status": "ok"}` — unauthenticated, used by the compose healthcheck |
| `GET /info` | `{detector, embedder, model_version, dim}` |
| `POST /faces` | multipart `image``{model_version, faces: [...]}` |
Each face:
```jsonc
{
"bbox": [x, y, w, h], // ORIGINAL image pixels, not detection-scaled
"score": 0.94,
"landmarks": [[x, y], ...], // 5: right eye, left eye, nose, right mouth, left mouth
"yaw": -1.42, // degrees, approximate (see pipeline.py)
"pitch": -25.33,
"blur": 2579.5, // variance of Laplacian on the aligned crop; higher = sharper
"embedding": [...] // 512 floats, L2-normalized
}
```
`404`/`400` mean "this image is a lost cause" — the backend marks the photo
failed. `5xx` and connection failures mean "try later" — the backend returns
the photo to `pending` with backoff, so turning this container off for a week
does not require a manual re-scan.
## Models
YuNet (detection, MIT) + FaceNet-512 (embedding, MIT), both baked into the
image and verified by SHA-256 at build time. See [LICENSES.md](LICENSES.md)
for why these two and not the more obvious InsightFace weights — the short
version is that InsightFace's are non-commercial-only and PicPeak's users are
working photographers.
### Building the image
`facenet512.onnx` is **not** fetched automatically, because deepface
distributes FaceNet-512 as Keras `.h5` only. Convert it once, publish it,
then pass the URL and checksum:
```bash
cd ml
python3.11 -m venv .venv && . .venv/bin/activate # 3.11: TF has no 3.12+ wheels
pip install -r tools/requirements-convert.txt
curl -fsSL -o facenet512_weights.h5 \
https://github.com/serengil/deepface_models/releases/download/v1.0/facenet512_weights.h5
echo "3f76b5117a9ca574d536af8199e6720089eb4ad3dc7e93534496d88265de864f facenet512_weights.h5" | sha256sum -c -
python tools/convert_facenet.py facenet512_weights.h5 facenet512.onnx
```
The script verifies the converted graph against the Keras original before
writing (worst observed divergence: 2.1e-06 absolute, cosine 1.0000000000)
and prints the SHA-256 to publish. Output is ~89.6 MB, 23,497,424 parameters.
Publish `facenet512.onnx` as a release asset, set the repository variables
`FACENET_ONNX_URL` and `FACENET_ONNX_SHA256` (Settings → Variables — it's a
public URL, not a secret), and CI picks it up. To build locally:
```bash
docker build -t picpeak-ml \
--build-arg FACENET_ONNX_URL=https://github.com/PicPeak/picpeak/releases/download/<tag>/facenet512.onnx \
--build-arg FACENET_ONNX_SHA256=<sha256> \
ml/
```
The conversion sits outside the Docker build because TensorFlow is ~600MB of
build dependency for a file that never ships in the final image, and the
result is architecture-independent — no reason to run it on both legs of
every multi-arch build.
**The conversion is not byte-reproducible.** Two runs with the same pinned
versions on the same machine produce functionally identical graphs (same 336
nodes, same 271 initializers, weights matching to 0.000e+00) but differ in a
few initializer names, because tf2onnx's traced-op naming is not
deterministic. So a re-conversion **will** have a different SHA-256, and that
is expected rather than a sign of tampering. The checksum pins one published
artifact so its URL cannot start serving different bytes; validating a fresh
conversion is the parity check's job, not the hash's.
## Not available on the all-in-one image
The single-container image (`Dockerfile.aio`) sets
`PICPEAK_SINGLE_CONTAINER=true`, and the backend refuses to enable face
recognition when it sees that — the feature flag cannot be switched on, and
per-event detection stays off even if a restored database says otherwise.
This is a performance decision, not a licensing or packaging one. That image
runs the backend, the frontend, SQLite and every background worker inside one
container aimed at "one photographer plus guests browsing". It has no Redis,
SQLite gives it a single writer, and it contains no ML sidecar to talk to.
Adding a second image-processing pipeline that competes with Sharp for the
same CPU and RAM would not fail loudly — it would just make the whole install
slow and appear broken.
Run the standard multi-container deployment if you want this feature.
## Turning it on
Two deliberate actions, neither of which is installing this container:
1. Enable the `faces` feature flag in PicPeak's admin settings.
2. Enable "Detect people in this gallery" per event.
`FACE_ML_URL` defaults to `http://picpeak-ml:8000` — the compose service name
— so the standard deployment needs no URL configuration. **Nothing in the
backend touches that URL while the flag is off**, so an install without this
container never attempts a connection.
## Development
```bash
pip install -r requirements.txt pytest httpx
python -m pytest tests/ -q
```
The tests stub the models out: they cover the auth boundary, the request
guards and the alignment geometry — the places where a mistake is a security
problem or a silent accuracy problem. Model *quality* is not a unit-test
question; that is what the Phase 0 spike measured.
### The one thing to be careful about
The alignment in `pipeline.py` and the normalization in `_embed` must stay
identical to whatever the clustering threshold was tuned against. A tuned
cosine threshold does not transfer across an alignment change. If either
changes, bump `MODEL_VERSION` in `config.py` — the backend keys
re-derivation off that string and will re-cluster rather than silently mix
two incompatible embedding spaces.
View File
+72
View File
@@ -0,0 +1,72 @@
"""
Configuration for the picpeak-ml sidecar (#1074).
Everything is read once at import. There is no reload path and no settings
API this service is stateless by design, and an operator changing a knob
restarts the container.
"""
import os
# ---------------------------------------------------------------------------
# Models
# ---------------------------------------------------------------------------
MODEL_DIR = os.environ.get("FACE_MODEL_DIR", "/models")
DETECTOR_FILENAME = "face_detection_yunet_2023mar.onnx"
EMBEDDER_FILENAME = os.environ.get("FACE_EMBEDDER_FILENAME", "facenet512.onnx")
DETECTOR_NAME = "yunet_2023mar"
EMBEDDER_NAME = os.environ.get("FACE_MODEL", "facenet512")
# Stamped onto every face row the backend stores. Changing the detector, the
# embedder, the alignment or the normalization MUST bump this: embeddings from
# two different pipelines are not comparable, and a silent mix produces
# clusters that look plausible and are wrong. The backend keys re-derivation
# off this string.
MODEL_VERSION = f"{DETECTOR_NAME}+{EMBEDDER_NAME}+v1"
# ---------------------------------------------------------------------------
# Auth
# ---------------------------------------------------------------------------
# Shared secret, required. `main.py` refuses to start without it rather than
# defaulting to open: an accidentally published port must not be a free
# face-detection API.
TOKEN = os.environ.get("FACE_ML_TOKEN", "").strip()
TOKEN_HEADER = "X-Face-ML-Token"
# ---------------------------------------------------------------------------
# Detection
# ---------------------------------------------------------------------------
# YuNet's own confidence floor. Deliberately permissive: the backend applies
# the *product* quality floor (score, blur, bbox size) because those
# thresholds are admin-tunable there and baked-in here. This service's job is
# to report what it saw, not to decide what counts.
DET_SCORE_THRESHOLD = float(os.environ.get("FACE_DET_SCORE_THRESHOLD", "0.6"))
NMS_THRESHOLD = float(os.environ.get("FACE_NMS_THRESHOLD", "0.3"))
TOP_K = int(os.environ.get("FACE_DET_TOP_K", "5000"))
# Guard rails on untrusted input. The backend only ever sends its own preview
# renditions (≤1920px), but this service must not fall over if something else
# reaches it.
MAX_IMAGE_BYTES = int(os.environ.get("FACE_MAX_IMAGE_BYTES", str(32 * 1024 * 1024)))
MAX_FACES = int(os.environ.get("FACE_MAX_FACES", "64"))
# Long edge the image is downscaled to before detection. Matches the backend's
# preview tier (imageProcessor.js generates ≤1920px), so the common case is a
# no-op; anything larger is scaled down here so detection cost stays bounded.
# Bboxes and landmarks are always reported in ORIGINAL image coordinates.
INPUT_LONG_EDGE = int(os.environ.get("FACE_INPUT_LONG_EDGE", "1920"))
# ---------------------------------------------------------------------------
# Runtime
# ---------------------------------------------------------------------------
# One ORT thread by default. The backend's face queue runs at concurrency 1
# (it shares a host with Sharp, which is the real memory pressure — see
# backgroundProcessor.js), so letting ORT fan out across every core buys
# nothing and costs RSS.
ORT_THREADS = int(os.environ.get("FACE_ORT_THREADS", "1"))
+133
View File
@@ -0,0 +1,133 @@
"""
picpeak-ml optional face-detection sidecar for PicPeak (#1074).
Three endpoints, no database, no volumes, no egress. Models are baked into
the image at build time, so an airgapped install works and nothing is
downloaded at runtime.
The service is deliberately dumb: it reports what it saw in one image and
forgets. Clustering, identity, thresholds and every privacy decision live in
the backend, where the data already is.
"""
import logging
from contextlib import asynccontextmanager
from fastapi import Depends, FastAPI, File, Header, HTTPException, UploadFile
from fastapi.responses import JSONResponse
from . import config
from .pipeline import FacePipeline
from .schemas import FacesResponse, HealthResponse, InfoResponse
logging.basicConfig(
level=logging.INFO, format="%(asctime)s %(levelname)s [picpeak-ml] %(message)s"
)
logger = logging.getLogger(__name__)
_pipeline: FacePipeline | None = None
@asynccontextmanager
async def lifespan(app: FastAPI):
global _pipeline
# Refuse to run open. An accidentally published port must not be a free
# face-detection API, and a service that silently accepted anonymous
# requests when the operator forgot the variable would be exactly that.
if not config.TOKEN:
raise RuntimeError(
"FACE_ML_TOKEN is not set. picpeak-ml will not start without a "
"shared secret — set it on both this container and the backend."
)
logger.info("Loading models from %s", config.MODEL_DIR)
_pipeline = FacePipeline()
logger.info(
"Ready: detector=%s embedder=%s version=%s dim=%d",
config.DETECTOR_NAME,
config.EMBEDDER_NAME,
config.MODEL_VERSION,
_pipeline.dim,
)
yield
_pipeline = None
app = FastAPI(
title="picpeak-ml",
description="Face detection and embedding sidecar for PicPeak",
lifespan=lifespan,
# No interactive docs: this is a machine-to-machine service on a private
# network, and /docs is just surface.
docs_url=None,
redoc_url=None,
openapi_url=None,
)
def require_token(x_face_ml_token: str = Header(default="")) -> None:
"""Constant-time-ish shared-secret check.
Python's `==` on str short-circuits, so this leaks a timing signal in
principle. It is not worth `hmac.compare_digest` gymnastics for a token
that only travels over a private Docker network but it IS worth
rejecting with a bare 401 and no detail, so a prober learns nothing about
whether the header name was even right.
"""
if x_face_ml_token != config.TOKEN:
raise HTTPException(status_code=401, detail="Unauthorized")
@app.get("/health", response_model=HealthResponse)
def health() -> HealthResponse:
"""Unauthenticated on purpose — the compose healthcheck calls it.
Reports only liveness. It deliberately does not confirm the models
loaded, because lifespan raises on failure and the container never
reaches a serving state at all.
"""
return HealthResponse(status="ok")
@app.get("/info", response_model=InfoResponse, dependencies=[Depends(require_token)])
def info() -> InfoResponse:
assert _pipeline is not None
return InfoResponse(
detector=config.DETECTOR_NAME,
embedder=config.EMBEDDER_NAME,
model_version=config.MODEL_VERSION,
dim=_pipeline.dim,
)
@app.post("/faces", response_model=FacesResponse, dependencies=[Depends(require_token)])
async def faces(image: UploadFile = File(...)) -> FacesResponse:
assert _pipeline is not None
data = await image.read()
if not data:
raise HTTPException(status_code=400, detail="Empty upload")
if len(data) > config.MAX_IMAGE_BYTES:
raise HTTPException(status_code=413, detail="Image too large")
try:
detected = _pipeline.process(data)
except ValueError as exc:
# Undecodable input is the caller's problem, not ours — 400 so the
# backend marks the photo 'failed' instead of retrying it forever.
raise HTTPException(status_code=400, detail=str(exc)) from exc
return FacesResponse(model_version=config.MODEL_VERSION, faces=detected)
@app.exception_handler(Exception)
async def unhandled(request, exc: Exception) -> JSONResponse:
"""Never leak a traceback to the caller.
The backend treats 5xx as "sidecar unhealthy" and puts the photo back to
pending with backoff, which is the right behaviour for a genuine internal
fault so the useful detail belongs in our log, not in the response.
"""
logger.exception("Unhandled error: %s", exc)
return JSONResponse(status_code=500, content={"detail": "Internal error"})
+313
View File
@@ -0,0 +1,313 @@
"""
Detection alignment embedding (#1074).
The alignment step in the middle is the part that decides whether this
feature works. FaceNet was trained on similarity-aligned crops; handing it a
raw bbox crop costs far more accuracy than any model swap would recover. So
YuNet's five landmarks are used to warp every face onto the same canonical
template before it ever reaches the embedder.
IMPORTANT the alignment and normalization below must stay identical to
whatever the Phase 0 spike measured its cosine threshold on. A tuned
threshold does not transfer across alignment changes; if either is touched,
`MODEL_VERSION` bumps and the backend re-derives.
"""
import threading
import cv2
import numpy as np
import onnxruntime as ort
from . import config
# Canonical 5-point template (ArcFace's, the de-facto standard), expressed for
# a 112x112 crop and scaled to whatever the embedder actually wants. Points are
# in IMAGE coordinates, left to right:
# 0 subject's right eye (appears image-left)
# 1 subject's left eye
# 2 nose tip
# 3 subject's right mouth corner
# 4 subject's left mouth corner
# YuNet emits its landmarks in exactly this order, so the mapping is index-wise
# with no reshuffling — `test_pipeline.py` pins that assumption.
_TEMPLATE_112 = np.array(
[
[38.2946, 51.6963],
[73.5318, 51.5014],
[56.0252, 71.7366],
[41.5493, 92.3655],
[70.7299, 92.2041],
],
dtype=np.float32,
)
class FacePipeline:
"""Loads both models once and serves them under a lock.
cv2.FaceDetectorYN carries mutable input-size state across setInputSize/
detect, so it is NOT safe to call from two threads. FastAPI runs sync
endpoints in a threadpool, so every inference path is serialized here.
That is not a throughput loss worth fixing: the backend's face queue
defaults to concurrency 1, and a single lock keeps RSS predictable, which
is the constraint that actually matters on a 2 GB VPS.
"""
def __init__(self) -> None:
detector_path = f"{config.MODEL_DIR}/{config.DETECTOR_FILENAME}"
embedder_path = f"{config.MODEL_DIR}/{config.EMBEDDER_FILENAME}"
self._lock = threading.Lock()
self._detector = cv2.FaceDetectorYN.create(
model=detector_path,
config="",
input_size=(320, 320), # replaced per-image via setInputSize
score_threshold=config.DET_SCORE_THRESHOLD,
nms_threshold=config.NMS_THRESHOLD,
top_k=config.TOP_K,
)
so = ort.SessionOptions()
so.intra_op_num_threads = config.ORT_THREADS
so.inter_op_num_threads = config.ORT_THREADS
self._embedder = ort.InferenceSession(
embedder_path, sess_options=so, providers=["CPUExecutionProvider"]
)
# Read the embedder's geometry off the model rather than hardcoding
# 160x160 NHWC. FACE_MODEL is documented as swappable (an operator who
# has cleared the InsightFace licence may point this at buffalo_l,
# which is 112x112 NCHW), and guessing wrong produces a confident
# garbage embedding rather than an error.
inp = self._embedder.get_inputs()[0]
shape = inp.shape
if len(shape) != 4:
raise RuntimeError(f"Embedder input must be 4-D, got {shape}")
self._input_name = inp.name
# NCHW iff the channel axis is second.
self._nchw = shape[1] == 3
self._crop_size = int(shape[2] if self._nchw else shape[1])
out = self._embedder.get_outputs()[0]
self._dim = int(out.shape[-1])
self._template = _TEMPLATE_112 * (self._crop_size / 112.0)
# -- introspection ----------------------------------------------------
@property
def dim(self) -> int:
return self._dim
# -- inference --------------------------------------------------------
def process(self, image_bytes: bytes) -> list[dict]:
"""Decode, detect, align, embed. Returns one dict per face."""
buf = np.frombuffer(image_bytes, dtype=np.uint8)
img = cv2.imdecode(buf, cv2.IMREAD_COLOR)
if img is None:
raise ValueError("Image could not be decoded")
# Downscale for detection, then map coordinates back. Everything the
# caller sees is in ORIGINAL image pixels — the backend stores bboxes
# to crop avatars from the same rendition later, so a scaled
# coordinate would silently offset every cover face.
h, w = img.shape[:2]
long_edge = max(h, w)
if long_edge > config.INPUT_LONG_EDGE:
scale = config.INPUT_LONG_EDGE / long_edge
det_img = cv2.resize(
img, (round(w * scale), round(h * scale)), interpolation=cv2.INTER_AREA
)
else:
scale = 1.0
det_img = img
with self._lock:
dh, dw = det_img.shape[:2]
self._detector.setInputSize((dw, dh))
_, raw = self._detector.detect(det_img)
if raw is None:
return []
faces = []
for row in raw[: config.MAX_FACES]:
faces.append(self._one_face(img, row, scale))
return faces
def _one_face(self, img: np.ndarray, row: np.ndarray, scale: float) -> dict:
"""Build the response entry for a single YuNet detection.
`row` is YuNet's 15-wide output: x, y, w, h, then five (x, y)
landmark pairs, then the confidence score. All in detection-image
coordinates, hence the division by `scale`.
"""
inv = 1.0 / scale
x, y, bw, bh = (float(v) * inv for v in row[0:4])
landmarks = (row[4:14].reshape(5, 2).astype(np.float32)) * inv
score = float(row[14])
aligned = self._align(img, landmarks)
embedding = self._embed(aligned)
yaw, pitch = _pose_from_landmarks(landmarks)
return {
"bbox": [x, y, bw, bh],
"score": score,
"landmarks": landmarks.tolist(),
"yaw": yaw,
"pitch": pitch,
"blur": _blur_score(aligned),
"embedding": embedding.tolist(),
}
def _align(self, img: np.ndarray, landmarks: np.ndarray) -> np.ndarray:
"""Similarity-warp the face onto the canonical template."""
matrix = _umeyama(landmarks, self._template)
if matrix is None:
# Degenerate landmarks (all coincident/collinear). Fall back to a
# plain centre crop so the face still gets an embedding rather
# than vanishing from the gallery.
matrix = _fallback_transform(landmarks, self._crop_size)
return cv2.warpAffine(
img,
matrix,
(self._crop_size, self._crop_size),
flags=cv2.INTER_LINEAR,
borderValue=0,
)
def _embed(self, aligned: np.ndarray) -> np.ndarray:
# BGR (OpenCV) → RGB (what FaceNet was trained on). Getting this
# backwards does not error, it just quietly degrades every embedding.
rgb = cv2.cvtColor(aligned, cv2.COLOR_BGR2RGB).astype(np.float32)
# deepface's "Facenet" normalization: per-image standardization. This
# is what the published FaceNet-512 benchmark numbers were produced
# with, so it is what the threshold in the backend assumes.
mean, std = rgb.mean(), rgb.std()
rgb = (rgb - mean) / max(float(std), 1e-6)
batch = rgb[None, ...]
if self._nchw:
batch = batch.transpose(0, 3, 1, 2)
vec = self._embedder.run(None, {self._input_name: batch})[0][0]
# L2-normalize so the backend's cosine similarity is a plain dot
# product and centroid means stay on the unit sphere.
norm = float(np.linalg.norm(vec))
return (vec / norm) if norm > 0 else vec
def _umeyama(src: np.ndarray, dst: np.ndarray) -> np.ndarray | None:
"""Least-squares similarity transform (Umeyama 1991) over ALL five points.
Deliberately NOT cv2.estimateAffinePartial2D. That function's estimators
are RANSAC (its default) and LMEDS, both of which exist to *reject
outliers* among many noisy correspondences. Given exactly five points and
no outliers they fit a three-point subset perfectly and let the rest
drift: measured on a real off-frontal portrait, both pinned the eyes and
nose to 0.11px and left the mouth corners 11.8px out on a 160px crop.
Umeyama distributes the residual instead (max 6.5px, rms 5.1 vs 7.4) and
is what insightface's norm_crop and skimage's SimilarityTransform use.
Also fully deterministic no random consensus sampling which matters
beyond accuracy: the same photo must embed identically on every re-scan,
or clusters churn between runs for no reason.
Returns a 2x3 affine matrix, or None if the points are degenerate.
"""
src = np.asarray(src, dtype=np.float64)
dst = np.asarray(dst, dtype=np.float64)
src_mean, dst_mean = src.mean(axis=0), dst.mean(axis=0)
src_c, dst_c = src - src_mean, dst - dst_mean
variance = float((src_c**2).sum() / len(src))
if variance < 1e-9:
return None # all points coincident
cov = dst_c.T @ src_c / len(src)
u, s, vt = np.linalg.svd(cov)
# Guard against the SVD handing back a reflection instead of a rotation —
# a mirrored face would embed as a different person.
d = np.array([1.0, 1.0])
if np.linalg.det(u @ vt) < 0:
d[-1] = -1.0
rotation = u @ np.diag(d) @ vt
scale = float((s * d).sum() / variance)
translation = dst_mean - scale * (rotation @ src_mean)
return np.hstack([scale * rotation, translation.reshape(2, 1)]).astype(np.float32)
def _fallback_transform(landmarks: np.ndarray, size: int) -> np.ndarray:
"""Centre the landmark centroid in the crop at the template's scale.
Only reached when the landmarks are degenerate enough that no similarity
transform exists. The resulting embedding will be poor, but the face
still appears in "this photo contains" rather than vanishing and the
backend's quality floor will keep it from spawning its own person.
"""
centre = landmarks.mean(axis=0)
s = size / 112.0
return np.array(
[[s, 0.0, size / 2.0 - s * centre[0]], [0.0, s, size / 2.0 - s * centre[1]]],
dtype=np.float32,
)
def _blur_score(aligned: np.ndarray) -> float:
"""Variance of the Laplacian — low means soft/out-of-focus.
Computed on the ALIGNED crop, not the original frame, so the number is
comparable between a face that fills the frame and one in the background:
both arrive here at the same pixel size. The backend's quality floor
compares against it directly.
"""
grey = cv2.cvtColor(aligned, cv2.COLOR_BGR2GRAY)
return float(cv2.Laplacian(grey, cv2.CV_64F).var())
def _pose_from_landmarks(landmarks: np.ndarray) -> tuple[float, float]:
"""Rough head pose in degrees from the five landmarks.
An approximation, deliberately: a real 3-D pose estimate needs a face
model and solvePnP, and the only consumers are a quality floor and the
"candid" auto-category rule, neither of which needs better than
"clearly turned away vs. not". Returned as degrees so the admin-facing
threshold reads in a familiar unit.
yaw negative = turned toward the subject's right, positive = left
pitch negative = looking down, positive = looking up
"""
right_eye, left_eye, nose, right_mouth, left_mouth = landmarks
eye_centre = (right_eye + left_eye) / 2.0
mouth_centre = (right_mouth + left_mouth) / 2.0
eye_span = float(np.linalg.norm(left_eye - right_eye))
if eye_span < 1e-6:
return 0.0, 0.0
# Yaw: on a frontal face the nose sits midway between the eyes. As the
# head turns, it slides toward the nearer eye. Offset is normalized by
# eye span so it is scale-free, then mapped through arcsin.
yaw_ratio = float((nose[0] - eye_centre[0]) / (eye_span / 2.0))
yaw = float(np.degrees(np.arcsin(np.clip(yaw_ratio, -1.0, 1.0))))
# Pitch: the nose sits ~40% of the way down the eye→mouth axis on a
# frontal face. Higher means the head is tilted back, lower means down.
vertical = float(mouth_centre[1] - eye_centre[1])
if abs(vertical) < 1e-6:
return yaw, 0.0
nose_ratio = float((nose[1] - eye_centre[1]) / vertical)
pitch = float(np.degrees(np.arcsin(np.clip((0.40 - nose_ratio) * 2.0, -1.0, 1.0))))
return round(yaw, 2), round(pitch, 2)
+41
View File
@@ -0,0 +1,41 @@
"""
Response models (#1074).
These are the wire contract the backend's `faceClient.js` codes against.
Changing a field name here is a breaking change for a deployment mid-upgrade,
where an old sidecar and a new backend run side by side for a few seconds.
"""
from pydantic import BaseModel, Field
class Face(BaseModel):
# [x, y, w, h] in ORIGINAL image pixels — the backend crops cover-face
# avatars from the same rendition, so these must not be detection-scaled.
bbox: list[float] = Field(min_length=4, max_length=4)
score: float
# Five (x, y) pairs: subject's right eye, left eye, nose tip, right mouth
# corner, left mouth corner.
landmarks: list[list[float]]
yaw: float
pitch: float
# Variance of the Laplacian on the aligned crop. Higher = sharper.
blur: float
# L2-normalized, `dim` floats (512 for FaceNet-512).
embedding: list[float]
class FacesResponse(BaseModel):
model_version: str
faces: list[Face]
class InfoResponse(BaseModel):
detector: str
embedder: str
model_version: str
dim: int
class HealthResponse(BaseModel):
status: str
+24
View File
@@ -0,0 +1,24 @@
# picpeak-ml runtime dependencies (#1074).
#
# Pinned exactly. This image bakes in model weights and is meant to produce
# byte-identical embeddings across rebuilds — a floating dependency that
# changes how an image is decoded or resized would silently shift the
# embedding space and invalidate every stored cluster.
#
# opencv-python-HEADLESS, not opencv-python: the GUI build pulls in X11/GTK
# for highgui, which this service never calls and which is pure attack
# surface in a container. Pinned to the 4.x line — cv2.FaceDetectorYN is
# what loads YuNet, and 5.x reworks parts of that API.
opencv-python-headless==4.14.0.94
# CPU execution provider only. Wheels exist for manylinux x86_64 AND
# aarch64, so both legs of the multi-arch build install a prebuilt wheel
# and neither compiles from source.
onnxruntime==1.29.0
numpy==2.5.2
fastapi==0.141.1
uvicorn[standard]==0.52.3
# Required by FastAPI to parse multipart/form-data — the only way images
# reach this service.
python-multipart==0.0.32
+136
View File
@@ -0,0 +1,136 @@
"""
API contract tests (#1074).
The pipeline is stubbed out these cover the auth boundary and the request
guards, which is where a mistake is a security problem rather than an
accuracy problem. Model behaviour is the spike's job, not a unit test's.
"""
import pytest
from fastapi.testclient import TestClient
TOKEN = "test-token-not-a-secret"
class StubPipeline:
dim = 512
def process(self, image_bytes: bytes):
if image_bytes == b"undecodable":
raise ValueError("Image could not be decoded")
return [
{
"bbox": [10.0, 20.0, 30.0, 40.0],
"score": 0.99,
"landmarks": [[1.0, 2.0]] * 5,
"yaw": 0.0,
"pitch": 0.0,
"blur": 123.4,
"embedding": [0.1] * 512,
}
]
@pytest.fixture
def client(monkeypatch):
from app import config, main
monkeypatch.setattr(config, "TOKEN", TOKEN)
monkeypatch.setattr(main.config, "TOKEN", TOKEN)
monkeypatch.setattr(main, "FacePipeline", StubPipeline)
with TestClient(main.app) as c:
yield c
def _image_file(data: bytes = b"fake-jpeg-bytes"):
return {"image": ("photo.jpg", data, "image/jpeg")}
class TestAuth:
def test_health_needs_no_token(self, client):
# The compose healthcheck calls this without a secret.
r = client.get("/health")
assert r.status_code == 200
assert r.json() == {"status": "ok"}
def test_info_without_token_is_401(self, client):
assert client.get("/info").status_code == 401
def test_info_with_wrong_token_is_401(self, client):
r = client.get("/info", headers={"X-Face-ML-Token": "wrong"})
assert r.status_code == 401
def test_faces_without_token_is_401(self, client):
r = client.post("/faces", files=_image_file())
assert r.status_code == 401
def test_401_body_leaks_nothing(self, client):
# A prober should not learn whether the header name was even right.
r = client.post("/faces", files=_image_file())
assert r.json() == {"detail": "Unauthorized"}
def test_startup_refuses_an_empty_token(self, monkeypatch):
from app import config, main
monkeypatch.setattr(config, "TOKEN", "")
monkeypatch.setattr(main.config, "TOKEN", "")
monkeypatch.setattr(main, "FacePipeline", StubPipeline)
# Running open is the one failure mode this service must not have.
with pytest.raises(RuntimeError, match="FACE_ML_TOKEN"):
with TestClient(main.app):
pass
class TestInfo:
def test_reports_the_model_identity_the_backend_stores(self, client):
from app import config
r = client.get("/info", headers={"X-Face-ML-Token": TOKEN})
assert r.status_code == 200
body = r.json()
assert body["model_version"] == config.MODEL_VERSION
assert body["dim"] == 512
class TestFaces:
def test_returns_faces_with_the_model_version(self, client):
from app import config
r = client.post(
"/faces", files=_image_file(), headers={"X-Face-ML-Token": TOKEN}
)
assert r.status_code == 200
body = r.json()
assert body["model_version"] == config.MODEL_VERSION
assert len(body["faces"]) == 1
assert len(body["faces"][0]["embedding"]) == 512
assert body["faces"][0]["bbox"] == [10.0, 20.0, 30.0, 40.0]
def test_empty_upload_is_400(self, client):
r = client.post(
"/faces", files=_image_file(b""), headers={"X-Face-ML-Token": TOKEN}
)
assert r.status_code == 400
def test_undecodable_image_is_400_not_500(self, client):
# 4xx matters: the backend must mark the photo failed rather than
# retry it forever, which is what it does for 5xx.
r = client.post(
"/faces",
files=_image_file(b"undecodable"),
headers={"X-Face-ML-Token": TOKEN},
)
assert r.status_code == 400
def test_oversize_upload_is_413(self, client, monkeypatch):
from app import main
monkeypatch.setattr(main.config, "MAX_IMAGE_BYTES", 10)
r = client.post(
"/faces",
files=_image_file(b"x" * 100),
headers={"X-Face-ML-Token": TOKEN},
)
assert r.status_code == 413
+156
View File
@@ -0,0 +1,156 @@
"""
Unit tests for the parts of the pipeline that don't need model weights.
The pose and blur helpers are pure functions over landmark geometry, and the
landmark ORDER assumption is the one thing in this service that fails
silently if it's wrong — a shuffled template still produces 512 confident
floats, just from a face warped inside out. So it gets pinned here.
"""
import numpy as np
import pytest
from app.pipeline import (
_TEMPLATE_112,
_blur_score,
_pose_from_landmarks,
_umeyama,
)
def _frontal_landmarks() -> np.ndarray:
"""A synthetic, perfectly frontal face in YuNet's landmark order.
Order: subject's right eye, left eye, nose, right mouth, left mouth.
The subject's right eye appears on the IMAGE-left, so it carries the
smaller x the same convention `_TEMPLATE_112` encodes.
"""
return np.array(
[
[40.0, 50.0], # right eye (image-left)
[80.0, 50.0], # left eye
[60.0, 70.0], # nose, centred between the eyes
[45.0, 92.0], # right mouth
[75.0, 92.0], # left mouth
],
dtype=np.float32,
)
class TestTemplate:
def test_landmark_order_is_left_to_right_for_paired_features(self):
# Eyes: index 0 must sit left of index 1. Mouth corners: 3 left of 4.
assert _TEMPLATE_112[0][0] < _TEMPLATE_112[1][0]
assert _TEMPLATE_112[3][0] < _TEMPLATE_112[4][0]
def test_nose_sits_between_the_eyes_horizontally(self):
assert _TEMPLATE_112[0][0] < _TEMPLATE_112[2][0] < _TEMPLATE_112[1][0]
def test_features_are_vertically_ordered_eyes_nose_mouth(self):
eye_y = (_TEMPLATE_112[0][1] + _TEMPLATE_112[1][1]) / 2
mouth_y = (_TEMPLATE_112[3][1] + _TEMPLATE_112[4][1]) / 2
assert eye_y < _TEMPLATE_112[2][1] < mouth_y
class TestUmeyama:
"""The alignment estimator. Every test here is a regression guard.
An estimator that fits three of the five landmarks perfectly and lets the
mouth drift still produces a face-shaped crop and 512 confident floats
it just degrades every embedding. That is why this is pinned numerically
rather than eyeballed.
"""
def test_recovers_a_known_similarity_transform_exactly(self):
src = _frontal_landmarks()
angle = np.radians(20.0)
rot = np.array(
[[np.cos(angle), -np.sin(angle)], [np.sin(angle), np.cos(angle)]]
)
dst = (2.5 * (src @ rot.T)) + np.array([17.0, -9.0])
m = _umeyama(src, dst.astype(np.float32))
projected = (src @ m[:, :2].T) + m[:, 2]
assert np.allclose(projected, dst, atol=1e-3)
def test_distributes_residual_across_all_five_points(self):
# A face whose eye-to-mouth proportion differs from the template —
# no similarity transform can satisfy all five, so the question is
# how the error is spread. An outlier-rejecting estimator (RANSAC,
# LMEDS) parks it all on the mouth; least squares shares it out.
src = _frontal_landmarks()
src[3][1] = 110.0 # mouth further from the eyes than the template
src[4][1] = 110.0
m = _umeyama(src, _TEMPLATE_112)
residual = np.linalg.norm((src @ m[:, :2].T) + m[:, 2] - _TEMPLATE_112, axis=1)
assert residual.max() > 0, "expected an imperfect fit for this input"
# No single point may absorb the bulk of the error.
assert residual.max() < 3.0 * residual.mean()
def test_never_returns_a_reflection(self):
# A mirrored warp yields a confident embedding of a face that does
# not exist, which would cluster as a separate person.
m = _umeyama(_frontal_landmarks(), _TEMPLATE_112)
assert np.linalg.det(m[:, :2]) > 0
def test_is_deterministic(self):
# Re-scans must not churn clusters.
src = _frontal_landmarks()
first = _umeyama(src, _TEMPLATE_112)
for _ in range(5):
assert np.array_equal(_umeyama(src, _TEMPLATE_112), first)
def test_coincident_points_return_none_rather_than_dividing_by_zero(self):
assert _umeyama(np.zeros((5, 2), dtype=np.float32), _TEMPLATE_112) is None
class TestPose:
def test_frontal_face_is_near_zero_yaw(self):
yaw, _ = _pose_from_landmarks(_frontal_landmarks())
assert abs(yaw) < 1.0
def test_nose_toward_subject_left_eye_gives_positive_yaw(self):
lm = _frontal_landmarks()
lm[2][0] = 75.0 # nose slides toward the image-right (subject's left)
yaw, _ = _pose_from_landmarks(lm)
assert yaw > 10.0
def test_nose_toward_subject_right_eye_gives_negative_yaw(self):
lm = _frontal_landmarks()
lm[2][0] = 45.0
yaw, _ = _pose_from_landmarks(lm)
assert yaw < -10.0
def test_yaw_is_scale_invariant(self):
lm = _frontal_landmarks()
lm[2][0] = 72.0
small, _ = _pose_from_landmarks(lm)
large, _ = _pose_from_landmarks(lm * 4.0)
assert small == pytest.approx(large, abs=0.01)
def test_nose_low_on_the_eye_mouth_axis_reads_as_looking_down(self):
lm = _frontal_landmarks()
lm[2][1] = 85.0 # nose drops toward the mouth
_, pitch = _pose_from_landmarks(lm)
assert pitch < 0
def test_degenerate_landmarks_do_not_raise(self):
flat = np.zeros((5, 2), dtype=np.float32)
assert _pose_from_landmarks(flat) == (0.0, 0.0)
class TestBlur:
def test_flat_image_scores_near_zero(self):
flat = np.full((160, 160, 3), 128, dtype=np.uint8)
assert _blur_score(flat) < 1.0
def test_sharp_edges_score_higher_than_a_blurred_copy(self):
import cv2
sharp = np.zeros((160, 160, 3), dtype=np.uint8)
sharp[:, ::8] = 255 # high-frequency vertical stripes
blurred = cv2.GaussianBlur(sharp, (15, 15), 0)
assert _blur_score(sharp) > _blur_score(blurred)
+113
View File
@@ -0,0 +1,113 @@
"""
Phase 0 spike (#1074): does our pipeline separate different people?
Runs LFW's standard 1000-pair test protocol (500 same, 500 different) through
the PRODUCTION FacePipeline YuNet detection, our Umeyama alignment, our
per-image standardization, FaceNet-512 ONNX and reports the cosine
distribution, the best threshold, and the error rates that matter for
CLUSTERING specifically.
Clustering is not verification. For verification a false accept and a false
reject cost the same. For clustering they do not: a false merge puts a
stranger into someone's "download my photos", while a false split just makes
a duplicate row in the strip that the photographer can merge away. So the
operating point is chosen to hold false merges low, not to maximise accuracy.
"""
import os
import sys
import numpy as np
import cv2
# Run from the ml/ directory with FACE_MODEL_DIR pointing at a directory
# holding face_detection_yunet_2023mar.onnx and facenet512.onnx:
#
# pip install -r requirements.txt scikit-learn
# FACE_MODEL_DIR=/path/to/models python tools/benchmark_threshold.py
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
os.environ.setdefault('FACE_MODEL_DIR', '/models')
from app.pipeline import FacePipeline # noqa: E402
from sklearn.datasets import fetch_lfw_pairs # noqa: E402
print('Loading LFW test pairs…')
data = fetch_lfw_pairs(subset='test', color=True, resize=1.0,
slice_=(slice(0, 250), slice(0, 250)), funneled=True)
pairs, labels = data.pairs, data.target
print('Loading pipeline…')
pipe = FacePipeline()
def embed(arr):
"""sklearn hands back float32 RGB normalized to [0, 1] — NOT 0..255.
Casting straight to uint8 produces a black frame and zero detections."""
rgb8 = np.clip(arr * 255.0, 0, 255).astype(np.uint8)
bgr = cv2.cvtColor(rgb8, cv2.COLOR_RGB2BGR)
ok, buf = cv2.imencode('.jpg', bgr, [cv2.IMWRITE_JPEG_QUALITY, 95])
if not ok:
return None
faces = pipe.process(buf.tobytes())
if not faces:
return None
# Largest detection — LFW is one centred subject per frame.
best = max(faces, key=lambda f: f['bbox'][2] * f['bbox'][3])
return np.asarray(best['embedding'], dtype=np.float32)
sims, kept, missed = [], [], 0
for i, (a, b) in enumerate(pairs):
ea, eb = embed(a), embed(b)
if ea is None or eb is None:
missed += 1
continue
sims.append(float(ea @ eb))
kept.append(int(labels[i]))
if (i + 1) % 100 == 0:
print(f' {i+1}/{len(pairs)}', flush=True)
sims = np.array(sims)
kept = np.array(kept)
same, diff = sims[kept == 1], sims[kept == 0]
print()
print('=' * 66)
print(f'Pairs evaluated : {len(sims)} of {len(pairs)} '
f'({missed} skipped — no face detected in one or both)')
print(f'Detection rate : {1 - missed/len(pairs):.1%}')
print()
print(f'SAME person cosine: mean {same.mean():.4f} sd {same.std():.4f} '
f'p5 {np.percentile(same,5):.4f} min {same.min():.4f}')
print(f'DIFF person cosine: mean {diff.mean():.4f} sd {diff.std():.4f} '
f'p95 {np.percentile(diff,95):.4f} max {diff.max():.4f}')
print(f'Separation (mean gap): {same.mean() - diff.mean():.4f}')
# Sweep thresholds.
grid = np.linspace(0.0, 1.0, 1001)
acc = [( (same >= t).sum() + (diff < t).sum() ) / len(sims) for t in grid]
best_i = int(np.argmax(acc))
best_t, best_acc = grid[best_i], acc[best_i]
print()
print(f'Best accuracy : {best_acc:.2%} at threshold {best_t:.3f}')
# The clustering-relevant operating points: pick the threshold where the
# false-MERGE rate (different people scored as the same) is capped.
print()
print('Operating points (false merge = different people judged the same):')
print(f' {"thresh":>7} {"false merge":>11} {"false split":>11} {"accuracy":>8}')
for target in (0.10, 0.05, 0.02, 0.01):
t = float(np.quantile(diff, 1 - target))
fm = (diff >= t).mean()
fs = (same < t).mean()
a = ((same >= t).sum() + (diff < t).sum()) / len(sims)
print(f' {t:7.3f} {fm:10.1%} {fs:10.1%} {a:7.1%} (target {target:.0%})')
print()
print(f'Currently seeded default: 0.620 → '
f'false merge {(diff >= 0.62).mean():.1%}, '
f'false split {(same < 0.62).mean():.1%}, '
f'accuracy {(((same >= 0.62).sum() + (diff < 0.62).sum()) / len(sims)):.1%}')
print('=' * 66)
# Last run (2026-08-18), 1000/1000 pairs, 100% detection:
# same 0.6958 +/- 0.1415 | diff 0.0849 +/- 0.1674 | separation 0.6109
# peak accuracy 96.60% @ 0.405
# shipped default 0.50 -> 1.0% false merge, 8.2% false split, 95.4% accuracy
+158
View File
@@ -0,0 +1,158 @@
#!/usr/bin/env python3
"""
Produce `facenet512.onnx` from deepface's published Keras weights (#1074).
Run this ONCE, by hand. Publish the resulting file as a release asset and
pass its URL + SHA-256 to the image build. It is deliberately not part of the
Docker build:
1. TensorFlow is ~600MB of build dependency for a file that never ships in
the final image.
2. The result is architecture-independent, so converting once beats
converting on both legs of every multi-arch build.
NOT byte-reproducible. Two runs on the same machine with the same pinned
versions produce functionally identical graphs same 336 nodes, same 271
initializers, weights matching to 0.000e+00 but a handful of initializer
names differ (tf2onnx's traced-op naming is not deterministic), so the file
bytes and therefore the SHA-256 differ. Measured, not assumed.
The consequence for anyone re-running this: **your checksum will not match
the published one, and that is expected it is not evidence of tampering.**
The SHA-256 in the image build pins one specific published artifact so that
URL cannot start serving different bytes. To validate a fresh conversion,
rely on the parity check below (which compares against the Keras original),
not on reproducing a hash.
Why we may redistribute at all: deepface ships FaceNet-512 under MIT. That
was the deciding factor over the more accurate InsightFace weights, which are
non-commercial only see ml/LICENSES.md and #1074 §1.
Usage
-----
python3.11 -m venv .venv && . .venv/bin/activate # 3.11: TF has no 3.12+ wheels
pip install -r tools/requirements-convert.txt
curl -fsSL -o facenet512_weights.h5 \\
https://github.com/serengil/deepface_models/releases/download/v1.0/facenet512_weights.h5
echo "3f76b5117a9ca574d536af8199e6720089eb4ad3dc7e93534496d88265de864f facenet512_weights.h5" \\
| sha256sum -c -
python tools/convert_facenet.py facenet512_weights.h5 facenet512.onnx
The script verifies the converted graph against the Keras original before it
writes anything permanent, then prints the SHA-256 to publish alongside it.
"""
import argparse
import hashlib
import sys
from pathlib import Path
# Input geometry of deepface's FaceNet-512. pipeline.py reads this off the
# model at runtime rather than assuming it, but this is what it will find.
INPUT_SHAPE = (None, 160, 160, 3)
EMBEDDING_DIM = 512
# A converted graph that is subtly wrong still returns 512 plausible floats,
# so parity is checked rather than assumed. Tolerance is float32 noise: the
# observed worst case over random inputs was 2.1e-06 absolute, cosine
# 1.0000000000.
PARITY_SAMPLES = 3
MIN_COSINE = 0.99999
def _sha256(path: Path) -> str:
h = hashlib.sha256()
with path.open("rb") as fh:
for chunk in iter(lambda: fh.read(1024 * 1024), b""):
h.update(chunk)
return h.hexdigest()
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("weights", type=Path, help="facenet512_weights.h5")
parser.add_argument("output", type=Path, help="destination .onnx")
parser.add_argument(
"--opset",
type=int,
default=17,
help="ONNX opset (default 17 — supported by onnxruntime 1.29)",
)
parser.add_argument(
"--skip-verify",
action="store_true",
help="skip the Keras/ONNX parity check (not recommended)",
)
args = parser.parse_args()
if not args.weights.is_file():
print(f"error: {args.weights} not found", file=sys.stderr)
return 1
# Imported late so `--help` works without a TensorFlow install.
import numpy as np
import tensorflow as tf
import tf2onnx
from deepface.models.facial_recognition.Facenet import InceptionResNetV1
# Build the architecture and load the checksummed file from disk rather
# than going through deepface's own loader — that one downloads the
# weights itself, which would defeat the point of pinning them.
print("Building InceptionResNetV1(dimension=512)…")
model = InceptionResNetV1(dimension=EMBEDDING_DIM)
model.load_weights(str(args.weights))
print(f" {model.count_params():,} parameters")
print(f"Converting to ONNX (opset {args.opset})…")
spec = (tf.TensorSpec(INPUT_SHAPE, tf.float32, name="input"),)
tf2onnx.convert.from_keras(
model, input_signature=spec, opset=args.opset, output_path=str(args.output)
)
if not args.skip_verify:
import onnxruntime as ort
print("Verifying ONNX output matches Keras…")
sess = ort.InferenceSession(
str(args.output), providers=["CPUExecutionProvider"]
)
name = sess.get_inputs()[0].name
rng = np.random.default_rng(0)
worst_cosine, worst_abs = 1.0, 0.0
for _ in range(PARITY_SAMPLES):
x = rng.standard_normal((1, *INPUT_SHAPE[1:])).astype("float32")
keras_out = model.predict(x, verbose=0)[0]
onnx_out = sess.run(None, {name: x})[0][0]
worst_abs = max(worst_abs, float(np.abs(keras_out - onnx_out).max()))
cosine = float(
(keras_out / np.linalg.norm(keras_out))
@ (onnx_out / np.linalg.norm(onnx_out))
)
worst_cosine = min(worst_cosine, cosine)
print(f" worst abs diff {worst_abs:.3e}, worst cosine {worst_cosine:.10f}")
if worst_cosine < MIN_COSINE:
print(
f"error: parity check FAILED (cosine {worst_cosine} < {MIN_COSINE}). "
"The converted graph does not match the original — do not publish it.",
file=sys.stderr,
)
args.output.unlink(missing_ok=True)
return 1
size_mb = args.output.stat().st_size / (1024 * 1024)
print()
print(f"Wrote {args.output} ({size_mb:.1f} MB)")
print(f"SHA-256: {_sha256(args.output)}")
print()
print("Publish it as a release asset, then set the repository variables")
print("FACENET_ONNX_URL and FACENET_ONNX_SHA256 (Settings → Variables).")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+22
View File
@@ -0,0 +1,22 @@
# Dependencies for tools/convert_facenet.py ONLY. These never enter the
# picpeak-ml image — the conversion runs once, by hand, and only its output
# (facenet512.onnx) is shipped.
#
# Pinned to the exact set that produced the published artifact, so anyone can
# reproduce it and get the same SHA-256 rather than a file that differs for
# reasons nobody can reconstruct later. See ml/README.md for the expected
# checksum.
#
# Requires Python 3.11 (TensorFlow has no 3.12+/3.14 wheels at these
# versions). The picpeak-ml image itself runs 3.12 — the two are unrelated,
# since nothing from this file ships.
tensorflow==2.21.0
tf2onnx==1.17.0
onnx==1.22.0
deepface==0.0.100
# Transitive, pinned because the conversion output is checksummed:
# keras 3.15 builds the graph, protobuf serializes the ONNX.
keras==3.15.1
protobuf==7.35.1
numpy==2.4.6