2072 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Paul Nothaft	06da1b9f7e	fix(external-media): one row per external file per event (#1162) (#1167) ... * fix(external-media): one row per external file per event (#1162) Two overlapping import-external runs against the same event inserted every file twice. The route checked for an existing external_relpath and then inserted, with an fs.stat and a sharp().metadata() read sitting in between — a window wide enough for both runs to see "not there". A reporter's event held 8004 rows for 6012 distinct paths. Nothing at the storage layer stopped it: migration 041 created only a NON-unique (event_id, source_origin) index. - migration 186 removes the duplicates that already exist and adds a partial unique index on (event_id, external_relpath). The survivor is the lowest id that has a thumbnail, so a half-finished import does not cost a grid tile, and hero references are repointed first because the FK is SET NULL. - the route treats a unique violation as a skip and carries on, so a second writer this process cannot see (another replica) converges instead of duplicating or 500ing. - a second import while one is already running now gets a 409 rather than walking the whole tree to have every insert bounce. The duplicates' thumbnail files are left behind as unreferenced bytes — a migration is the wrong place to reach into storage, which may be S3. * fix(external-media): keep dependent rows and legacy restores intact (#1162) External review found two real defects in the dedupe half of this fix. Dangling rows on SQLite. Every FK into photos declares ON DELETE CASCADE, but PicPeak never sets `PRAGMA foreign_keys = ON` — the codebase says so where it deletes an event (adminEvents/helpers.js:245) — so on every SQLite install the cascade is inert and deleting a duplicate photo left its face embeddings, guest feedback and admin marks behind, pointing at an id that no longer exists. Biometric data outliving its photo is exactly the invariant the event delete goes out of its way to hold. Dependents are now handled explicitly, and moved rather than discarded where they can be: the duplicates were separate tiles in the grid, so a guest's comment or an admin's rating could legitimately be on either, and dropping it inside a fix for silent data loss would be its own bug. Where the target already holds an equivalent row — the same guest's like, the same admin's mark, the same transfer's entry — the loser is dropped, because those tables mean one row per (photo, actor). photo_faces is the deliberate exception: both rows were scanned, so moving would duplicate every embedding and split the person clusters built from them. Legacy restores. Suspending FK enforcement does not suspend a UNIQUE index on either engine, so a .picpeak backup taken before migration 186 — carrying exactly the duplicates it removes — would hit the new index mid-batchInsert and roll the whole restore back, after every table had already been emptied. The restore now drops the index for the load and rebuilds it after running the same dedupe. Also: a failed CREATE INDEX is no longer swallowed. Recording the migration as applied without it leaves the install permanently racy, with nothing to trigger a retry. The shared work moves to services/externalPhotoDedupe.js, which the migration and the restore both call. * fix(external-media): reconcile derived state around the dedupe (#1162) Second review round, four more real findings. The index throw did not actually stop anything. run-migrations-safe.js treats 23505 as "schema already exists" and marks the migration applied (run-migrations-safe.js:138) — and a CREATE UNIQUE INDEX that finds duplicate rows raises exactly 23505 on Postgres. A replica inserting one between the dedupe and the index lock is a real rolling-deploy shape, and the outcome was the thing the throw was added to prevent. The index is now verified against the catalog afterwards, and failure raises a code-less error the runner cannot mistake for idempotence. Two people sharing a device were treated as one. photo_feedback carries both guest_identifier (per device) and guest_id (per person, migration 078), and feedbackService scopes by guest_id when present. Keying equivalence on the identifier alone deleted one of two different people's ratings. It now uses the same COALESCE rule the service does. Deleting faces raw left ghost people. event_people counts and centroids are derived from the photo_faces rows being removed, and #1132's separation snapshots hold a copy of each side's centroid — which is why faceProcessor exposes purgePhotoFaces and says it is "called from every photo-deletion path". The dedupe now goes through it. Reparenting feedback left the survivor's totals stale. photos carries denormalized feedback_count / like_count / average_rating / favorite_count and the later reaction and colour counts, so a survivor that now owns feedback kept rendering zero. updatePhotoFeedbackStats takes a trx so the dedupe can recompute on its own connection. Also: the equivalence-key delimiter was a literal NUL byte, which made git classify the whole file as binary and hide its diff. Escaped. * fix(external-media): stop the dedupe discarding half-states (#1162) Third review round. Five findings, four applied. - is_hidden joins the feedback equivalence key. feedbackService lets a moderator-hidden row coexist with the guest's visible replacement and counts only the visible one, so ignoring it deleted the visible row as redundant. - admin marks merge instead of dropping. rating and color_label are written independently, so the same admin can have rated one tile and coloured the other; the loser now hands over any field the winner has no value for. - a survivor that loses the only completed scan is requeued. Otherwise the purge takes the sole embeddings and nothing re-queues it — the photo just silently stops having a face. - view_count and download_count are carried over. Those are real interactions recorded per row, and dropping them quietly lowered the engagement the admin grid shows. Not applied: repointing a category hero can in principle land on a survivor in another category. It needs the two duplicate rows to have been re-categorised apart after the racing import, and the result is a cosmetic hero mismatch that the admin category routes already guard on write. Not worth the extra branch in a data migration. * fix(external-media): invalidate the download zip when duplicates are removed (#1162) External review of the stable twin. Applies to both branches. The pre-built "download everything" archive still contained the duplicate rows the dedupe had just deleted, so guests kept receiving them until something else happened to invalidate it. Every ordinary photo-deletion path calls downloadZipService.invalidate for exactly this reason. The columns are cleared rather than the service being called: that service carries debounce timers and a regeneration queue, which is not something a migration should start. getZipInfo already treats a cleared record as a cache miss and rebuilds on the next request, so this is the durable half of what invalidate does. The stale object is left in storage for the same reason the duplicates' thumbnails are — a migration is the wrong place to reach into a backend that may be S3. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-08-26 08:28:48 +02:00
Paul Nothaft	bb5d496495	chore(main): release 3.115.0-beta.0 (#1158) v3.115.0-beta.0	2026-08-23 20:17:29 +00:00
Paul Nothaft	c305ad4146	feat(faces): make "not the same person" survive a re-scan (#1132) (#1145) ... A separation — an explicit dismissal, or the implicit one a Split records — was stored as a pair of event_people.id. Those ids do not survive re-derivation: recluster() deletes every person, and a full re-scan replaces a photo's faces outright, so face ids die too. The only thing that survives both is the embedding, so the decision is keyed on the two centroids the pair had when the photographer separated them. It binds while both sides still look like the clusters that were separated, and lapses once they have drifted past recognition. The constraint is now honoured at assignment time as well as in consolidate(), which is what makes it hold across a re-scan rather than being reformed before any later pass could object. Six review rounds shaped the matching itself: each candidate must resolve to the OPPOSITE side rather than merely matching something (a split leaves two similar halves, and the loose test fragmented the person the split was not even about); assignment judges both sides at the ordinary match threshold, since a single face — or a cluster of one part-way through a recluster — cannot resemble a settled centroid; separations carry their own model_version; and the projections are hoisted out of the innermost loop, which took a 2000-photo scan from ~15s of dot products to 0.23s. Lifecycle closed three ways: purgePhotoFaces re-anchors each side onto the live cluster it still describes and drops rows that describe nothing left, deleteEventCascade and the permanent archive delete clear the table (which deliberately has no event FK), and a later manual merge drops the separations it reverses. All of it matched on vectors rather than ids, since a row that has outlived a recluster names people who no longer exist. Merged with admin privileges: the author cannot self-approve.	2026-08-23 22:10:28 +02:00
Paul Nothaft	2c81888eaf	fix(gallery): a guest's own hidden feedback is hidden from them too (#1150) (#1153) ... Everything in the system treats a hidden row as absent — getPhotoFeedback drops it even for the guest's own feedback, and updatePhotoFeedbackStats does not count it. The per-viewer is_liked heart and my_color_label badge read the row without looking at is_hidden, so a like the photographer had hidden still showed as liked on a photo whose like_count was zero. Making those agree exposes why it had not been fixed: the duplicate check behind like/favorite toggling did not skip hidden rows either, so the now-empty heart, when clicked, found the hidden row and toggled it OFF — the click did nothing visible and the moderation was silently undone. Skipping hidden rows there makes the click create a fresh, visible row. Review found four more surfaces still treating a hidden row as present: the per-guest caps (an at-cap guest with one hidden met their own click with limit_reached), /my-feedback (which drives the Liked/Favorited/Rated chips in guest identity mode), getEventFeedbackSummary (disagreeing with the photo counters in the same response), and unhide (leaving two visible rows for one guest). The rating-clear and single-value delete scopes are visible-only now, so a follow-up mutation no longer destroys the admin's hidden record, and the unhide collapse is skipped when there is no stable identity to scope by — that fallback was 'guest_identifier IS NULL', i.e. other visitors' rows. Not taken: refusing to hide non-comment feedback, which the issue recommended. #839 and #1044 both ship hiding for reactions and colour labels with tests asserting a hidden one stops counting; only the admin UI's Hide button is comment-only. Merged with admin privileges: the author cannot self-approve.	2026-08-23 22:09:07 +02:00
Paul Nothaft	00b20b2d72	fix(gallery): guest filters respect show_feedback_to_guests, and marks survive a mid-write clear (#1147) ... Two follow-ups from the review of #1137. Filters were a second way to read hidden feedback. Every token on /photos is an OR of two halves: what THIS viewer marked, and what ANYONE marked. The response fields built from the second half — like_count, comment_count, color_label_count — are all gated on show_feedback_to_guests. The filter was not, so with the setting off a guest could still send ?filter=liked and get back exactly the photos other people liked, across all five tokens. Reachable by a direct API caller holding a gallery token; the frontend never sends filter to this endpoint. The half it left standing was also the wrong half. It read guest_identifier from the guest_id QUERY PARAMETER, which never matched anything — the frontend invents that string in localStorage and never sends it when submitting feedback, while submissions store generateGuestIdentifier(req). So gating the aggregate would have emptied these filters rather than narrowing them to 'mine', and accepting a caller-supplied identifier was a way back through the gate. Resolved from the request now, hidden rows excluded to match what the viewer can see. A mark whose row is cleared mid-write lost its value. #1137 fixed two calls both writing; this is one clearing while another sets. The clear empties the row, the row is deleted for being empty, and the setter's update matches nothing — the caller told 'no mark'. A zero-row update now reports itself and the caller re-reads, bounded at three passes, throwing rather than reporting a success that did not happen. Merged with admin privileges: the author cannot self-approve.	2026-08-23 22:07:50 +02:00
Paul Nothaft	e4a8be8e7e	fix(gallery): no Logout button on galleries that don't require a password (#1149) (#1152) ... showLogout was hard-coded true, so a gallery with no password showed a Logout button. Logging out of it is meaningless — no credential to drop, nothing to return to — and it stranded the visitor: GalleryPage's auto-login is a one-shot latch, so clearing the session left the page on its skeleton until a manual reload. That is the 'turns blank' in the report. The button is gated on requiresPassword || isClient || viaCustomer at both call sites. The full-page layouts render it on the callback being present rather than on a flag, so withholding the callback is how the gate reaches them. Session kind now comes from /auth/session rather than sessionStorage, which is per-tab while the cookie is per-browser: a gallery reopened in a second tab lost 'client' while the backend kept serving it as one. viaCustomer marks a portal token, which bypasses reveal mode and so is a credential that does not look like one. The public-gallery branch no longer returns the skeleton unconditionally — once auto-login has run and left us unauthenticated it shows the reason and a Retry. That state was otherwise unrecoverable, and it also swallowed loginError entirely. Merged with admin privileges: the author cannot self-approve.	2026-08-23 22:06:47 +02:00
Paul Nothaft	b581267031	fix(scripts): regenerate-thumbnails resolves external sources through ensureThumbnail (#1148) (#1151) ... The CLI fallback carried the defect #1129 fixed in the admin route: it computed `storage/events/active/<photo.path>` and fs.access'd it, a location that does not exist for external or reference rows. Every one failed the check and was counted as an error, so on an external-media install the script was inert while reporting one error per photo. Resolution now goes through ensureThumbnail, which already branches on source_origin and owns the per-photo ext<id>_ output name — sharing it is what stops the script and the route drifting apart again. Also: videos skipped on every marker they can carry (fileWatcher writes type and mime_type but never media_type), responsive tiers backfilled alongside the canonical rendition, skip-vs-generate asked from isThumbnailValid rather than inferred from an unchanged path, tier failures counted rather than swallowed, and a nonzero exit when the backfill was incomplete. The script is now importable with the CLI behind a require.main guard; it previously ran on require and called process.exit, so it could not be tested at all — which is why this survived #1129. Merged with admin privileges: the author cannot self-approve.	2026-08-23 22:06:11 +02:00
Paul Nothaft	66a9b5eba1	chore(main): release 3.114.0-beta.0 (#1146) v3.114.0-beta.0	2026-08-23 09:20:30 +00:00
Luca	e2844d1909	feat(gallery): colour labels for client proofing, and one global default per feedback type (#1044) (#1137) ... Colour labels for client proofing, plus the photographer's own stars and colours in the admin grid. - Guest colour labels alongside likes/reactions, opt-in per event (defaults off so live galleries do not change mid-proofing), with 'colors' and 'lightroom' keybind schemes. - One global default per feedback type, replacing the per-type scatter. - Admin marks live in their own table (photo_admin_marks) so they can never reach a guest-facing surface. - XMP export prefers a real label, keeping the rating-derived mapping as a fallback. Review: concurrent-write loss on the mark update path, migration index idempotency and error classification all fixed in 7139bcae; migrations renumbered to 182/183 in 8fecdfae after 180/181 were taken on main. Merged with admin privileges: bypass-size-gate is a required check that fails on size alone for review-bypass authors and never re-evaluates on review, which is its designed behaviour once a maintainer has approved.	2026-08-23 11:15:01 +02:00
Paul Nothaft	7b77bbf243	chore(main): release 3.113.0-beta.0 (#1144) v3.113.0-beta.0	2026-08-22 19:46:22 +00:00
Paul Nothaft	3583c924da	feat(faces): consolidate look-alike clusters after a scan, and suggest the rest (#1107) ... consolidate() has existed since #1074 and described this exact symptom in its own comment, but its only caller was recluster() — i.e. when an admin pressed Re-group people. After a normal background scan the centroids converged and nobody looked, so a gallery settled with 14 people that should have been 8. It now runs when a scan drains. There is no scan-finished event to hook, so an idle worker asks whether the events it touched have actually drained — 'a worker went idle' is deliberately not treated as sufficient, because with concurrency above one the others may still be working. The uncertain band asks instead of acting: pairs between the assignment threshold and the stricter auto-merge one surface as accept/dismiss suggestions, with sticky dismissals. Nothing merges silently — a pass that merged anything reports it and points at Split. Review rounds hardened it against overruling explicit decisions: it no longer absorbs ignored clusters (mergePeople ORs is_ignored onto the survivor, which would have hidden a real person), no longer merges dismissed pairs, no longer undoes a manual Split (which now records a separation), and no longer runs after detection is switched off. The dismissal read fails closed, a failed pass is retried with backoff rather than lost or hot-looped, and the new table follows event_people out of exports and backups. Name autocomplete needs no endpoint — the people list already open is the source, and it is event-scoped on purpose. Known limitation, tracked in #1132: separations are keyed on person ids, so a full re-scan loses them. Reported by @BraynArts.	2026-08-22 21:39:15 +02:00
Paul Nothaft	25fbefc703	docs(faces): link the face-recognition guidance from where people look (#1125) ... Every other feature routes readers to docs.picpeak.app. Face recognition was the one that either pointed somewhere else or pointed at nothing — poor placement for the feature with the highest read-before-you-enable burden anything here ships. .env.example referenced docs/feature-face-recognition.md, which does not exist — and creating it is not the fix, because .gitignore:89 ignores docs/feature-*.md outright, so the file would be invisible to anyone who cloned. That was the only pointer to legal guidance an operator got while editing the variables that turn Art. 9 processing on. Also: the README linked the sidecar's developer README for the feature name and had no row in the documentation table, docs/single-container.md left readers who wanted the feature nowhere to go, ml/README.md had no backlink, and the admin consent callout had no link at all. It does now, inline at the end of the obligation. Reported by @Luca-Timo.	2026-08-22 21:38:25 +02:00
Paul Nothaft	87115b28e8	fix(gallery): give masonry tiles their real shape back (#1130, #1131) ... Two independent causes of the same symptom — an aspect-ratio layout that does not lay anything out. gallery-premium discarded the tile height MasonryPhotoAlbum computed from photos.width/height and set height:auto on both card and image, so the rendered shape came from the intrinsic ratio of whatever rendition was served. With thumbnail_fit seeded 'cover' by migration 040 every rendition is square, so the layout drew identical squares and was indistinguishable from grid. The card now uses the height it is given and the stylesheet's existing height:100% applies. The bundled CSS templates pinned images to a fixed pixel height, which has specificity (0,1,1) and beats the .h-full utility (0,1,0) six of the seven layouts use. Elegant Dark is seeded is_default, so that was the out-of-the-box result for any layout other than grid/timeline. Migrations 052/053 corrected for fresh installs; 181 repairs the rows already seeded. Whitespace-tolerant because sanitizeCSS strips newlines from any template ever saved through the editor — an exact-text migration would have silently no-opped on most real installs. The height property is matched with a lookbehind so line-height/max-height/min-height are untouched, grouped selectors are handled, and nested rules are skipped rather than mis-rewritten. Both reported, measured in the live DOM, by @BraynArts.	2026-08-22 21:38:17 +02:00
Paul Nothaft	97d92f8428	fix(thumbnails): regenerate external photos instead of dropping their tiers (#1129) ... POST /admin/thumbnails/regenerate resolved every source as storage/events/active/<photo.path> and fs.access'd it. External and reference rows are not there, so every one failed and was counted as an error — and because the tier deletion runs first, the endpoint dropped every ?w= tier and rebuilt nothing, leaving the library worse than before it ran. The UI reported success either way. Now routed through ensureThumbnail, which resolves both source kinds, uses the per-photo ext<id>_ output name, and writes thumbnail_path back itself. Review rounds also removed both destructive deletes in generateThumbnail: the pre-delete ran before sharp opened the source, so an unreadable source left the previous rendition gone and the database pointing at it — across a bulk run, the whole gallery. Neither delete was needed, since put stages to a temp file and renames atomically and is the last statement in the try. Videos are filtered out, and the superseded rendition is removed only when the storage key actually moved, compared through the same canonicalisation the backends apply so a legacy backslash path is not mistaken for a different object. Reported by @BraynArts, who also identified the fix.	2026-08-22 21:37:49 +02:00
Paul Nothaft	f735d26422	fix(gallery): a missing thumbnail tier must not take the backend down (#1128) ... The first load of a gallery whose ?w= tiers do not exist yet could exit the Node process — not 500 one tile, kill the backend. Two defects stacked. The reader: LocalFsStorage.get() returns a lazy fs.createReadStream, so an ENOENT arrives after the await returned and outside the route's try/catch. An unhandled 'error' event is a process-level throw. pipeStreamToResponse attaches the handler the routes were missing — 404 for a vanished source, connection destroyed if bytes are already on the wire, file headers cleared so the JSON error is not served as image/jpeg or cached as a broken tile for an hour. Applied to all nine streaming responses in gallery.js. The writer: ensureThumbnailAtWidth passed regenerate:true, whose first act is to DELETE the target — on a path only reached when the tier is absent. A grid fires one request per tile, so one request unlinked the file another had just published and handed to a reader. Without the flag the write is an atomic rename. Generation is now also deduped per tier key: 8 concurrent requests ran 5 Sharp passes before, 1 after. Reported with a full diagnosis by @BraynArts.	2026-08-22 21:37:15 +02:00
Paul Nothaft	e243a88410	chore(main): release 3.112.0-beta.0 (#1139) v3.112.0-beta.0	2026-08-22 16:41:36 +00:00
Luca	7223118b89	feat(deploy): make the all-in-one image installable without a shell (#1124) ... The all-in-one image could not be installed from a GUI at all — the deployment it exists for. validateEnv treats a missing JWT_SECRET as critical and exits, and the documented run command supplies it with `openssl rand`, a shell command a Synology Container Manager or QNAP Container Station form cannot run. wait-for-db.sh now generates one on first start and persists it next to the database, extending the existing /run/secrets hydration rather than adding a second mechanism. Explicit env still wins, then /run/secrets, then the generated file. The write is load-bearing: JWT_SECRET is exported only when the file actually persisted, because an unpersisted secret would mint a new one every restart and sign every session out. Creation writes to a private temp file and hard-links it into place — atomic, fails with EEXIST when another container won, and the loser adopts the winner's value. Non-regular paths are rejected before the link, since POSIX ln links INTO a directory rather than failing, which would make a mistyped -v target unrecoverable. Also repairs the onboarding paths a new install actually walks: the installer no longer rotates the secrets of a running install on re-run, deprecates the dead scripts/install.sh in place, corrects the CONTRIBUTING dev loop, and fixes the vite proxy target that had been pointing at a stray local port since 0da45e69. Reviewed over three rounds. Co-authored by @Luca-Timo.	2026-08-22 18:37:12 +02:00
Paul Nothaft	8f23118782	chore(main): release 3.111.1-beta.0 (#1138) v3.111.1-beta.0	2026-08-22 16:29:22 +00:00
Paul Nothaft	24e11df299	fix(faces): dark-mode styling for the People surfaces (#1106) (#1126) ... Every component added by #1074 was styled for light mode only. In the admin dark theme the three toggle labels rendered invisible — including "Detect people in this gallery", the switch that starts GDPR Art. 9 processing — and the Manage-people modal rendered as a light panel over a dark page because its shell was a hardcoded bg-white. Admin surfaces pair each neutral with a dark: variant; guest surfaces read the gallery theme tokens, because galleries carry their own dark themes that the admin dark class knows nothing about. Beyond the issue's inventory: the cover picker and face-in-context viewer that landed after it was filed, the magnifier chip whose bg-white/90 would have carried light glyphs, PeopleSheet's own hardcoded bg-white shell, the selected avatar's white ring-offset halo, and both dismiss buttons whose hover darkened into the background. External review found one defect, fixed: the sheet's avatars ring against --color-surface, not the page background.	2026-08-22 18:24:13 +02:00
Paul Nothaft	2a43d95ff8	chore(main): release 3.111.0-beta.0 (#1123) v3.111.0-beta.0	2026-08-21 17:41:19 +00:00
Paul Nothaft	38c27d097c	feat(faces): show a detected face in its source photo, outlined (#1120) ... Phase 2 of #1096. Stacked on the phase-1 branch — it needs the Postgres fix there, or the face list this reads comes back empty. A 64px avatar answers "is this a person", not "is this the same person as that other cluster". The reporter's revised use for this is the pre-merge decision: who they were standing next to, what the occasion was. So the face opens in its own photo with the detected box drawn, and prev/next walks that person's other appearances without leaving the modal. The box is positioned in PERCENTAGES of the original frame, not measured pixels: the container carries the photo's aspect ratio, so the same four numbers land correctly at any rendered size, with no resize listener. Verified against real data before writing the component — bbox [221.9, 174.9, 294.5, 405.8] on a 750x750 frame resolves to left 29.6% / top 23.3% / width 39.3% / height 54.1% and lands squarely on the face. Preview rendition, never thumbnail, and that is load-bearing rather than a quality preference: thumbnail_fit is seeded to 'cover' on every install, so a thumbnail has had its edges cut off and ratios taken against the ORIGINAL land nowhere on it. That was #1100, and it presented as a broken detector. Not built on AdminPhotoViewer, deliberately. It wants full AdminPhoto objects (this endpoint returns photo_id + bbox + dimensions), it carries delete and category actions that are wrong for "who is this?", and there is no seam to draw the box. Three things review caught, all real: - The container had a height cap but no width cap, so a panorama derived its width from the aspect ratio and overflowed the modal sideways, taking part of the outlined face off-screen. - The per-tile affordance was hover-only, so on a tablet it was permanently invisible and there was no way to inspect a specific tile. - The row action opened index 0, which is the TOP-SCORING face — the same thing as the cover only until someone uses phase 1 to pick a different one, at which point the row showed one face and opened another. It now resolves to the cover's own index. Round 2 found three more, all real: - The counter called a list truncated whenever it hit 500, so a person with exactly 500 faces was told their complete list was capped. It now compares against total_face_count. - facesLoading goes false with an empty array on a zero-face person or a failed request, so the panel sat on a spinner that would never resolve. - Five 32px actions plus a 64px avatar exceed a 320px row, and the name is what got pushed out. flex-wrap alone did not fix it — the toolbar still claimed its max-content width first — so its basis is capped at small sizes and the buttons wrap to a second line instead. A cover that falls outside the capped list opens the first face instead. That case implies the list IS capped, so the truncation note already explains it — real pagination is a bigger change and is not in this. Verified end to end: picked the 5th of 13 faces as cover, and the row action opened at 5 / 13 rather than 1 / 13. Frontend suite 178 passing, build clean, no new type errors.	2026-08-21 19:36:59 +02:00
Paul Nothaft	ffc6bc1530	chore(main): release 3.110.0-beta.0 (#1122) v3.110.0-beta.0	2026-08-21 17:34:11 +00:00
Paul Nothaft	bbce3cd2a2	feat(faces): let the photographer choose which photo represents a person (#1119) ... Phase 1 of #1096. Clustering picks the cover, and its idea of a good one and a human's do not always agree. A cluster whose avatar is turned away or softer than the rest stays that way in the guest-facing people strip too, and nothing in the UI could change it. A picker reachable from each person row, reusing the face list the split dialog already loads — same query, same grid, different action on a click. Making the choice actually stick took four changes --------------------------------------------------------------------------- event_people.cover_face_id has existed since migration 177 and the PATCH already accepted it, so the first version of this was frontend-only. It was also a no-op: - facePeopleService.listPeople SELECTED cover_face_id and then discarded it, recomputing the cover as the best-scoring VISIBLE face on every read. The picker saved, said so, and the avatar reverted immediately. It now prefers the stored pick whenever this audience can see it, and falls back to the score-ordered choice otherwise — so visibility scoping still wins, and a guest is never handed a crop of a photo they cannot open. - recomputeCentroid overwrote cover_face_id unconditionally. It runs on rescan and on photo replacement, so any reprocessing silently undid a deliberate choice. It now keeps the chosen face while it is still a member of the cluster. - The face list is cached per person, and split/merge move faces between people. Until now the only reader closed itself after acting, so nobody saw the stale copy; the picker is a second reader of the same key. - cover_face_id meant two things. assignFaces seeded it with whichever face opened the cluster and recomputeCentroid overwrote it with the highest scoring one, so an automatic guess was indistinguishable from a deliberate choice — and honouring it would have pinned every UNCURATED person to that guess, which is worse than the fallback it replaced (the fallback is computed per audience and skips photos a guest cannot open). Both writers are gone, migration 179 clears the stored guesses, and the column now means one thing. That also removes the need to defend the choice against rescans: nothing overwrites it, and a dangling id self-heals to the derived cover. Clearing existing values is safe rather than destructive: no install has ever been able to SET a cover, so every stored value is an automatic guess by construction. Also fixes a PostgreSQL-only 500 --------------------------------------------------------------------------- GET /admin/events/:id/people/:personId/faces joined `photos` but did not table-qualify its WHERE, and photo_faces and photos BOTH have an event_id: column reference "event_id" is ambiguous Postgres refuses it, so the endpoint 500s and the Split dialog — its only consumer until now — has been broken on every PostgreSQL install since the join was added. SQLite resolves the ambiguity silently, which is why the suite stayed green. Reproduced against a real Postgres before and after. The query is now a named builder the route calls and the test imports, rather than a copy: an earlier version of that test re-declared the query, so the route could regress to the bare form while the assertions kept passing. Merge and recluster preserve the choice as well. Both already carried labels and privacy flags across; the chosen cover is human state of the same kind, so it now rides along — through a merge when the target has none, and through a recluster by following its FACE into whichever cluster ends up holding it, rather than the majority-descendant rule the label uses. The picker and the endpoint disagree past 500 faces, so the picker now says when it is showing a capped list rather than presenting it as exhaustive. Frontend suite 178 passing, backend 23 across the touched suites, build clean, no new type errors. Mutation-checked twice: dropping the cover preference fails the new listPeople test while the visibility-scoping test still passes, and restoring the auto-seed in assignFaces fails it too.	2026-08-21 19:26:52 +02:00
Paul Nothaft	887bdbe6e5	feat(gallery): responsive grid thumbnails (#1095) (#1109) ... * feat(gallery): responsive grid thumbnails (#1095) The half of #1095 that #1099 deliberately left out. Grid tiles are ~175 CSS px at the mobile 2-column default — about 530 device px on a DPR-3 phone — so the 300px thumbnail is upscaled ~1.8x and faces visibly mush. Backend mirrors the preview tiers exactly: ?w= on the gallery thumbnail route, whitelisted to 300/600/900, cached by width in storage, never written to photos.thumbnail_path, and keyed by photo id for every source type — basenames are not unique across events and a tier is served from a cache hit without re-reading the source, which is how the preview tiers nearly leaked one gallery's photo into another. The tier is in the ETag, or a client holding the 300px file gets a 304 for its 600px request. Cleanup and regenerate invalidation are wired the same way. generateThumbnail now takes width/height overrides; it keeps the configured `fit`, because the grid renders with object-cover and tiers that were framed differently would visibly jump as the viewport changes. The srcset only advertises tiers the SOURCE can fill. Thumbnails are generated withoutEnlargement, so a 400px original asked for 900 comes back at 400 — advertising "900w" would have the browser pick that candidate and upscale it, which is the reported softness made worse. That exact trap is why this was held back from #1099; the photo's own dimensions are now the guard, measured on the SHORT edge because thumbnails are square and a 4000x600 panorama can still only fill a 600 tile. A source that clears only one tier gets no srcset at all rather than a single pointless candidate. Two things this surfaced, both worth knowing separately: `npx tsc --noEmit` type-checks NOTHING in this project — the root tsconfig is `files: []` with project references, so the real command is `tsc -b`, which is what build:check runs. Under tsc -b the repo has 43 files with pre-existing type errors; this branch adds none, and the one error in a file I touched (PeopleManagerModal:91) is on main already and unrelated to the line I changed. * fix(gallery): wire grid tiers into the component that actually renders The srcSet landed in PhotoGrid.tsx, which nothing imports — GalleryView renders PhotoGridWithLayouts, and every grid layout funnels its tile through the shared PhotoCard. The frontend half of #1095 shipped nothing. Moved to PhotoCard, and switched from srcSet to a single sized URL, the same shape PhotoLightbox already uses for preview tiers. AuthenticatedImage fetches its src with the gallery bearer token and renders the blob; an <img> carrying a w-descriptor srcSet ignores src entirely, so that fetch would have been discarded and the browser would have issued its own — unauthenticated, and resolved against the page origin rather than the configured API host. One URL keeps the auth path and halves the requests. The tier comes from the tile's measured width via the IntersectionObserver entry, read on the same render that reveals the image so nothing is fetched twice. Column counts differ per layout and shift again with thumbnailScale, so the breakpoint table is only a fallback. Also closes what the tier cache leaked or served stale: - ensureThumbnailAtWidth short-circuits videos. Their thumbnail is a poster frame, so the tier path handed the video file to Sharp — after downloading it in full on S3, uncached, once per request. - The ETag names the tier actually served, not the one requested. A fallback to the canonical thumbnail was caching a 300px image under a 900px key. - Tier height scales from the configured aspect ratio instead of forcing a square; with fit:'cover' a 300x200 canonical and a 600x600 tier are two different crops and the photo reframed between tiers. - The canonical short-circuit compares against the configured thumbnail_width, not the 300 default, so a 600px install stops generating duplicate tiers. - Tier invalidation on /admin/thumbnails/regenerate, above the local-file check that skips S3 and external rows. - Tier cleanup in replacePhoto and deleteEventCascade. Both derive keys from the photo row, so the rows have to be read before they change or vanish. Preview tiers had the same two holes and are swept alongside. The clamp no longer drops a tier when the source falls between them: a 400px short edge asked for 600 returns all 400 pixels, where clamping to 300 threw 100 of them away. Backend 18 tier tests, frontend 22. Full suites green: 293 backend across the touched areas, 185 frontend, build clean, no new type errors. * fix(gallery): measure the tile, and stop regenerating the w300 tier Follow-up to the review of #1095. Closes the three items left open there, plus a defect the previous commit introduced. **The w300 tier regenerated on every request.** Decoupling the canonical short-circuit from the hardcoded 300 left generateThumbnail still tagging against DEFAULT_THUMBNAIL_WIDTH. On an install with thumbnail_width=600 a w=300 request wrote `thumb_<name>` while the caller probed for `thumb_w300_<name>`: the cache never hit, so every request re-downloaded the original and ran Sharp, and the file it left behind was in no cleanup list. The tag now follows the configured width, and thumbnailTierKeys lists all three widths — which one is canonical is a setting, so excluding 300 stranded exactly the file a 600-configured install generates. **The tier is chosen from the tile's measured width.** The observer entry only exists for `lazy` cards, and Mosaic, Masonry and Timeline don't pass it — Mosaic is 1-up on mobile where Grid is 2-up, so they are the layouts a breakpoint guess gets most wrong. Measured in a layout effect and gated: the image is not rendered until the width is known, so AuthenticatedImage never mounts with a src it has to replace. Attaching the observer ref unconditionally instead refetches every tile, since React flushes passive effects before the sync re-render a layout effect triggers — removing the gate makes the new single-request test fail, which is how that was confirmed rather than assumed. **Gallery Premium has its own card** and never reached the shared one, so its tiles kept pulling the canonical thumbnail. MasonryPhotoAlbum already hands the laid-out width to the render prop, so it needed no measurement. **Event rename orphaned tiers.** The key embeds the basename, so the DB update is the point past which the old keys cannot be derived. Dropped inside the filename-changed branch, not the loop body: unconditional would fire four storage deletes per photo on every rename, 20k calls against S3 for a 5,000-photo event that merely had its slug adjusted. Preview tiers had the same hole and are swept alongside. Carousel is the seventh layout and deliberately gets no tiering: its filmstrip thumbs are 80 CSS px, under the canonical 300 even at DPR 3. Tests: first PhotoCard suite (6), backend tier suite 21. Both new behaviours mutation-checked — reverting the width tag, the render gate, the measurement, or the rename sweep each fails a test. Full suites green: 298 backend across the touched areas, 191 frontend, build clean, no new type or lint findings. * fix(gallery): mount masonry cards once, into a measured layout Found while capturing screenshots for this PR, by attributing every thumbnail request to a photo id rather than eyeballing the grid. Masonry columns mode starts at 3 columns and runs its greedy distribution off a hardcoded 300px estimate until the container has been measured. Cards mounted into that guess are torn down when it settles — photos move to a different parent column, so React unmounts them — and since #1095 each mount picks its tier from its own width, the two mounts request two DIFFERENT urls. Measured on a 1440px desktop, production build, 62 photos: before 45 photos fetched at canonical AND w600, 17 stuck on w600 107 requests after 62 photos, canonical only, 62 requests Mobile was already landing on one tier either way, so both mounts produced the same url and the second was a cache hit — which is why it looked clean and the desktop case did not. The fix is the gate the rows/justified mode in this same file already applies for the same reason (line 346): hold the cards back until containerWidth is known. Only columns mode was missing it. Grid and Justified take their column counts from CSS breakpoints, so they have no transient measured value to discard and are unaffected. Worth noting this was NOT visible on main: without tiering both mounts request the same url, so the browser cache absorbs the duplicate. Tiering is what turns a harmless remount into a second download — the regression is this PR's, which is why it is fixed here rather than deferred. Frontend suite 194 passed (3 new). Mutation-checked: removing the gate fails the mount-once and placeholder tests. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-08-21 19:26:21 +02:00
Paul Nothaft	00776234fd	fix(security): let cors() own Access-Control-Allow-Origin on protected images (#1118) ... Closes #1116. secureImageMiddleware set its own Access-Control-Allow-Origin, overwriting the one cors(corsOptions) had already computed. server.js:247 mounts cors() on all of /api with credentials:true, so by the time the route handler ran the correct header was already there — and the local assignment replaced it with a worse answer in BOTH directions: unresolved -> '*'. Combined with the credentials:true that cors() sets, that is an invalid pair browsers reject outright. Unreachable on Docker until #1104 stopped compose injecting FRONTEND_URL; reachable on a fresh install from then until the wizard stores general_site_url. resolved -> the frontend origin, even when the request legitimately came from the allowlisted ADMIN_URL. A split admin host got a header naming the wrong origin and the browser rejected a request cors() had allowed. Deleting the line fixes both. cors() already validates the request Origin against the allowlist, sets Vary: Origin, omits the header entirely for a disallowed or absent Origin, and pairs correctly with credentials. Methods, Headers and Max-Age stay here: they are route-specific and cors() does not contradict them. Observed against a running instance before and after: allowlisted Origin ACAO: <that origin> + Vary: Origin + credentials:true disallowed Origin no ACAO no Origin header no ACAO Six tests, mounted on a real Express app with server.js's middleware order. Deliberately NOT a unit test against a response double: the first version of this fix was a guarded assignment that looked correct in isolation and still overwrote cors() whenever an origin resolved. A double cannot see middleware composition, which is exactly how that slipped through. Mutation-checked both ways — restoring the original `|| '*'` fails 5 of 6, and restoring the guarded assignment fails 3 of 6 including the admin-origin case.	2026-08-21 19:25:56 +02:00
Paul Nothaft	d241919604	fix(ui): stop iOS Safari zooming in on 14px form fields (#1113) ... Closes #1105. iOS Safari zooms the whole page in when a focused form control computes to under 16px, and it does not zoom back out. Unlocking a gallery is a client-side transition rather than a document navigation, so the zoom the password field triggers carries straight into the gallery: the layout pans horizontally and the header actions sit off-screen until the visitor pinch-zooms out by hand. A real page load would have reset it. `.input` and `.input-themed` are `text-sm`, so the field is 14px on every phone, and GalleryPage inverts the breakpoint on top of that (`text-sm sm:text-base` — 14px below 640px, where iOS zooms, and 16px above it, where it never does). Keyed to the POINTER, not a width. The zoom depends on the computed font size and a touch device, never on how wide the viewport is — and a phone in landscape is 667-956 CSS px, above any width you could call "phone". Measured on the admin login page, which has no `sm:` override: main portrait 390x844 14px zooms main landscape 844x390 14px zooms main iPad 820x1180 14px zooms fixed all three 16px fixed desktop (mouse) 14px unchanged, no zoom off touch One media query rather than flipping each call site. `.input` alone backs 334 `<Input>` usages, but there are also ~440 raw inputs, selects and textareas carrying their own `text-sm`, and Tailwind utilities sit in a later layer than @layer components — so a fix at the component definition misses most controls and any new `text-sm` silently reintroduces the bug. The `:not()` on each selector is load-bearing, not decoration: it buys the specificity to beat a utility class. Measured in a browser — input.text-sm 16px (0,2,1 beats .text-sm) select.text-sm 14px (0,0,1 loses) textarea.text-sm 14px (0,0,1 loses) 24 selects and textareas in the tree carry `text-sm`, so the bare form would have left them zooming. Checkbox and radio stay excluded so font-size never sizes their box. max(16px, 1em, 1rem) is a FLOOR, not a size. A flat 16px would make controls that are already bigger smaller: Typography -> Large sets --font-size-base to 18px on body, so anything inheriting it would be clamped down and the setting quietly ignored. Each term covers a case the others miss: normal (body 16) 16px Large theme (body 18) 18px Small theme (body 14) 16px browser default 20px 20px The viewport meta is deliberately left alone: `maximum-scale=1` would suppress the zoom by disabling pinch-to-zoom for everyone.	2026-08-21 19:25:47 +02:00
Paul Nothaft	69edee36fc	ci: give the backend job headroom over its observed tail (#1108) ... The backend job normally finishes in about 3 minutes — the last eight runs on main were 2.6 to 3.4 — but it is the only one that boots Postgres and runs the full integration suite, so it is the only one exposed to runner contention. The observed spread has reached 9.2 minutes against a 10-minute cap, and release PR #1088 was cancelled at 10.3 with every test in the log passing and jest still running. That failure mode is expensive out of proportion to how often it happens: a cancelled job is a red X on a branch that is actually green, so it costs a diagnosis and a re-run each time, and it lands on release PRs because those are the ones that run when everything else does. The cap is a runaway guard rather than a performance budget, so 20 buys real headroom over the worst run seen while still killing a genuinely hung suite well inside the hour GitHub would otherwise allow. frontend and ml keep 10: they finish in seconds and have never been close. Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-08-21 19:25:23 +02:00
Paul Nothaft	ba221ba2a0	chore(main): release 3.109.0-beta.0 (#1115) v3.109.0-beta.0	2026-08-21 10:58:27 +00:00
Luca	9431b9f094	feat(setup): configure the public address and SMTP in the wizard, not .env (#1104) ... * feat(setup): configure the public address and SMTP in the wizard, not .env A fresh install could not configure its own public address. `general_site_url` and the `email_configs` row already existed as admin settings, but nothing could reach them: - docker-compose.yml injected FRONTEND_URL=${FRONTEND_URL:-http://localhost:3000} and Dockerfile.aio baked in ENV FRONTEND_URL=http://localhost:3000, so getFrontendBaseUrl() returned on its first branch every time and the setting was never read. .env.example shipped the same value as an uncommented placeholder for FRONTEND_URL / ADMIN_URL / API_URL. - the wizard never asked for the address at all, and skipped its whole config step unless a CRM-ish feature was selected — so a gallery-only install was also never offered SMTP, despite gallery links, guest invites and expiry warnings all going out through email_configs. - eleven call sites read process.env.FRONTEND_URL directly rather than the resolver, three of them defaulting to placeholder hosts that reached real recipients: https://app.example.com in payment-reminder emails, localhost:3005 in admin invitation emails, https://app.example.com in dev template previews. Stop injecting a default anywhere, and resolve the origin instead: FRONTEND_URL -> general_site_url -> the origin the request arrived on -> whichever exists -> ''. A loopback candidate is treated as unconfigured so the installs that already have http://localhost:3000 baked into their environment self-heal; the same guard previously lived inline in routes/gallery.js for the slideshow QR (#848) and is now shared. The empty return is preserved because shareLinkService and the SSO redirects in routes/auth rely on it to emit relative urls — callers needing an absolute url use getAbsoluteFrontendUrl(), which still ends at http://localhost:3000. The wizard now persists window.location.origin right after the admin account is created, so an install that skips the rest still has a usable origin for background jobs that have no request to derive one from, and offers it as an editable "Public address" field. Settings -> General shows the field read-only when FRONTEND_URL pins it, instead of silently ignoring edits. Also drop the `|| 'mailhog'` fallback when seeding email_configs: that host only exists in the dev compose profile (which does not even start by default), so a fresh install came up with a live config pointing nowhere while the wizard showed empty SMTP fields. With no row, blank fields are the truth and emailProcessor logs "No email configuration found". Developers set SMTP_HOST=mailhog explicitly. backend/src/services/emailService.js is deleted: nothing in backend/ references it, and it was the only consumer of the SMTP_* variables, which misrepresented how mail is configured. Refs #705 * fix(setup): keep FRONTEND_URL ahead of ADMIN_URL/APP_URL when resolving links The previous commit routed two call sites through the resolver but put the site-specific variable FIRST, silently reversing precedence: userManagementService was: FRONTEND_URL || ADMIN_URL || localhost:3005 became: ADMIN_URL || resolver adminEvents/crud was: FRONTEND_URL || APP_URL || '' became: APP_URL || resolver An install with both variables set would have flipped which one won. Call the resolver first instead — it starts with FRONTEND_URL, so the original relative order is preserved and only the final fallback changes: localhost:3005 (not even the frontend's port) and '' (a relative link inside an email) both become the resolved origin. Refs #705 * fix(setup): unpin loopback FRONTEND_URL, keep ADMIN_URL/APP_URL reachable Review feedback on #1104. isEnvPinned() reported ANY FRONTEND_URL as authoritative, including the loopback values getFrontendBaseUrl() deliberately demotes. An install upgrading with the old compose default FRONTEND_URL=http://localhost:3000 therefore resolved its origin from general_site_url correctly, but got the Site URL field rendered read-only in Settings and skipped by the wizard's seeding - locking the exact operators this change exists to unblock out of configuring a public address anywhere. The predicate now mirrors the resolver, and the derived general_site_url_effective the General tab reads comes from the same helper instead of re-normalising process.env inline. APP_URL and ADMIN_URL had become dead code: getFrontendBaseUrl() only returns falsy when NOTHING is configured, so `|| process.env.ADMIN_URL` after it never ran once a site URL existed - which after this PR is the normal case. A split-origin install pointing ADMIN_URL at a separate admin host got invite links on the public gallery origin instead. They are now passed as an explicit `override` that resolves directly below FRONTEND_URL, preserving the historic FRONTEND_URL-before-ADMIN_URL order while beating the database- and request-derived fallbacks. general_site_url now feeds the CORS allowlist and the Access-Control-Allow-Origin header, not just email links, so a schemeless value is an allowlist entry no browser origin can match. Validate it server-side in PUT /general (isURL with require_protocol, require_tld off so LAN/NAS installs on http://nas:3000 still work) and client-side in both surfaces that write it - type="url" never fires in either, since neither input sits inside a form. Two more wizard fixes: the General tab no longer reposts general_site_url while it is env-pinned, because the field then holds the effective env value rather than the stored one and the round-trip read as a change to a protected key, 403ing a settings.edit-without-settings.domains admin on an unrelated save. And SetupConfigStep validates the From address before posting - /admin/email/config rejects a blank one, which used to surface as a generic warning while the wizard advanced from its finally block anyway, discarding every SMTP value the user had typed, password included. A failed save now keeps them on the step. * fix(setup): surface a rejected public address instead of swallowing it Review round 2 follow-up on #1104, pushed onto the branch. saveSiteUrl() caught and discarded every error. That was defensible before round 2 added a server-side URL check, but PUT /general can now answer 400 — and the two validators disagreed: http://my_nas.local client: accepted server: rejected http://foo_bar:3000 client: accepted server: rejected validate() let those through, the 400 was swallowed, `failed` stayed false and onDone() ran. The operator finished the wizard believing the public address was stored when nothing had been. That is the silent misconfiguration this whole change exists to remove, landing on the LAN and NAS installs it targets. Three parts: - saveSiteUrl() throws. finish() resolves it before anything else is posted and puts the message on the address field rather than the generic "some settings could not be saved" warning. Skip for now still always leaves, by contract, but warns instead of dropping the value in silence. - allow_underscores on the server check, for the same reason require_tld is off: browsers resolve http://my_nas.local and the client accepts it, so rejecting it server-side only produced the mismatch above. Both validators now agree across the LAN/NAS, IDN, bare-IP and scheme-less cases. - LOOPBACK_BASE_RE anchors its host token. Bare prefix matching also demoted https://localhost-nas.example.com, and now that this predicate gates the whole resolver rather than just the slideshow QR, being demoted means a configured address is silently ignored. 127. stays a bare prefix on purpose: all of 127.0.0.0/8 is loopback. Resolver suite 31 passing, up from 26. Mutation-checked: restoring the unanchored regex fails the three new host-boundary cases. * fix(settings): don't lock the General tab on a site URL nobody typed Review follow-up on #1104, pushed onto the branch. general_site_url was free-text until this PR added a server-side check, so an upgraded install can hold something schemeless that predates it. The tab flagged that on load, and `disabled={!!siteUrlError}` then killed Save for EVERY General setting. An admin holding settings.edit but not settings.domains could not clear it either: correcting the address is a change to a protected key and 403s. The tab has no permission gating, so that role was simply locked out of the tab with no self-service way back. That is the same role adminSettings.js:85-95 documents the no-op round-trip allowance for. The allowance only helps if the request is made, and this blocked it in the browser first. Validation now waits until the field is actually edited, and an unchanged value is dropped from the payload rather than reposted — matching what the env-pinned case already does one line above, and for the same reason. stored value invalid, untouched Save works, key not sent edited to something unusable Save blocked edited to a usable absolute url saved Four tests, first coverage for this feature. Mutation-checked: removing the dirty gate fails the untouched-value case. --------- Co-authored-by: Paul Nothaft <53005142+the-luap@users.noreply.github.com>	2026-08-21 12:54:14 +02:00
Paul Nothaft	7583b4f6b0	chore(main): release 3.108.1-beta.0 (#1103) v3.108.1-beta.0	2026-08-20 12:22:34 +00:00
Paul Nothaft	b3a7ab27ea	fix(faces): face avatars were cropped against a cropped rendition (#1100) ... * fix(faces): face avatars were cropped against a cropped rendition Found while triaging #1096, which reported the People manager showing unusable cluster covers — a bare shoulder, the back of a head, a patch of background — and asked for more sample faces to compensate. Most of that is not a detector problem and not a UI limitation. It is a bug. faceCropStyle positions an avatar by scaling the WHOLE frame and offsetting so the face lands centre. That holds only while the rendition shown is the entire image at a uniform scale. Thumbnails are not: imageProcessor.js:93 DEFAULT_THUMBNAIL_FIT = 'inside' migration 040:6 thumbnail_fit seeded to 'cover' imageProcessor.js:229 fit: settings.fit The 'inside' constant is only a fallback for a missing settings row, and the row is seeded on every install — so thumbnails are centre-cropped essentially everywhere, and every face avatar rendered against one is silently offset on any non-square photo. The reporter read the setting as safe because of that constant, and the code comment at :87-92 says the same thing; all three places disagree with what is actually stored. It presents as a bad detector, which is why it survived: the boxes are right, the frame they are drawn against is not. All three surfaces — the admin manager and the guest-facing strip and sheet — now read a preview, which uses fit: 'inside' and is therefore the whole frame. At w=640: plenty for a 64px avatar at DPR 3, and small enough that a strip of a dozen people does not pull a dozen 1920px renditions. Face scanning already calls ensurePreviewImage for anything it scans, so a preview exists for every photo that has a face. Adds the admin preview route the manager needed; the gallery already had one. Both whitelist ?w= the same way. The first version of the call-site test passed with every surface still reading thumbnail_url, because an import alone satisfied it. It now matches inside the src={...} expression, and each of the three surfaces was individually reverted to confirm the test fails. * fix(faces): size the face tier by bbox, and keep admin_preview auth The face half of the external review; the tier-key and long-edge fixes live on the #1099 branch this is stacked on. Face avatars used one fixed 640 tier. In a 6000px group shot a 200px face is ~21px there, and faceCropStyle then blows that up ~9x to fill a 64px avatar at DPR 3 — mush, and indistinguishable from the mis-positioning bug this PR exists to fix. The tier is now derived from the bbox's share of the frame, so a face across a hall gets 1920 and a close-up still gets 640. The synthesized face URL also dropped admin_preview. verifyGalleryAccess only accepts the admin cookie when admin_preview=1 is on the request (middleware/gallery.js:28), and the preview flow deliberately mints no gallery JWT — so every avatar 401'd in exactly the mode an admin uses to check a gallery before sending it to a client. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-08-20 14:17:25 +02:00
Paul Nothaft	56034af0f0	chore(main): release 3.108.0-beta.0 (#1102) v3.108.0-beta.0	2026-08-20 12:14:09 +00:00
Paul Nothaft	011f6ae7ec	feat(gallery): sized preview tiers so phones stop pulling 1920px (#1095) (#1099) ... * feat(gallery): sized preview tiers so phones stop pulling 1920px (#1095) A phone can display ~1170px at most, but the preview tier is a single 1920px JPEG with no size parameter — so every lightbox swipe ships roughly twice the bytes it can use, and the slide track preloads neighbours, which multiplies it. On the reporter's all-external install a null preview_url falls back to the untouched NAS original, which makes it worse again. Backend: ?w= on the gallery preview route, whitelisted to 640/1280/1920. A whitelist rather than a free-form width because every distinct value is a permanent rendition on disk — an open parameter is an invitation to fill the volume. Unrecognised or absent values fall through to the canonical 1920 preview, so old clients and hand-typed URLs behave exactly as today. Extra tiers are cache, not state: ensurePreviewImageAtWidth keys them by width, looks them up in storage and generates on miss, and never writes photos.preview_path. That column owns the canonical rendition, and threading a width through it would mean the last size anyone requested silently becomes "the" preview. Requesting 1920 resolves to the existing preview rather than a w1920 duplicate, so no install grows a second copy of every preview it already has. The tier is part of the ETag. Without it a client holding the 1920 rendition gets a 304 for its 640 request and renders the wrong size, which is this feature inverted. Frontend: the lightbox picks a tier from innerWidth x devicePixelRatio, capped at DPR 3 — uncapped, a DPR-10 device asks for 3900px and lands straight back on the desktop rendition. At the top tier the URL is left byte-identical so existing caches and ETags stay valid and desktop sees no change at all. saveData and a 2g/3g effectiveType drop one tier; both are Chromium-only, so they are a bonus rather than the mechanism. Grid thumbnails are NOT tiered here, deliberately. generateThumbnail resolves its width from admin settings rather than an argument, so tiering it is a separate change — and shipping a srcset whose candidates the server ignores would be worse than shipping none: the browser would take the "600w" candidate, receive the 300px image and upscale it, which is the reported softness made slightly worse. That half of #1095 lands separately. * fix(gallery): scope tier keys per photo, size by long edge, clean up tiers External review. Three findings against the tier work, one a cross-gallery leak. The tier cache key was the photo's BASENAME. Managed uploads keep camera basenames, so two events can each hold an IMG_0001.jpg — and a tier is served straight from a cache hit without re-reading the source, so the second gallery gets the first gallery's photo. Keys are now scoped by photo id for every source type. The RAW branch passed proc.outputBasename, which would have dropped that scoping again; it now passes the scoped name. Tier selection used viewport WIDTH, but ?w= bounds the LONG edge (fit:'inside'). On a 390x844 phone at DPR 3 a 2:3 portrait is bound by height and renders ~1755 device px, so width-only picked 1280 and made portraits softer than today; landscape on the same phone needs ~1170. It now computes the rendered long edge from the photo's own dimensions and falls back to the top tier — today's behaviour — when they are unknown. Tiers live outside photos.preview_path, so nothing else knew they existed: delete, bulk-delete and archive left them orphaned in previews/ forever, and regenerate-previews refreshed only the canonical rendition while phones kept the stale copy. previewTierKeys derives them from the same deterministic scheme and all four paths clean up. Deliberately outside the preview_path guard — a tier can exist when the canonical rendition never did, so keying cleanup off preview_path would strand precisely the photos only ever viewed on a phone. The existing tier tests encoded the old width-only semantics and were updated rather than kept; that is a behaviour change, not a test fix. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-08-20 14:08:07 +02:00
Paul Nothaft	0b886ed942	fix(faces): defer on unreachable storage, and commit the import path first (#1097) ... * fix(faces): defer on unreachable storage, and commit the import path first The two follow-ups left open by #1091, both consequences of external photos becoming scannable at all. **A dropped mount no longer burns the gallery.** ensurePreviewImage returns null for "this JPEG is corrupt" and "the NFS share is gone" alike, and faceProcessor marked both 'failed'. Nothing re-queues a failure automatically and the queue only ever claims 'pending', so a mount that blinked mid-scan cost the whole event a manual Re-scan — on external libraries, where network storage drops far more often than local disk, that is the common case rather than the corner one. faceProcessor now probes the containing DIRECTORY before failing, and throws TransientSourceError when it cannot be reached; faceQueue treats that exactly like SidecarUnavailableError — release to pending, back off, retry — with the warning rate-limited to one per five minutes, since an outage hits every photo in the event. The directory rather than the file is the whole point: a missing file inside a healthy directory is a broken photo and should still fail, and it still does. Anything that goes wrong deciding which case it is falls through to 'failed', because guessing 'transient' on an unknown condition would retry forever. **The import commits its path before inserting rows.** enqueueEvent accepts processing_status NULL (faceProcessor.js:243-246), which these inserts leave unset, so an admin hitting the toggle or Re-scan during a long import could queue partial rows while the event still resolved against the old directory — and burn them to 'failed'. Moving events.external_path ahead of the loop closes that, and fixes a pre-existing bug on the same line: an import that died at photo 500 of 1000 used to leave those 500 rows pointing into the new tree while the event still resolved against the old one, making every one of them unreadable. Safe to do first because the path is already validated above, and existing photos are unaffected — photo.source_origin takes precedence over event.source_mode in both resolvers and is NOT NULL defaulting to 'managed'. The #1090 test that asserted a missing source fails needed its setup corrected rather than its intent: it created no directory at all, which is now (correctly) a dropped mount. It now creates one, so it tests the case it always meant to — healthy storage, dead photo. Verified both fixes discriminate: removing the probe fails the defer test, and moving the event update back after the loop fails the ordering test. Face suite 59 passed across 8 suites; the full backend suite fails the same 10 pre-existing suites as unmodified main, no more. * fix(faces): stop a dead mount stalling the whole queue External review, and the first finding is one my own change created. Deferring by returning the row to 'pending' was a trap: claimNextPhoto orders by id ascending and the queue defaults to a single worker, so the same unreachable row becomes the oldest pending one after every backoff and the worker never reaches a higher id. One dead mount would have stalled face scanning for the entire install — unrelated events, fresh uploads, everything. Strictly worse than the permanent 'failed' this set out to replace. The row is now left parked in 'processing' with face_started_at intact. It is not claimable, so the worker moves straight on; the janitor that already exists returns it to 'pending' past STUCK_TIMEOUT_MS, which is the retry. No new column and no new timer. The sidecar branch still releases, because a down sidecar blocks every photo anyway — there is no other work to get on with. Second: probing existence was not enough. Unmounting an NFS or SMB share usually leaves the mountpoint behind as an ordinary empty directory, so fs.access succeeded on storage that was entirely gone and the photo was failed anyway — the exact case this was written for. An empty directory where the photo should live now counts as unreachable. The trade is deliberate and documented: a directory an admin genuinely emptied is retried rather than failed, which now costs one attempt per janitor sweep and nothing else. Third: a comment in adminExternalMedia claimed source_origin isolates existing photos from the early external_path update. That is true of managed rows and false of external ones — resolveExternalPath prefixes every external row with event.external_path, so importing folder B into an event referencing folder A rebases the A rows. Pre-existing rather than introduced here (the update always did this, just later), but the comment asserted otherwise, so it now says what actually happens and names the underlying single-base-path limitation. The deferral test initially passed against the blocking version too — database state alone cannot tell the fix from the bug. It now inspects the branch directly, the way the #596 contract tests do, and fails when releaseToPending is put back or the two branches are merged. * fix(faces): back off per event, and stop clobbering concurrent scans Round two of external review. Parking a row in 'processing' fixed the head-of-line block but not the cost: every janitor sweep handed the whole dead gallery back, and the worker walked all of it again — one stat per photo against storage that may be hard-mounted and slow to time out — before reaching any healthy event. Every one of those attempts also went through generatePreviewImage first, which logs an error per photo, so a down mount produced a recurring flood that the rate-limited warning did nothing about. So the backoff is now per EVENT and separate from the janitor: TransientSourceError carries the event id, the queue records a cooldown, and claimNextPhoto excludes those events while it lasts. The janitor keeps doing its own job, which is rescuing rows a crashed worker abandoned. Cooldown is in memory on purpose — a restart is usually what follows fixing a mount, so it should retry at once. Second: committing the event path before the loop means a toggle or Re-scan firing mid-import can now genuinely queue and finish some of those rows. The final bulk update was unconditional, so it dragged 'done' rows back to 'pending' for a duplicate sidecar scan and knocked 'processing' rows out from under the worker. It is now whereNull — only rows nothing has touched are ours to queue. Also corrected a comment of mine that had gone stale in the same file: it still described the enqueue as happening after the event path was written "below", which stopped being true when that update moved above the loop. * fix(faces): judge the mount, the path and the file separately Round three of external review. The probe was too coarse in both directions. It read any ENOENT on the photo's own directory as a mount-wide outage, so a deleted or renamed subfolder — individual/ gone while collages/ is healthy — deferred the entire event and starved every sibling folder, renewing the cooldown on each retry. It now judges the EVENT ROOT for that verdict: root missing, or present-but-empty, is an outage; anything below a populated root is a broken path and fails. And it read a listable directory as proof the photo was at fault, so EACCES on a reconnected share, EIO, or the classic NFS ESTALE handle were burnt as permanent failures. Only ENOENT now means genuinely gone; any other error opening the file defers. The event-wide backoff was also too broad. A reference event can hold managed uploads alongside imported external ones, and those live in local storage that is fine — excluding the whole event id left them unscanned for as long as external rows kept renewing the cooldown, which during a real outage is indefinitely. The exclusion is now scoped to external and reference rows. One of my own tests had modelled the unmount wrongly: it emptied the photo's subdirectory rather than the event root, which under the corrected logic is a populated mount with a missing folder — a failure, not an outage. It now empties the root, which is what an unmount actually leaves behind. Dropped the path require the first version of this probe needed; the event-root form does not. All three fixes mutation-checked: reverting each one fails the test written for it. Face suite 69 passed across 9 suites; full backend suite fails the same 10 pre-existing suites as main, no more. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-08-20 14:06:21 +02:00
Paul Nothaft	b7f04f6992	chore(main): release 3.107.5-beta.0 (#1101) v3.107.5-beta.0	2026-08-20 11:18:12 +00:00
Luca	2a84efef71	Merge pull request #1098 from Luca-Timo/docs/aio-readme-existing-tag ... docs(readme): point the single-container install at a tag that exists	2026-08-20 14:13:12 +03:00
Luca	e47c103c2a	docs(readme): point the single-container install at a tag that exists ... The all-in-one quickstart tells people to pull `ghcr.io/picpeak/picpeak/aio:stable`, which has never been published, so the documented one-liner fails for anyone who copies it: docker: Error response from daemon: failed to resolve reference "ghcr.io/picpeak/picpeak/aio:stable": not found `merge-aio` does gate `:stable`/`:latest` on `refs/heads/stable` or a non-prerelease `v*` tag, same as backend/frontend — but `Dockerfile.aio` only landed on `main` in 0874a30a (#1068, 2026-08-18), and the current `stable` head (3.46.1) does not contain it. So the aio image has only ever built off `main`/beta refs, and its full tag list on GHCR and Docker Hub is `main`, `beta`, and `3.10x.y-beta.0`. backend and frontend both have `stable` and `latest`; aio is the only image that does not. Point the quickstart at `:main`, which exists today, and note when `:stable` will start working so this can flip back after the next stable promotion.	2026-08-20 11:51:36 +02:00
Paul Nothaft	731df9dc6e	chore(main): release 3.107.4-beta.0 (#1094) v3.107.4-beta.0	2026-08-20 06:39:49 +00:00
Luca	375bc5303d	Merge pull request #1093 from Luca-Timo/ci/aio-dockerhub-mirror ... ci(docker): publish the all-in-one image to Docker Hub, and give aio + ml a Hub page	2026-08-20 09:31:56 +03:00
Luca	899c9b3407	docs(docker): Hub pages for aio + ml, and the image table in the README ... picpeak/ml has an empty Hub overview and picpeak/aio has none at all, while backend and frontend carry hand-written ones — so the two newest images are the two with nothing on their registry page. Adds .github/dockerhub/{aio,ml}.md as the source of those pages and a dockerhub-descriptions job that pushes them on every main merge, so the page cannot drift from the release it describes. backend/frontend stay hand-maintained for now: capturing their current Hub text into files is a prerequisite, not a side effect of this change. README gains a registry table for all four images (both registries share digests and tags), the org-move callout lists the full set, and the feature list finally mentions People in this gallery, which shipped in #1074 without a README line.	2026-08-20 08:24:19 +02:00
Luca	2df455784c	ci(docker): mirror the all-in-one image to Docker Hub ... The aio image (#1042) shipped GHCR-only with a TODO to wire the Docker Hub mirror once the Hub repo existed. backend, frontend and the ml sidecar all publish to docker.io/picpeak/*; aio was the only image a Docker Hub user could not pull. merge-aio now follows merge-backend/merge-ml verbatim: DOCKERHUB_ENABLED computed from the repository slug (so forks stay GHCR-only), a gated Docker Hub login, docker.io/picpeak/aio added to the metadata images list, and a Docker Hub manifest inspect. Tag scheme is untouched — the same beta/main/stable/latest/semver tags land in both registries. The build summary drops the "Docker Hub mirror pending" note and lists the aio (and ml) Hub images when the mirror is active.	2026-08-20 07:49:53 +02:00
Paul Nothaft	54b68fe6e8	chore(main): release 3.107.3-beta.0 (#1092) v3.107.3-beta.0	2026-08-20 05:23:01 +00:00
Paul Nothaft	576924fa57	fix(faces): scan external/reference photos instead of skipping them (#1090) (#1091) ... * fix(faces): scan external/reference photos instead of skipping them (#1090) faceProcessor short-circuited every photo with source_origin 'external' or 'reference' straight to 'skipped', before the sidecar was ever contacted. On an external-media install that is the entire library — the reporter's gallery sat at 0/3230 with every row skipped and no error, and a rescan changed nothing. The guard was correct when written: resolvePhotoStorageKey returns null for anything outside managed storage, so ensurePreviewImage could not build a preview and there was nothing to send. #1078 removed that limitation one release earlier — ensurePreviewImage now reads externals straight off the mount via resolvePhotoFilePath and writes the preview into managed storage, so the key faceProcessor already fetches through getStorage() is readable like any other. The guard outlived its reason. Photos whose source is genuinely gone still return a null preview key and land in the existing 'failed' branch, which is the honest outcome: that is a broken photo, not an unsupported one. The blanket skip was absorbing those too. No migration or manual reset needed — enqueueEvent already re-queues rows with face_status in (NULL, 'failed', 'skipped'), so previously skipped photos get picked up on the next scan. * fix(faces): queue external imports for scanning (#1090) The other half of the same bug, found by external review — and my first counter-argument against it was wrong. Managed uploads are enqueued by photoProcessor, which writes face_status 'pending' once a photo is processed (photoProcessor.js:573, commented as "the only correct place to enqueue"). External media never goes through photoProcessor at all: adminExternalMedia inserts rows directly, leaving face_status NULL. faceQueue.claimNextPhoto only claims 'pending' (faceQueue.js:64), so an import into an already-enabled event produced nothing until someone pressed Re-scan. Lifting the skip guard alone made external photos scannable but still not scanned — which looks like a complete fix right up until you import a photo. Resolved once per import rather than per file, since it is a per-event setting and the loop can run to a thousand files, and guarded on both the global flag and the per-event toggle exactly as photoProcessor guards it, so installs without the feature still never write a face_status. A failure to read the setting logs and imports anyway — the photos are the point. No video guard: walkDir only collects jpg/jpeg/png/webp, so nothing faceProcessor would skip as video can arrive through this route. * fix(faces): enqueue imports only after the event path is written External review caught a race I introduced in the previous commit. Marking rows 'pending' as they were inserted published claimable work while events.external_path still held the old value — or none at all, on a first import, since the route only writes it after the entire thumbnail loop. The face worker polls continuously, so on any import long enough to matter (the loop is ~100-300ms per photo, and the reporter's library is 6500+) it would claim those rows, resolve them against the wrong directory and mark them permanently 'failed' — a state only an explicit Re-scan clears. That is strictly worse than the unscanned photos this set out to fix. Ids are now collected during the loop and marked pending in one pass after the event path is written, chunked at 500 because SQLite caps a statement at 999 bound parameters. The test now drives the real route instead of re-implementing its logic, and observes the mid-loop state from inside the per-photo thumbnail call — the only hook that can see the window the race lived in. Verified it discriminates: deleting the enqueue fails two tests, and moving it back onto the insert fails the ordering test specifically. * fix(faces): read the face setting after the import, not before Third external-review round. The setting was captured before a loop that runs for many minutes on a large library, so an admin who enabled detection during an import left every photo imported after that moment at NULL forever — the toggle endpoint only queues rows that already existed when it fired. Ids are now collected unconditionally and the setting is evaluated immediately before the queue update, off a freshly read event row. The guard is unchanged in substance: both the global flag and the per-event toggle, so installs without the feature still never write a face_status. Test flips the toggle from inside the mocked per-photo thumbnail call, which is the same mid-loop hook the ordering test uses. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-08-20 07:18:45 +02:00
Paul Nothaft	198df02d2a	test(ml): add an embedding-space fingerprint tool (#1084) (#1089) ... * test(ml): add an embedding-space fingerprint tool (#1084) requirements.txt is pinned exactly so rebuilds produce byte-identical embeddings, but nothing verified that. The API tests stub FacePipeline, so a decode/resize/kernel change could move every stored cluster without failing anything. This prints a hash per stage — decode, resize, cvtColor, YuNet detect, FaceNet forward pass — so the old and new image can be diffed on the same host before any base-image or dependency bump lands. Used it to answer the open question in #1084: Debian/Python 3.12 and Wolfi/Python 3.14 produce identical hashes at every stage, so a base swap would not invalidate stored clusters. The input is generated rather than a fixture, and the hashes are deliberately not compared across architectures — OpenCV and onnxruntime dispatch different SIMD kernels on x86 and aarch64, so this answers "did this change move the numbers", not "is every platform identical". * test(ml): fingerprint the production path, not a parallel one External review found the first cut was largely theatre. The documented command could not run: tools/ is in .dockerignore and the Dockerfile copies only app/, so the script is never inside the image. It has to be mounted — which is what I actually did when producing the numbers, while documenting something else. Three stages were fingerprinting the wrong thing: - The detector recorded "none" plus a return status, because synthetic input has no face to find. It would have stayed green through any change to YuNet or its kernels. Now the ONNX graph is driven directly, so all twelve output heads always produce numbers, and the reported thresholds are the service's (0.6/0.3) rather than FaceDetectorYN's 0.9 default. - The embedding used a hand-rolled tensor, bypassing everything that actually places a face in the embedding space: umeyama + warpAffine, BGR->RGB, per-image standardization, layout, and the L2 normalization the backend's cosine similarity depends on. It now calls _align and _embed directly. Private, deliberately — reimplementing the maths here would drift from pipeline.py and fingerprint a path nothing runs. - Decode exercised PNG, but the worker only ever receives the preview rendition, which imageProcessor.js writes as JPEG. Now a fixed JPEG, embedded as bytes so the input cannot depend on the encoder version being held still. Verified SOI/EOI-clean; the first attempt at this produced "Corrupt JPEG data: 22 extraneous bytes". Sensitivity checked rather than assumed: a one-pixel landmark nudge moves align_warp and embed and leaves decode and the detector heads alone, which is exactly the dependency structure expected. Debian/Python 3.12 vs Wolfi/Python 3.14 remain identical across all sixteen stages, so the #1084 parity conclusion still holds under the stronger check. * test(ml): measure the image's own pipeline, and the detector OpenCV runs Two more from external review, both of which let matching hashes mean less than they claimed. The documented bind mount put the checkout's app/ ahead of the image's /app/app, so comparing two images built from different revisions would have executed the same pipeline source twice and reported a match no matter how the images differed. /app now wins whenever it exists, so the tool measures the image under test however it is invoked, and the loaded path is printed as _app_source so that is auditable rather than assumed. The detector was fingerprinted through onnxruntime, but production runs cv2.FaceDetectorYN — OpenCV's own preprocessing, DNN engine and NMS/landmark decode, none of which ORT touches. An OpenCV upgrade could therefore move real landmarks, and with them alignment and embeddings, while every detector hash held still. It now runs the OpenCV path too, with the score threshold at the floor so synthetic input still yields candidates (594 here) instead of the empty result the production 0.6 gives on an image with no face. The ORT pass is kept alongside it to separate a model change from an OpenCV change. Parity across debian/3.12 and wolfi/3.14 still holds across all 19 stages, and a one-pixel landmark nudge still moves align_warp and embed and nothing else. * test(ml): cover the orchestration and progressive decode too Round three of external review found two more ways the hashes could match while production moved. The isolated stages never fed the detector's output into alignment — _align got fixed landmarks — so INPUT_LONG_EDGE resizing and the row -> landmark scaling in _one_face were invisible. process() now runs end to end on the fixture, with the pipeline's own detector threshold dropped so a faceless frame still yields rows to carry through (26 faces here). A first attempt at that still missed the resize: the embedded fixture is 48px, so `long_edge > INPUT_LONG_EDGE` never fired and changing 1920 to 960 moved nothing. It now runs a second pass with the threshold lowered under the fixture, which executes the same downscale and inverse landmark scaling without carrying a 1920px image in the source. Verified sensitive: moving that bound 32 -> 24 changes both the face count and the embedding. The fixture was also a baseline JPEG, while generatePreview writes progressive (imageProcessor.js:236/480/617) — a different path through libjpeg. Swapped for a progressive fixture, SOF2 confirmed present and SOF0 absent. 24 stages now. Debian/3.12 and Wolfi/3.14 remain identical across all of them. * test(ml): close three more false-negative paths in the fingerprint Round four of external review. All three let hashes match while production moved. INPUT_LONG_EDGE was used but never printed. The fixture is too small to trip the resize in either image, and the forced pass overrides the value in both, so a production change from 1920 to 960 moved no hash at all. It is now emitted alongside the other thresholds, where a reviewer sees it in the diff. The fixture was square, so a width/height swap in setInputSize or the resize produced identical dimensions and identical hashes. It is now 64x48. The forced-downscale pass hashed only an embedding, which is derived from separately scaled landmarks — a regression in the inverse scaling of row[0:4] would have shown up nowhere, because the normal pass runs at scale 1. That bbox is now hashed too; a wrong one is what breaks avatar crops and area calculations. Changing the fixture to 64x48 also broke the forced pass: at the old bound of 32 the downscaled frame is 32x24 and YuNet returns nothing, so the stage pinned nothing. The NO-DETECTIONS-STAGE-VACUOUS marker added last round caught it immediately rather than printing a reassuring hash of an empty result. Bound moved to 48, which still triggers the resize and still yields rows. 25 stages, no vacuous markers. Debian/3.12 and Wolfi/3.14 identical across all of them. * test(ml): hash every detection, not just the first Round five of external review. Both end-to-end passes hashed only candidate 0, so a change that moved candidates 1..n — or merely reordered them — matched as long as the count and the first candidate held. With the threshold at the floor those passes return 24 and 27 candidates, so that was most of the evidence being thrown away. Both now stack every returned face, in order, via a shared _hash_all. Stacking preserves order, so a reshuffle is caught too. Verified against the exact case: reversing candidates 1..n while leaving the count and candidate 0 untouched now moves process_embedding and process_bbox. Before this it moved nothing. * test(ml): hash every persisted field, and emit the model version Round six of external review, plus the adjacent gaps it implied. Two findings: MODEL_VERSION was never emitted, and _hash_all discarded score. Both matter to the backend rather than to the numbers — a model_version change makes faceClustering.js:190 refuse to compare new faces against existing people, forcing a rescan, and det_score decides via meetsQualityFloor (faceClustering.js:96-100) whether a face joins clustering at all. Either could change while every hash held still. Rather than fix only the two named, I checked what faceProcessor.js actually stores per face (:157-167) and covered all of it: bbox, score, yaw, pitch, blur, embedding. yaw/pitch/blur were heading for the same finding next round. One hash per field, so a diff says which thing moved rather than only that something did. model_version is emitted as a compatibility key alongside the thresholds, not hashed — it is a string, and its job is to be read. Verified: scaling score alone by 0.999 now moves process_score and nothing else. 33 stages, no vacuous markers, debian/3.12 and wolfi/3.14 still identical. * test(ml): split verdict from diagnostic, and stop masking the threshold Round seven of external review. The ORT detector hashes were being read as part of the compatibility verdict, but production never runs YuNet through onnxruntime. An ORT change touching a YuNet operator would have moved them while real behaviour was untouched, and the docstring said any difference means re-scan — so the tool could have ordered a full-gallery rescan for nothing. They are now diag_-prefixed, and the docstring states which keys carry a verdict, which are diagnostic, and which are metadata a reviewer has to read rather than diff. setScoreThreshold(1e-6) also overwrote the detector's real threshold before anything recorded it, and _thresholds.det_score only echoes config. If FacePipeline ever stopped applying DET_SCORE_THRESHOLD — falling back to OpenCV's 0.9 default — production would detect a different face set while every hash matched. The constructed value is now read first and emitted as _effective_det_score; simulating the regression makes it read 0.9 instead of 0.6. MAX_FACES is emitted for the same reason INPUT_LONG_EDGE is: the fixture never reaches the pipeline.py:138 slice, so 64 -> 128 would move no hash while real group photos persisted a different face set. 21 verdict keys, 12 diagnostic, no vacuous markers, debian/3.12 and wolfi/3.14 still identical across both sets. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-08-19 22:43:58 +02:00
Paul Nothaft	3c5bedc2cf	chore(main): release 3.107.2-beta.0 (#1088) v3.107.2-beta.0	2026-08-19 15:10:22 +00:00
Paul Nothaft	37a15e3d49	fix(faces): restore the :beta image tag and surface sidecar health (#1087) ... * fix(faces): restore the :beta image tag and surface sidecar health Both halves of what a user hit on discussions/1069: the People card sat at "Scanning… 0 of 227" for 30 minutes with no explanation, because the sidecar container could never have started. docker-build.yml — republish `:beta`. It used to come for free via `type=ref,event=branch` when the active development branch was literally named `beta`; the rename to `main` silently retired it. backend:beta has been frozen at 2026-06-29 (448da950) ever since while :main moved on, so PICPEAK_CHANNEL=beta has been serving a seven-week-old build across every image. The ml sidecar was added after the rename and so never had a `:beta` at all, which left docker-compose.production.yml:158 unable to resolve ghcr.io/picpeak/picpeak/ml for any documented channel — the image simply does not exist as :beta or :stable, only as :main and pinned versions. Tag added to all four merge jobs, gated on main. `:stable` stays absent for ml on purpose: it is gated on refs/heads/stable and the sidecar does not exist there. stable's docker-compose.production.yml carries no picpeak-ml service, so nothing can reference the missing tag. FaceRecognitionCard — show when the sidecar is unreachable. An unreachable sidecar is not an error by design: faceQueue.js:132-136 releases the photo back to `pending` and retries forever so a restart does not burn the queue. The cost was that a stopped container looked exactly like a slow scan, indefinitely, and the only signal was a backend log line rate-limited to once per five minutes. /admin/events/faces/health already existed and nothing in the frontend called it. It is now polled while a scan is in progress, and a failing check replaces the spinner with the sidecar URL, the underlying error (which distinguishes a stopped container from a token mismatch) and the command to start it. Health is only polled while a scan is running — an idle card has no reason to care whether the sidecar is up. * fix(faces): tell the three sidecar failure modes apart Follow-up to the health surface in this branch, from an external review pass. The original warning was right about "the sidecar is not working" and wrong about almost everything after that. faceClient.checkHealth now returns a `reason` rather than only a message, because the caller has to know whether photos survive: - 'unauthorized' (401) and 'rejected' (any other 4xx) both become SidecarRejectedError in classify(), which workerLoop does NOT retry — every claimed photo is marked 'failed'. Telling the admin the scan resumes on its own was simply untrue there; both now say to fix the cause and Re-scan. - 'unreachable' (refused/DNS/timeout/5xx) is the retryable one. The card also no longer cries wolf. /faces runs inference synchronously inside an `async def`, so one slow photo blocks the event loop and stalls /info past its 5s timeout — a healthy sidecar can fail a probe. Verified with an isolated uvicorn repro: a blocking call in an async handler stalled the sync /info endpoint to 5.01s. The warning now needs three consecutive failures AND no drop in `pending`. Three because a single /faces call may legitimately run to FACE_ML_TIMEOUT_MS (30s) and two probes 15s apart both fit inside that window; `pending` rather than `scanned` because scanned counts only 'done', so a run producing skipped/failed photos is progress that counter misses. A 4xx burns the queue with no backoff, so it can empty before anyone opens the card — in_progress goes false and only "227 failed" is left. The probe therefore also runs when a finished scan has failures, and the notice renders under the counts instead of replacing them. It is worded as present-tense service state, not as a claim about those specific failures: a live probe cannot know whether they came from this misconfiguration or from corrupt images earlier. Attributing them exactly needs stored face_error rows, which is a bigger change than this. Also adds the missing-token case to the unreachable text: FACE_ML_TOKEN has no default and the container refuses to start without it, so the most likely first run fails as a plain connection refusal that "just start it" does not fix. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-08-19 16:49:47 +02:00
Paul Nothaft	74a8f9bf24	chore(security): shrink the ML image's CVE surface, override deepmerge-ts (#1083) ... Three Trivy cleanups off the code-scanning tab. ml/Dockerfile — install no runtime apt packages at all. Neither libgl1 nor libglib2.0-0 is needed: opencv-python-headless 4.14 bundles what it needs and `ldd .../cv2/cv2*.so` resolves fully on a bare slim base. The old comment claimed the headless wheel still links libGL, which was true of much older wheels. libgl1 was dragging in 36 transitive packages (mesa, LLVM, X11) for a service that never opens a display. Measured with `trivy image` on locally built variants: before: 165 findings — 88 low / 49 med / 19 high / 6 crit without libgl1: 133 findings — 58 low / 48 med / 19 high / 5 crit without either: 123 findings — 57 low / 46 med / 13 high / 4 crit 42 findings gone, image 1.05GB -> 774MB. Not one of the 165 had an upstream fix available, so not installing the packages is the only lever there is. docker-build.yml — set ignore-unfixed on all four Trivy steps. All 123 remaining ML findings are unfixed base-OS CVEs; Debian has them resolved in sid and pending backport to trixie, and apt-get upgrade -y behind CACHEBUST picks each one up automatically. Reporting them buries anything actionable, and suppressing them is the precondition for ever setting exit-code: 1. backend — deepmerge-ts <8.0.0 has a stack-exhaustion advisory (CVE-2026-40345, high) reached via mailparser -> html-to-text, which pins ^7.1.5 so npm cannot get there alone. Not reachable in our code: html-to-text only feeds deepmerge-ts its options object (html-to-text.mjs:1468, :1442), never parsed email content. npm audit goes 3 high -> 0. The lockfile also picks up the version field release-please had left at 3.103.1-beta.0, plus some "peer": true metadata npm 11.6 recomputes. Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-08-19 13:56:01 +02:00
Paul Nothaft	9f825be01e	chore(main): release 3.107.1-beta.0 (#1081) v3.107.1-beta.0	2026-08-19 08:23:14 +00:00
Paul Nothaft	af7970b069	fix(preview): generate lightbox previews for external/reference photos (#1078) (#1079) ... * fix(preview): generate lightbox previews for external/reference photos (#1078) ensurePreviewImage() resolved its source only via resolvePhotoStorageKey(), which returns null for external/reference photos by design — those live on a media mount outside the managed storage tree. The null went straight into withLocalCopy(), which throws ("LocalFsStorage: invalid relative path: null"), so the preview route fell back to redirecting at the full-size original. A gallery whose photos are all external got no benefit from the preview tier (#492) at all: guests paid 5-12 MB on every lightbox open, with nothing surfaced in the admin UI. Add the external branch ensureThumbnail() has had since #423: resolve via resolvePhotoFilePath() and feed the mount path to generatePreviewImage() directly, with an ext<id>_ output basename so two events referencing the same NAS filename can't clobber each other's preview. Also close the adjacent hole that made the failure a throw rather than the documented null: a row with no source_origin in a reference-mode event takes its mode from the event, so resolvePhotoStorageKey returns null for it too. Return null instead of handing that to withLocalCopy. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * fix(preview): select the columns the external branch needs on bulk regenerate POST /api/admin/thumbnails/regenerate-previews selected only id, event_id, path, media_type, mime_type and preview_path, so photo.source_origin was undefined by the time ensurePreviewImage branched on it. Every external row in a reference gallery took the managed path, resolvePhotoStorageKey returned null for it, and the endpoint reported success while generating nothing. Add source_origin, external_relpath and filename to the select, plus a source-inspection test pinning the caller contract and a service-level test showing a column-starved row is indistinguishable from a managed one. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * style(test): single-quote the source-inspection needles Matches the repo eslint quotes rule (no avoidEscape) by dropping the nested quotes from the search strings rather than escaping them. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>	2026-08-19 10:17:44 +02:00
Paul Nothaft	98fe9c7699	chore(main): release 3.107.0-beta.0 (#1077) v3.107.0-beta.0	2026-08-18 20:41:12 +00:00