25 Commits

Include renames

Author	SHA1	Message	Date
Luca	8db8527f9e	feat(api): Lightroom round-trip — read proofing marks, put finished edits back (#745) (#1165) ... * feat(api): Lightroom round-trip — read marks, put edits back (#745) Gets a client's proofing verdict into a desktop catalogue and a finished edit back over its proof, without anyone re-matching files by hand. Three parts: **Keep the camera filename.** photos.original_filename is the only carrier of `IMG_1234.JPG` — the stored filename is rewritten by generatePhotoFilename. But replacePhoto() overwrites original_filename with whatever name the new file arrives under, so the first re-upload of a renamed render destroys the key the NEXT round-trip needs. Migration 185 adds photos.source_filename, written once at ingest and never touched by a replace, backfilled from original_filename so existing galleries can still match on their first pass. The backfill sits outside the column guard and keys on whereNull, so a run that dies partway self-heals instead of leaving half the rows empty forever. **Read the marks.** GET /api/v1/events/:id/photos returns each photo with its client colour tallies, the caller's own marks, and a merged colour + rating. Guards copied from the sibling upload route (apiTokenAuth + read scope + photos.view + requireEventOwnership). Filters: marked_only, mark_source, color_labels, my_color_labels, min_rating, my_min_rating. The route filters to a page of ids with PhotoFilterBuilder, then enriches just those through photoExportService.getPhotosWithFeedback — the two halves already existed and neither does both, and going id-first keeps the per-colour tally query bounded by page size. services/markMerge.js decides how three possible opinions (guest colours, guest star average, the photographer's own row in photo_admin_marks) collapse into the one colour and one rating Lightroom has room for. Colour goes to the photographer on a tie — one deliberate choice beats an aggregate a tie-break already had to guess at. Rating takes the max, because a rating is a magnitude and losing the higher one quietly demotes a photo somebody rated highly. Its roundRating matches XmpGenerator.mapRating exactly so the API and an XMP sidecar can never disagree about how many stars a photo has. **Put the edit back.** POST /api/v1/events/:id/photos accepts an optional replaces_photo_id and routes to the existing replacePhoto(), preserving the photo's id, feedback, comments and position. The plugin stores the picpeak id on the catalogue photo, so the id survives the editor renaming the render — which makes it the reliable key, not the filename. Scoped to the event in the URL: a token inherits its owner's powers across every event they can see, so an unscoped id would let one gallery overwrite another's photo. For renders whose RAW never went through the plugin, findReplacementCandidate gains an opt-in number_token mode matching on the trailing digit run. Deliberately the LONGEST run and never a fixed last-N slice: multi-camera shoots disambiguate by prefixing the camera index into the number (cam11234.jpg / cam21234.jpg), and a last-4 slice reads 1234 from both bodies and reintroduces exactly the collision the prefix removes. Ambiguity is refused, never guessed. Also drops the multer temp file on the two new early returns — this route only unlinks in its catch block. * refactor(api): one rating-rounding rule, and apply match_mode where it counts Three things the pre-review pass turned up on the round-trip work: - `match_mode` reached the photo-cap pre-count but not the loop that actually picks the replacement target, so asking for `number_token` would have been counted and then quietly ignored. Both call sites now take it. - `number_token` matching read `select('*')` over every photo in the event to compare one digit run. It now reads the three columns the match needs and re-reads the single winner in full, so a 5000-photo event doesn't pull 5000 full rows through memory to answer one question. - `XmpGenerator.mapRating` and `markMerge.roundRating` were the same five thresholds written twice — the second way to do one thing that drifts the moment either is touched. The thresholds now live in markMerge and the generator delegates, which is what keeps a sidecar and the v1 API from ever disagreeing about a photo's star count. * fix(api): keep the new route in the generated OpenAPI spec The `color_labels` description carried an inline JSON example. In an unquoted YAML scalar `{ "green": 2 }` parses as a flow mapping, so swagger-jsdoc threw YAMLSemanticError and dropped the WHOLE route from the spec — visible only as a warning on boot, with the route still serving normally, which is exactly the kind of failure that survives to release. Found by booting a real instance rather than by reading the diff. * fix(api): close the four blockers from review on #1165 1. Replacing an external photo silently kept serving the old file. resolvePhotoStorageKey gives photo.source_origin precedence and returns null for 'reference'/'external', so the edit was uploaded, the row updated and 200 returned while every viewer kept getting the untouched NAS original and the upload sat orphaned. replacePhoto now repoints the row to managed and clears external_relpath. The file on the share is never touched — this moves the pointer, not the data. 2. Every replacement leaked its temp file. putFromFile COPIES on local and uploads on S3; neither consumes the source, and replacePhoto never unlinked it — while the v1 route had disabled its own cleanup on the belief that replacePhoto moved the file. Cleanup now lives in replacePhoto, which closes the admin path too (adminPhotos only unlinks in its new-files branch, so replaced files leaked there as well). The v1 route also unlinks on the FAILURE path, which returned before any cleanup ran. 3. The download-all ZIP is invalidated after a replacement, as adminPhotos.js already does. Without it guests kept downloading the pre-edit photo indefinitely, which defeats the point of the feature. 4. The round-trip could not see reference or watcher galleries at all. fileWatcher and adminExternalMedia never set original_filename — the camera name lives in `filename` for those rows — so the backfill and the GET fallback both produced NULL for exactly the galleries most likely to be driven from Lightroom. The backfill now COALESCEs, both ingest paths set source_filename, and the GET falls back to filename. Concerns: - number_token no longer reads every photo row in the event per file. A LIKE on the digit run narrows the candidate set in SQL first; the exact trailing-run check still decides, so semantics are unchanged. The token is a regex-extracted digit run, so it cannot carry a wildcard. - The replacement's activity entry is scoped to event.id instead of null. The dashboard feed excludes NULL-event rows for scoped callers (GHSA-jhcf), so it was vanishing from the audit trail of the photographer who owns the event. Nit: dropped the unused higherPriorityColor export from markMerge. Three regression tests cover the external repoint, the temp cleanup and the COALESCE backfill. 21/21 pass. * chore(migrations): renumber 185 -> 193 after gallery-folders landed 185_add_category_is_folder.js merged to main while this was in review, so the number the PR reserved is taken and main is now at 192. Knex keys on filename rather than the prefix, so both would have run — but picpeakImportService guards restores with migrationOrder(), which parses that prefix, and two files answering 185 make the forward-only check pass a backup onto a schema missing its columns. Renumbered with every reference: the header comment, the test that requires the path, and the four call-site comments that cite it. The 'migration 182' reference inside it is the colour-labels migration and is unrelated; gallery.js:1134 cites upstream's 185 and is untouched. * fix(api): keep external_relpath when a replacement converts the row The external-photo blocker fix cleared external_relpath along with flipping source_origin, which closed one hole and opened another. adminExternalMedia dedupes a re-scan on (event_id, external_relpath) — routes/adminExternalMedia.js:195 — and migration 186 puts a unique index on exactly that pair. With the column nulled, the next scan of the share would not recognise the NAS original as already imported and would insert it again, so the gallery would end up holding both the edit and a fresh copy of the file it replaced. Only source_origin needs to change: it is what resolvePhotoStorageKey keys on, and every other consumer of external_relpath reads the two together and lets source_origin decide. The stale relpath on a managed row is inert for resolution and still correct as a dedupe key. Test updated to assert the value is kept rather than cleared. * fix(uploads): say when exiftool is missing instead of blaming the RAW A server without exiftool reported `No usable embedded preview in RAW file X.CR3: spawn exiftool ENOENT` for every RAW upload. The headline describes a corrupt photo; the actual cause is a package that was never installed, demoted to a trailing detail. It sends people hunting through their camera files. Hit while testing the Lightroom round-trip (#745): an export of RAW originals failed 11 times with that message, and the file was fine. RAW upload is the only feature that needs exiftool, so an install can be missing it indefinitely and only find out when someone uploads a CR3 — which makes the wording the whole diagnosis. ENOENT now produces a message naming the dependency and the install command for Debian/Alpine/macOS, and breaks out of the tag loop instead of spawning the same missing binary twice more to report the last failure as if it described the photo. A genuinely preview-less RAW still gets the original message. Verified both paths by making exiftool unreachable via PATH rather than mocking: missing tool and unreadable file now report differently. * fix(external): a delivered edit must win a relpath-fold collision Follow-up to keeping external_relpath on a replaced photo. Keeping it is what lets adminExternalMedia still dedupe the folder re-scan, but it also leaves the row inside externalRelpathFold's sweep — and that sweep does not merely rewrite paths, it DELETES collision losers via externalPhotoDedupe. The survivor was whichever row happened to be claimed first, which is iteration order. So a replaced photo — source_origin 'managed', holding the edit the photographer just delivered — could be deleted in favour of the untouched camera original sitting next to it on the share. Managed rows now claim first and therefore survive. The external row that loses is the recoverable one: it is still on the share and a re-scan re-imports it. The edit is not recoverable. Note this is deliberately NOT the "skip managed rows in the fold" shape suggested in review. Skipping would leave those rows holding a base-relative path while every other row moved to root-relative, so the scanner — which computes root-relative — would stop matching them and import the camera original again as a duplicate. That is the exact bug keeping external_relpath exists to prevent, reintroduced through a different door. Rebasing them and protecting them from deletion keeps both properties.	2026-08-28 15:45:56 +02:00
Paul NothaftandPaul Nothaft	edef4d7365	fix(images): backfill orientation for libraries that predate the fix (#1199) ... * fix(images): backfill orientation for libraries that predate the fix (#1198) #1194 corrected the generators and every ingest path, but did nothing for photos already in the database. Those rows end up worse than untouched ones: before the fix a rotated photo was CONSISTENTLY wrong — a sideways image in a tile shaped to match — and afterwards the regenerated thumbnail is correct while photos.width/height still describe the raw sensor order, so masonry and justified size a portrait photo with a landscape ratio. The dimension repair cannot reach them: it only selects rows with a NULL dimension, and an affected row has both, just transposed. Its own job rather than a mode of that one. They look alike but are not the same operation: the repair FILLS missing values and touches nothing else, while this RECOMPUTES and invalidates the derived data generated against the old orientation. Sharing a lease would also mean one blocks the other. A first attempt at this was reverted from #1194 after review found five problems. All five are addressed here: - Originals are read through resolvePhotoStorageKey + withLocalCopy + withProcessableImage, so the job works on S3 installs and on RAW/DNG. The dimension repair's direct fs read does neither, which stops being an edge case in a job that walks the whole library. - The canonical preview is cleared BEFORE faces are requeued. ensurePreviewImage returns a cached preview whenever it is still a valid image, and a pre-fix unrotated one is perfectly valid — so requeueing alone made the rescan read unrotated pixels and scale those boxes by the corrected dimensions, which is worse than leaving the data alone. - Invalidation keys off the EXIF transform, not a dimension delta. Orientations 2, 3 and 4 move every pixel while leaving width and height unchanged, as does 5-8 on a square image; a delta check skips exactly those rows. - Archived events are excluded — archiving deletes the originals and keeps the rows, so every one of them would fail its read. - The dimension write and the invalidation share a transaction. Split, a failure between them leaves stale face data that no retry can fix, because the retry computes "already correct". Tier deletion stays outside the transaction on purpose: it touches storage, and a failed object delete must not roll back a correct database write. A leftover tier regenerates on next read; a rolled-back write is silent corruption. * fix(images): invalidate every stale rendition, fence the writes, and give the job a button (#1198) Three things from review, one of which mattered a lot. The invalidation was too narrow. Clearing only preview_path fixed the face data and left the gallery worse off: ensureThumbnail and ensureHeroImage return their cached file whenever it is merely VALID, and a pre-fix sideways thumbnail is perfectly valid — so a corrected row rendered the old sideways image inside a newly-corrected portrait tile. All three canonical renditions are cleared now, their stored objects deleted, and both responsive tier sets with them. The responsive tiers also needed handling rather than a hopeful catch. Their helpers swallow delete errors, and ensurePreviewImageAtWidth treats storage.stat(key) as a cache hit — so a tier that survived deletion keeps serving unrotated forever and never regenerates. The keys are re-checked after deletion and survivors are counted into the result, so a run that could not clear them does not report itself as clean. Writes are fenced on the identity that was measured, not just the id. replacePhoto swaps a new file under an existing row and rewrites path/filename, and it IS reachable — from the replace_by_name upload path in adminPhotos.js. A replacement landing while this job read the old original would otherwise have had the previous file's dimensions written over it and its fresh renditions cleared. And the job had no way to start it: the endpoint existed with no caller, so an upgrade would have left every affected library untouched unless an operator found the API themselves. It gets a Status card like its two neighbours, with strings in en/de/fr/sl. No backlog counter, because unlike the other two it cannot know how many rows need it without doing the work. * fix(images): make the backfill idempotent, and stop it lying about what it did (#1198) Six things from review round 2. The job was not idempotent, and the way it failed was expensive. Its trigger is the EXIF tag on the ORIGINAL, which correcting a photo never changes — so every re-run threw away the renditions it had just regenerated and requeued every completed face scan. On a face-enabled install, running it twice meant re-detecting the whole library for nothing. Migration 191 adds photos.orientation_checked_at, written in the same transaction as the work it records, with `force` as the escape hatch for an interrupted run. The candidate query selected preview_path but not thumbnail_path or hero_path, which the deletion loop reads — so those two pointers were cleared in the database while the objects stayed in storage, still reachable through previously issued URLs. watermark_path was missed entirely. gallery.js serves it ahead of the original when branding watermarking is on, which makes it the most visible rendition of the lot. (Its generator needed rotating too — that went into #1185, where the other three live.) storage.stat() RESOLVES with null for a missing key rather than rejecting, so counting "the promise settled" marked every deleted — and every never-created — tier as a survivor. A perfectly clean run told the operator to re-run. Now a null means gone, and a rejection counts as stuck, since a storage error is not proof the object went away. Face data is invalidated whenever the stored dimensions change, not only when the change came from rotation: boxes are scaled by photo.width at read time, so any dimension change strands them. And `corrected` now comes from the affected-row count. If the fence rejected the write because the file was replaced mid-run, the photo was not corrected and the run must not claim it was. * fix(images): stop the backfill doing unnecessary work, and make its retry advice true (#1198) Round 3, four points, all narrower than the last two rounds. It re-processed photos that were already correct. A 5-8 rotation changes the dimensions, so a tagged photo whose stored dimensions are ALREADY oriented must have been ingested after #1185 — its renditions are fine and clearing them deletes valid files and rescans a completed face detection for nothing. Those are now skipped and simply marked. Orientations 2, 3 and 4 (and 5-8 on a square image) leave the dimensions identical either way, so they carry no such evidence and are still done once. The retry advice was impossible to follow. When a responsive tier could not be deleted the row was still marked, so the ordinary re-run the UI recommends found nothing and the stale tier kept serving unrotated forever. The marker is withheld when a tier survives, which is what makes that message honest. Storage cleanup now only runs when a fenced write actually landed. If the file was replaced mid-run every update matched zero rows, but the deletion went ahead anyway and could destroy renditions belonging to the REPLACEMENT — watermarks especially, which are keyed by photo id and alias straight onto the new file. And the full-photo ETag includes the backfill's timestamp. It was built from the ORIGINAL's mtime plus the watermark settings hash, neither of which this job touches — so a guest holding a pre-fix ETag would go on getting 304 and their cached sideways image no matter how many times the backfill succeeded. --------- Co-authored-by: Paul Nothaft <[email protected]>	2026-08-26 21:01:51 +02:00
Paul NothaftandPaul Nothaft	c18f54ede0	fix(images): respect EXIF orientation in thumbnails, heroes and previews (#1194) ... * fix(images): respect EXIF orientation in thumbnails, heroes and previews (#1185) generateThumbnail, generateHeroImage and generatePreviewImage went straight from sharp(imagePath) to .resize(), so a photo whose Orientation tag is not 1 — routine for portrait shots on bodies that tag rather than rotate the sensor data — was resized from the raw frame and came out sideways. The same pipelines then call .withMetadata(false), stripping the tag from the output, so nothing downstream could correct it either. The download path already had this right: resizeToBox calls probe.rotate() for stills, which is why the same photo looked correct on download and rotated in the gallery. All three generators now do the same, guarded to stills for the reason resizeToBox already documents — .rotate() flattens a multi-frame source. The reporter also spotted the half that compounds it: photos.width/height were stored from sharp's metadata, which reports pixels as STORED, not as displayed. For orientation 5-8 those are swapped, so a portrait photo landed in the database as landscape and masonry/justified sized its tile with the wrong aspect ratio on top of the image being unrotated. A shared orientedDimensions() helper now does that conversion at all four capture sites — managed upload, background processing, external import and the dimension repair — so the stored numbers describe the rotated result the generators now produce. Existing rows keep their pre-rotation dimensions until the photo is reprocessed; the images themselves correct on the next thumbnail/preview regeneration. Tests fail on the unfixed generators — verified by reverting the rotate calls and the swap, which fails 4 of the 7. * fix(images): orient dimensions on every ingest path, and stop guarding rotate where it protects nothing (#1185) Review found the first cut covered four of eight dimension-capture sites. The filesystem watcher, the S3 auto-importer, the v1 upload API and replace-by-name all still persisted raw metadata.width/height, so an orientation 5-8 photo arriving that way got a correctly rotated thumbnail and a database row describing it as landscape — the same aspect-ratio mismatch this PR set out to remove, just on the paths I had not grepped. (I searched for `metadata.width` and the v1 route aliases it to `meta`.) The animated guard was also wrong in two of the three generators. generateThumbnail and generateHeroImage never pass `animated: true`, so they already flatten a multi-frame source to its first frame — skipping .rotate() there protected an animation that was being discarded anyway, while leaving the output in raw orientation against swapped stored dimensions. Both now rotate unconditionally. generatePreviewImage keeps the guard, because it genuinely does open animated sources as animated and .rotate() would flatten them. That leaves one corner unsolved rather than papered over: a multi-frame source that also carries an orientation tag keeps its raw orientation in the preview while the thumbnail and stored dimensions describe the rotated one. GIF has no EXIF and animated WebP effectively never sets it, so it is a real gap but not a common one, and closing it means rotating frame by frame rather than quietly dropping the animation. Documented at the guard. * fix(images): add a recompute mode so existing libraries get corrected too (#1185) The orientation fix only helped new photos. A row affected by the bug has BOTH dimensions stored — just in the raw order — so the repair job's NULL filter could never reach exactly the rows that needed it. Worse, once their thumbnails regenerated rotated, those rows went from consistently-wrong (sideways image in a matching tile) to inconsistent: correct image, wrong-shaped tile. `recompute` widens the candidate set to every image row. Opt-in, because it re-reads every original. It also has to deal with the consequence for faces. Detection runs against the preview and stores boxes in ORIGINAL pixel space, scaled by `photo.width / previewMeta.width` (faceProcessor.js:220-224) — so a photo whose stored dimensions change has face data recorded against a coordinate system that no longer exists, and the overlays crop the wrong region. Photos whose dimensions actually change are requeued for scanning; ones that were already correct are not, or a routine repair would rescan the whole library. Rows with face_status NULL are left alone so installs that never enabled the feature don't start scanning because of a dimension repair. Writing the test for that last rule caught a real bug in it: the candidate query never selected photos.width/height, so `photo.width` was undefined and every row compared as changed. Both columns are selected now. * Revert "fix(images): add a recompute mode so existing libraries get corrected too (#1185)" This reverts commit cb771d08. Review round 3 found five problems, all of them in this addition rather than in the orientation fix itself, and one of them an own-goal: requeueing face scanning makes processPhotoFaces call ensurePreviewImage, which returns the CACHED pre-fix preview when it is still a valid image — so the rescan reads unrotated pixels and scales those boxes by the newly corrected dimensions. That is worse than leaving the data alone. The rest need work this PR should not be carrying: the dimension repair reads originals through resolvePhotoFilePath and plain sharp, so it does nothing on an S3 install and rejects RAW/DNG; recompute pulls archived rows whose originals were deleted on archive; orientation 2, 3 and 4 change the pixels without changing width or height, so a dimension-delta test never notices them; and the dimension write and the face invalidation are not atomic, so a failure between them leaves a row that no retry will ever requeue. Split out so it can be designed and reviewed on its own. The orientation fix — .rotate() in the three generators and orientedDimensions() at all eight ingest sites — is unaffected and stays. * fix(images): the watermarked rendition needs orienting too (#1185) A fourth generator with the same bug, found while reviewing the backfill that builds on this. watermarkService composites and re-encodes through its own sharp pipeline with no .rotate(), and gallery.js serves photos.watermark_path ahead of the original when branding watermarking is on — so on a watermarked gallery the sideways image is precisely what a guest sees. Two details this needed beyond the .rotate() itself: metadata() is read from a separate, unrotated handle. .rotate() does not change what metadata() reports — a 400x200 source tagged orientation 6 still reads 400x200 — and every use of those numbers here is positioning: watermark scale, font size, composite extent. They have to be the DISPLAYED dimensions or the mark is placed against the wrong axis, so they go through orientedDimensions. The composite offsets are floored. getPositionCoordinates derives from the SVG's estimated text extent and returns fractional pixels; sharp rejects a non-integer offset and applyWatermark catches its own error and returns the image unwatermarked. Landing on a whole pixel was luck, and changing the dimensions it is computed from ran out of it — the test surfaced a real "Expected integer for left but received 92.8". --------- Co-authored-by: Paul Nothaft <[email protected]>	2026-08-26 20:55:54 +02:00
Paul NothaftandPaul Nothaft	1366d6d14c	fix(previews): preserve alpha and animation in the preview tier (#1171) ... * fix(gallery): stop the lightbox loading originals to display a photo (#1166) The lightbox read `preview_url`, which the server only emits once an admin has flipped lightbox_preview_enabled — off by default. So a stock install fell straight through to `url`, the untouched original: a reporter measured 16.5 MB for a photo whose preview is 345 KB. The lightbox renders its neighbours too, so opening one photo pulled three originals. `slideshow_url` is the same /preview/:id URL, watermark query included, and has been emitted unconditionally for images since #1015 — the slideshow never had a fallback worth taking. Preferring it fixes every existing install with no migration and no admin action, and `url` still backstops videos, where both derivative URLs are null. Verified on the local rig with the toggle off, so the photos API returns preview_url: null exactly as filed. Opening one photo: before GET /photo/82, /photo/81, /photo/21 (3 originals) after GET /preview/82?w=1280, /preview/81, /preview/21 397 KB -> 23 KB per image on that gallery's test photos. The toggle no longer decides whether the lightbox uses previews, so its copy said something untrue; it now describes what it still does, which is pre-generate rather than wait for the first guest to open a photo. Updated in en/de/fr/sl, the locales that carry those keys. * fix(gallery): cover the layouts the lightbox fix missed (#1166) External review found the fix was incomplete, and the review of it found one more. Premium galleries were untouched. PhotoGridWithLayouts returns early for gallery-premium, which builds its own yet-another-react-lightbox slides with `src: photo.url` — so those galleries kept pulling full originals and the reported bandwidth problem remained. They now use lightboxImageUrl for the display source; `download` deliberately stays on photo.url, because what a guest saves must be the original. The Story layout was worse, and neither the issue nor the review caught it: StoryPhotoCard rendered the full original as its GRID TILE, at object-cover in a small card. That is the one place where "hundreds of megabytes for a gallery" was literally true. It now uses the per-device thumbnail tier like PhotoCard, and its PhotoSwipe source uses the preview tier. Animated GIFs keep the original. generatePreviewImage always encodes JPEG, so routing an animated source through the preview tier would have replaced the animation with its first frame — a regression the toggle-off default never had. Animated WebP has the same problem and cannot be distinguished by MIME alone; that needs the backend to report it (Sharp's `metadata.pages > 1`) and is left rather than costing every static-WebP gallery the bandwidth fix. The settings copy claimed too much. "Pre-generate lightbox previews" does not generate anything on save — it unlocks the regenerate button and keeps preview_url emitted. Reworded to say that, in en/de/fr/sl. Not changed: the review's P1 said this bypassed the secure-image route on enhanced/maximum galleries. It does not. AuthenticatedImage collects requiresToken and secureUrlTemplate into an explicitly-voided unusedProps and never substitutes {{token}}, so on those protection levels photo.url was a literal `.../secure/82/{{token}}` that returns 400 — the lightbox was falling back to the 300px thumbnail, not to a protected image. Verified against a live maximum-protection gallery. Codex withdrew the finding on that evidence. * fix(gallery): keep premium downloads working and story framing intact (#1166) Second review round, three findings — two of them regressions this PR introduced. Premium Download became a no-op. handleDownloadFromLightbox recovered the photo with `filteredPhotos.find(p => p.url === slide.src)`, and slide.src is a derivative now, so the lookup found nothing and the button silently did nothing. The slide carries the photo id and the handler resolves by that; what Download hands over is still the original. Story cards were reframed. thumbnail_fit is seeded to 'cover' on every install, so thumbnails are square centre-crops — and story cards are not square (400x500 in the carousel, fixed-height in the desktop grid), so the card's own object-cover cropped them a second time and every photo shifted. They now use the preview tier, which is fit:'inside' and therefore the whole frame: the card looks exactly as it did before, without pulling an original. APNG joins the animated-format guard. It declares image/apng and the preview route would serve a static frame. Animated WebP still cannot be detected from MIME and remains the documented gap. * fix(gallery): keep PNG on the original, alpha and all (#1166) Third review round. generatePreviewImage encodes JPEG, which drops ALPHA as well as animation — a transparent PNG came back flattened against a solid background. And an APNG is normally reported as image/png, so the image/apng check alone missed the common upload path. PNG now stays on the original: it is where transparency is the norm, and rare enough in an event gallery that the bandwidth given up is small. Animated or alpha WebP still cannot be detected from MIME and remains the documented gap; it needs the backend to report Sharp's `pages`/`hasAlpha`. Two further findings are acknowledged and deferred rather than fixed here: - Story cards now request /preview on mount, so a cold gallery generates its previews in one burst. That is a new CPU cost, not a regression — those cards previously fetched full ORIGINALS on mount, which is strictly worse. Doing it properly means viewport-gating AuthenticatedImage, which is a change to a component every gallery surface uses and belongs in its own PR. - The premium layout memoizes slide URLs, so rotating the device before opening the lightbox can leave a photo on the tier chosen for the old geometry. The result is a slightly undersized image, and the fix is a resize subscription this PR does not otherwise need. * fix(gallery): load Story images on approach, and give the hero its own tier (#1166) Every card in a Story gallery mounts at page load — `whileInView` gates the animation, not the render — and AuthenticatedImage fetches from an effect on mount, so all of them requested at once. That was tolerable while they pointed at photo.url, because nothing was generated; pointing them at the preview tier meant a gallery with cold previews would Sharp-decode every original in one burst. The image now waits until the card is within 200px of the viewport, using framer-motion's useInView — the same observer the entrance animation already relies on — with `once` so a card never unloads on scroll-away. Verified on a 62-photo Story gallery: 3 images fetched at load, growing to 15 as you scroll, where all 62 would have fired before. While confirming that, the hero turned out to be doing the same thing the cards were. StoryHero rendered photo.url as a full-bleed object-cover background — a full original on the critical path for first paint of every Story gallery — when hero_url exists for exactly this and is a 1920x1080 cover crop emitted unconditionally for every photo (gallery.js:1139). That gallery now issues no /photo/ request at all: hero_url for the hero, the preview tier for the cards, and only as they come into range. * fix(previews): preserve alpha and animation in the preview tier Follow-up to #1166, which had to bypass the preview tier for GIF, APNG and PNG to avoid a visible regression. This removes the cause. generatePreviewImage encoded JPEG unconditionally. JPEG has no alpha channel and no second frame, so a transparent PNG came back flattened onto a solid background and an animated GIF came back as its first frame — for every consumer of this tier, not just the lightbox: the slideshow (#1015), admin previews, and the face avatars that read it as a whole-frame rendition. It was only invisible by default because the lightbox served originals. Sources with alpha, or more than one page, are now encoded as WebP, which carries both and is still far smaller than the original. Ordinary photos stay JPEG — the common path pays nothing. Two things had to move with it: - The output extension now matches what was written. A PNG source previously produced `preview_foo.png` holding JPEG bytes; harmless while the route hard-coded image/jpeg, wrong once the encoding varies. Existing keys keep working — they are still JPEG and still served as such. - The preview route derives Content-Type from the key. With `nosniff` set, mislabelling would show a broken image rather than being silently corrected. The watermark branch re-encodes to JPEG, so it labels itself explicitly; preserving animation through the watermark compositor is a separate problem. The frontend guess-by-MIME goes away entirely — including the case it could never get right, since a still and an animated WebP declare the same type. Verified on the local rig: a transparent PNG round-trips as `Content-Type: image/webp`, `hasAlpha: true`, 8.3 KB; an ordinary photo still serves `image/jpeg` from a `.jpg` key. * fix(previews): retire the legacy preview keys, and stop mislabelling watermarked ones External review of the stable twin found two defects, both on this branch too. Legacy keys collide with the new naming. The old generator kept the SOURCE basename verbatim while always writing JPEG, so a `.webp` upload produced `previews/preview_shot.webp` holding a JPEG. My PR body claimed "pre-existing keys have no .webp suffix and are JPEG" — that was simply wrong. The route now derives Content-Type from the key and the response carries nosniff, so every photo uploaded as WebP would have rendered as a broken image in the lightbox. Legacy `.png` keys are wrong the other way: flattened JPEGs of what may have been transparent sources, which isPreviewValid would have let stand forever. Migration 188 clears photos.preview_path outright — all of it, not just the suspicious extensions, because a `.jpg` key can equally be a flattened rendition and nothing in the key says so. Previews regenerate lazily on next view under the new encoder, so the cost is one regeneration per photo actually viewed. Storage is untouched, as elsewhere. The watermark branch mislabelled its output. applyWatermark PRESERVES the source format (watermarkService.js:200-211: png stays png, webp stays webp), and its input is the preview — so the output already matches the key the header was derived from. Forcing image/jpeg mislabelled every watermarked WebP preview, and nosniff means the browser would not correct it. The override is gone; the animation loss through the compositor is documented where it happens. * fix(gallery): make the Story hero fix actually work on external galleries (#1166) External review of the stable twin, both applying here too. hero_url was inert for external media. ensureHeroImage only ever called resolvePhotoStorageKey, which returns null for external/reference photos by design — and that null was handed straight to withLocalCopy, which throws, so the hero route caught it and redirected to the full ORIGINAL. #1078 fixed exactly this shape for ensurePreviewImage and nobody carried it across. It stayed invisible until this PR pointed the Story hero at hero_url: on a managed gallery that is a real saving, on a reference-mode gallery it quietly changed nothing. ensureHeroImage now has the same external branch ensurePreviewImage does — direct fs read, per-photo output basename — and returns null instead of throwing for a reference-mode row with no source_origin. The format bypass trusted mime_type, which is not trustworthy here. Migration 039 backfilled every pre-existing photo to image/jpeg regardless of what it was, and adminExternalMedia inserts rows with no mime_type at all — so a mislabelled PNG sailed past the guard and came back flattened. It now checks the filename extension as well. --------- Co-authored-by: Paul Nothaft <[email protected]>	2026-08-26 08:53:05 +02:00
Paul NothaftandPaul Nothaft	77953c15c1	fix(gallery): stop the lightbox loading originals to display a photo (#1166) (#1169) ... * fix(gallery): stop the lightbox loading originals to display a photo (#1166) The lightbox read `preview_url`, which the server only emits once an admin has flipped lightbox_preview_enabled — off by default. So a stock install fell straight through to `url`, the untouched original: a reporter measured 16.5 MB for a photo whose preview is 345 KB. The lightbox renders its neighbours too, so opening one photo pulled three originals. `slideshow_url` is the same /preview/:id URL, watermark query included, and has been emitted unconditionally for images since #1015 — the slideshow never had a fallback worth taking. Preferring it fixes every existing install with no migration and no admin action, and `url` still backstops videos, where both derivative URLs are null. Verified on the local rig with the toggle off, so the photos API returns preview_url: null exactly as filed. Opening one photo: before GET /photo/82, /photo/81, /photo/21 (3 originals) after GET /preview/82?w=1280, /preview/81, /preview/21 397 KB -> 23 KB per image on that gallery's test photos. The toggle no longer decides whether the lightbox uses previews, so its copy said something untrue; it now describes what it still does, which is pre-generate rather than wait for the first guest to open a photo. Updated in en/de/fr/sl, the locales that carry those keys. * fix(gallery): cover the layouts the lightbox fix missed (#1166) External review found the fix was incomplete, and the review of it found one more. Premium galleries were untouched. PhotoGridWithLayouts returns early for gallery-premium, which builds its own yet-another-react-lightbox slides with `src: photo.url` — so those galleries kept pulling full originals and the reported bandwidth problem remained. They now use lightboxImageUrl for the display source; `download` deliberately stays on photo.url, because what a guest saves must be the original. The Story layout was worse, and neither the issue nor the review caught it: StoryPhotoCard rendered the full original as its GRID TILE, at object-cover in a small card. That is the one place where "hundreds of megabytes for a gallery" was literally true. It now uses the per-device thumbnail tier like PhotoCard, and its PhotoSwipe source uses the preview tier. Animated GIFs keep the original. generatePreviewImage always encodes JPEG, so routing an animated source through the preview tier would have replaced the animation with its first frame — a regression the toggle-off default never had. Animated WebP has the same problem and cannot be distinguished by MIME alone; that needs the backend to report it (Sharp's `metadata.pages > 1`) and is left rather than costing every static-WebP gallery the bandwidth fix. The settings copy claimed too much. "Pre-generate lightbox previews" does not generate anything on save — it unlocks the regenerate button and keeps preview_url emitted. Reworded to say that, in en/de/fr/sl. Not changed: the review's P1 said this bypassed the secure-image route on enhanced/maximum galleries. It does not. AuthenticatedImage collects requiresToken and secureUrlTemplate into an explicitly-voided unusedProps and never substitutes {{token}}, so on those protection levels photo.url was a literal `.../secure/82/{{token}}` that returns 400 — the lightbox was falling back to the 300px thumbnail, not to a protected image. Verified against a live maximum-protection gallery. Codex withdrew the finding on that evidence. * fix(gallery): keep premium downloads working and story framing intact (#1166) Second review round, three findings — two of them regressions this PR introduced. Premium Download became a no-op. handleDownloadFromLightbox recovered the photo with `filteredPhotos.find(p => p.url === slide.src)`, and slide.src is a derivative now, so the lookup found nothing and the button silently did nothing. The slide carries the photo id and the handler resolves by that; what Download hands over is still the original. Story cards were reframed. thumbnail_fit is seeded to 'cover' on every install, so thumbnails are square centre-crops — and story cards are not square (400x500 in the carousel, fixed-height in the desktop grid), so the card's own object-cover cropped them a second time and every photo shifted. They now use the preview tier, which is fit:'inside' and therefore the whole frame: the card looks exactly as it did before, without pulling an original. APNG joins the animated-format guard. It declares image/apng and the preview route would serve a static frame. Animated WebP still cannot be detected from MIME and remains the documented gap. * fix(gallery): keep PNG on the original, alpha and all (#1166) Third review round. generatePreviewImage encodes JPEG, which drops ALPHA as well as animation — a transparent PNG came back flattened against a solid background. And an APNG is normally reported as image/png, so the image/apng check alone missed the common upload path. PNG now stays on the original: it is where transparency is the norm, and rare enough in an event gallery that the bandwidth given up is small. Animated or alpha WebP still cannot be detected from MIME and remains the documented gap; it needs the backend to report Sharp's `pages`/`hasAlpha`. Two further findings are acknowledged and deferred rather than fixed here: - Story cards now request /preview on mount, so a cold gallery generates its previews in one burst. That is a new CPU cost, not a regression — those cards previously fetched full ORIGINALS on mount, which is strictly worse. Doing it properly means viewport-gating AuthenticatedImage, which is a change to a component every gallery surface uses and belongs in its own PR. - The premium layout memoizes slide URLs, so rotating the device before opening the lightbox can leave a photo on the tier chosen for the old geometry. The result is a slightly undersized image, and the fix is a resize subscription this PR does not otherwise need. * fix(gallery): load Story images on approach, and give the hero its own tier (#1166) Every card in a Story gallery mounts at page load — `whileInView` gates the animation, not the render — and AuthenticatedImage fetches from an effect on mount, so all of them requested at once. That was tolerable while they pointed at photo.url, because nothing was generated; pointing them at the preview tier meant a gallery with cold previews would Sharp-decode every original in one burst. The image now waits until the card is within 200px of the viewport, using framer-motion's useInView — the same observer the entrance animation already relies on — with `once` so a card never unloads on scroll-away. Verified on a 62-photo Story gallery: 3 images fetched at load, growing to 15 as you scroll, where all 62 would have fired before. While confirming that, the hero turned out to be doing the same thing the cards were. StoryHero rendered photo.url as a full-bleed object-cover background — a full original on the critical path for first paint of every Story gallery — when hero_url exists for exactly this and is a 1920x1080 cover crop emitted unconditionally for every photo (gallery.js:1139). That gallery now issues no /photo/ request at all: hero_url for the hero, the preview tier for the cards, and only as they come into range. * fix(gallery): make the Story hero fix actually work on external galleries (#1166) External review of the stable twin, both applying here too. hero_url was inert for external media. ensureHeroImage only ever called resolvePhotoStorageKey, which returns null for external/reference photos by design — and that null was handed straight to withLocalCopy, which throws, so the hero route caught it and redirected to the full ORIGINAL. #1078 fixed exactly this shape for ensurePreviewImage and nobody carried it across. It stayed invisible until this PR pointed the Story hero at hero_url: on a managed gallery that is a real saving, on a reference-mode gallery it quietly changed nothing. ensureHeroImage now has the same external branch ensurePreviewImage does — direct fs read, per-photo output basename — and returns null instead of throwing for a reference-mode row with no source_origin. The format bypass trusted mime_type, which is not trustworthy here. Migration 039 backfilled every pre-existing photo to image/jpeg regardless of what it was, and adminExternalMedia inserts rows with no mime_type at all — so a mislabelled PNG sailed past the guard and came back flattened. It now checks the filename extension as well. * test(gallery): the hero fixture follows the root-relative relpath contract (#1166) external_relpath has been resolved from EXTERNAL_MEDIA_ROOT rather than from event.external_path since #1163 landed. This fixture still carried the base-relative form — its own comment noted the change was 'a separate stack' — so the two tests stopped resolving and ensureHeroImage returned null the moment that stack merged. The production path was never affected. --------- Co-authored-by: Paul Nothaft <[email protected]>	2026-08-26 08:46:29 +02:00
Paul Nothaft	97d92f8428	fix(thumbnails): regenerate external photos instead of dropping their tiers (#1129) ... POST /admin/thumbnails/regenerate resolved every source as storage/events/active/<photo.path> and fs.access'd it. External and reference rows are not there, so every one failed and was counted as an error — and because the tier deletion runs first, the endpoint dropped every ?w= tier and rebuilt nothing, leaving the library worse than before it ran. The UI reported success either way. Now routed through ensureThumbnail, which resolves both source kinds, uses the per-photo ext<id>_ output name, and writes thumbnail_path back itself. Review rounds also removed both destructive deletes in generateThumbnail: the pre-delete ran before sharp opened the source, so an unreadable source left the previous rendition gone and the database pointing at it — across a bulk run, the whole gallery. Neither delete was needed, since put stages to a temp file and renames atomically and is the last statement in the try. Videos are filtered out, and the superseded rendition is removed only when the storage key actually moved, compared through the same canonicalisation the backends apply so a legacy backslash path is not mistaken for a different object. Reported by @BraynArts, who also identified the fix.	2026-08-22 21:37:49 +02:00
Paul Nothaft	f735d26422	fix(gallery): a missing thumbnail tier must not take the backend down (#1128) ... The first load of a gallery whose ?w= tiers do not exist yet could exit the Node process — not 500 one tile, kill the backend. Two defects stacked. The reader: LocalFsStorage.get() returns a lazy fs.createReadStream, so an ENOENT arrives after the await returned and outside the route's try/catch. An unhandled 'error' event is a process-level throw. pipeStreamToResponse attaches the handler the routes were missing — 404 for a vanished source, connection destroyed if bytes are already on the wire, file headers cleared so the JSON error is not served as image/jpeg or cached as a broken tile for an hour. Applied to all nine streaming responses in gallery.js. The writer: ensureThumbnailAtWidth passed regenerate:true, whose first act is to DELETE the target — on a path only reached when the tier is absent. A grid fires one request per tile, so one request unlinked the file another had just published and handed to a reader. Without the flag the write is an atomic rename. Generation is now also deduped per tier key: 8 concurrent requests ran 5 Sharp passes before, 1 after. Reported with a full diagnosis by @BraynArts.	2026-08-22 21:37:15 +02:00
Paul NothaftandPaul Nothaft	887bdbe6e5	feat(gallery): responsive grid thumbnails (#1095) (#1109) ... * feat(gallery): responsive grid thumbnails (#1095) The half of #1095 that #1099 deliberately left out. Grid tiles are ~175 CSS px at the mobile 2-column default — about 530 device px on a DPR-3 phone — so the 300px thumbnail is upscaled ~1.8x and faces visibly mush. Backend mirrors the preview tiers exactly: ?w= on the gallery thumbnail route, whitelisted to 300/600/900, cached by width in storage, never written to photos.thumbnail_path, and keyed by photo id for every source type — basenames are not unique across events and a tier is served from a cache hit without re-reading the source, which is how the preview tiers nearly leaked one gallery's photo into another. The tier is in the ETag, or a client holding the 300px file gets a 304 for its 600px request. Cleanup and regenerate invalidation are wired the same way. generateThumbnail now takes width/height overrides; it keeps the configured `fit`, because the grid renders with object-cover and tiers that were framed differently would visibly jump as the viewport changes. The srcset only advertises tiers the SOURCE can fill. Thumbnails are generated withoutEnlargement, so a 400px original asked for 900 comes back at 400 — advertising "900w" would have the browser pick that candidate and upscale it, which is the reported softness made worse. That exact trap is why this was held back from #1099; the photo's own dimensions are now the guard, measured on the SHORT edge because thumbnails are square and a 4000x600 panorama can still only fill a 600 tile. A source that clears only one tier gets no srcset at all rather than a single pointless candidate. Two things this surfaced, both worth knowing separately: `npx tsc --noEmit` type-checks NOTHING in this project — the root tsconfig is `files: []` with project references, so the real command is `tsc -b`, which is what build:check runs. Under tsc -b the repo has 43 files with pre-existing type errors; this branch adds none, and the one error in a file I touched (PeopleManagerModal:91) is on main already and unrelated to the line I changed. * fix(gallery): wire grid tiers into the component that actually renders The srcSet landed in PhotoGrid.tsx, which nothing imports — GalleryView renders PhotoGridWithLayouts, and every grid layout funnels its tile through the shared PhotoCard. The frontend half of #1095 shipped nothing. Moved to PhotoCard, and switched from srcSet to a single sized URL, the same shape PhotoLightbox already uses for preview tiers. AuthenticatedImage fetches its src with the gallery bearer token and renders the blob; an <img> carrying a w-descriptor srcSet ignores src entirely, so that fetch would have been discarded and the browser would have issued its own — unauthenticated, and resolved against the page origin rather than the configured API host. One URL keeps the auth path and halves the requests. The tier comes from the tile's measured width via the IntersectionObserver entry, read on the same render that reveals the image so nothing is fetched twice. Column counts differ per layout and shift again with thumbnailScale, so the breakpoint table is only a fallback. Also closes what the tier cache leaked or served stale: - ensureThumbnailAtWidth short-circuits videos. Their thumbnail is a poster frame, so the tier path handed the video file to Sharp — after downloading it in full on S3, uncached, once per request. - The ETag names the tier actually served, not the one requested. A fallback to the canonical thumbnail was caching a 300px image under a 900px key. - Tier height scales from the configured aspect ratio instead of forcing a square; with fit:'cover' a 300x200 canonical and a 600x600 tier are two different crops and the photo reframed between tiers. - The canonical short-circuit compares against the configured thumbnail_width, not the 300 default, so a 600px install stops generating duplicate tiers. - Tier invalidation on /admin/thumbnails/regenerate, above the local-file check that skips S3 and external rows. - Tier cleanup in replacePhoto and deleteEventCascade. Both derive keys from the photo row, so the rows have to be read before they change or vanish. Preview tiers had the same two holes and are swept alongside. The clamp no longer drops a tier when the source falls between them: a 400px short edge asked for 600 returns all 400 pixels, where clamping to 300 threw 100 of them away. Backend 18 tier tests, frontend 22. Full suites green: 293 backend across the touched areas, 185 frontend, build clean, no new type errors. * fix(gallery): measure the tile, and stop regenerating the w300 tier Follow-up to the review of #1095. Closes the three items left open there, plus a defect the previous commit introduced. **The w300 tier regenerated on every request.** Decoupling the canonical short-circuit from the hardcoded 300 left generateThumbnail still tagging against DEFAULT_THUMBNAIL_WIDTH. On an install with thumbnail_width=600 a w=300 request wrote `thumb_<name>` while the caller probed for `thumb_w300_<name>`: the cache never hit, so every request re-downloaded the original and ran Sharp, and the file it left behind was in no cleanup list. The tag now follows the configured width, and thumbnailTierKeys lists all three widths — which one is canonical is a setting, so excluding 300 stranded exactly the file a 600-configured install generates. **The tier is chosen from the tile's measured width.** The observer entry only exists for `lazy` cards, and Mosaic, Masonry and Timeline don't pass it — Mosaic is 1-up on mobile where Grid is 2-up, so they are the layouts a breakpoint guess gets most wrong. Measured in a layout effect and gated: the image is not rendered until the width is known, so AuthenticatedImage never mounts with a src it has to replace. Attaching the observer ref unconditionally instead refetches every tile, since React flushes passive effects before the sync re-render a layout effect triggers — removing the gate makes the new single-request test fail, which is how that was confirmed rather than assumed. **Gallery Premium has its own card** and never reached the shared one, so its tiles kept pulling the canonical thumbnail. MasonryPhotoAlbum already hands the laid-out width to the render prop, so it needed no measurement. **Event rename orphaned tiers.** The key embeds the basename, so the DB update is the point past which the old keys cannot be derived. Dropped inside the filename-changed branch, not the loop body: unconditional would fire four storage deletes per photo on every rename, 20k calls against S3 for a 5,000-photo event that merely had its slug adjusted. Preview tiers had the same hole and are swept alongside. Carousel is the seventh layout and deliberately gets no tiering: its filmstrip thumbs are 80 CSS px, under the canonical 300 even at DPR 3. Tests: first PhotoCard suite (6), backend tier suite 21. Both new behaviours mutation-checked — reverting the width tag, the render gate, the measurement, or the rename sweep each fails a test. Full suites green: 298 backend across the touched areas, 191 frontend, build clean, no new type or lint findings. * fix(gallery): mount masonry cards once, into a measured layout Found while capturing screenshots for this PR, by attributing every thumbnail request to a photo id rather than eyeballing the grid. Masonry columns mode starts at 3 columns and runs its greedy distribution off a hardcoded 300px estimate until the container has been measured. Cards mounted into that guess are torn down when it settles — photos move to a different parent column, so React unmounts them — and since #1095 each mount picks its tier from its own width, the two mounts request two DIFFERENT urls. Measured on a 1440px desktop, production build, 62 photos: before 45 photos fetched at canonical AND w600, 17 stuck on w600 107 requests after 62 photos, canonical only, 62 requests Mobile was already landing on one tier either way, so both mounts produced the same url and the second was a cache hit — which is why it looked clean and the desktop case did not. The fix is the gate the rows/justified mode in this same file already applies for the same reason (line 346): hold the cards back until containerWidth is known. Only columns mode was missing it. Grid and Justified take their column counts from CSS breakpoints, so they have no transient measured value to discard and are unaffected. Worth noting this was NOT visible on main: without tiering both mounts request the same url, so the browser cache absorbs the duplicate. Tiering is what turns a harmless remount into a second download — the regression is this PR's, which is why it is fixed here rather than deferred. Frontend suite 194 passed (3 new). Mutation-checked: removing the gate fails the mount-once and placeholder tests. --------- Co-authored-by: Paul Nothaft <[email protected]>	2026-08-21 19:26:21 +02:00
Paul NothaftandPaul Nothaft	011f6ae7ec	feat(gallery): sized preview tiers so phones stop pulling 1920px (#1095) (#1099) ... * feat(gallery): sized preview tiers so phones stop pulling 1920px (#1095) A phone can display ~1170px at most, but the preview tier is a single 1920px JPEG with no size parameter — so every lightbox swipe ships roughly twice the bytes it can use, and the slide track preloads neighbours, which multiplies it. On the reporter's all-external install a null preview_url falls back to the untouched NAS original, which makes it worse again. Backend: ?w= on the gallery preview route, whitelisted to 640/1280/1920. A whitelist rather than a free-form width because every distinct value is a permanent rendition on disk — an open parameter is an invitation to fill the volume. Unrecognised or absent values fall through to the canonical 1920 preview, so old clients and hand-typed URLs behave exactly as today. Extra tiers are cache, not state: ensurePreviewImageAtWidth keys them by width, looks them up in storage and generates on miss, and never writes photos.preview_path. That column owns the canonical rendition, and threading a width through it would mean the last size anyone requested silently becomes "the" preview. Requesting 1920 resolves to the existing preview rather than a w1920 duplicate, so no install grows a second copy of every preview it already has. The tier is part of the ETag. Without it a client holding the 1920 rendition gets a 304 for its 640 request and renders the wrong size, which is this feature inverted. Frontend: the lightbox picks a tier from innerWidth x devicePixelRatio, capped at DPR 3 — uncapped, a DPR-10 device asks for 3900px and lands straight back on the desktop rendition. At the top tier the URL is left byte-identical so existing caches and ETags stay valid and desktop sees no change at all. saveData and a 2g/3g effectiveType drop one tier; both are Chromium-only, so they are a bonus rather than the mechanism. Grid thumbnails are NOT tiered here, deliberately. generateThumbnail resolves its width from admin settings rather than an argument, so tiering it is a separate change — and shipping a srcset whose candidates the server ignores would be worse than shipping none: the browser would take the "600w" candidate, receive the 300px image and upscale it, which is the reported softness made slightly worse. That half of #1095 lands separately. * fix(gallery): scope tier keys per photo, size by long edge, clean up tiers External review. Three findings against the tier work, one a cross-gallery leak. The tier cache key was the photo's BASENAME. Managed uploads keep camera basenames, so two events can each hold an IMG_0001.jpg — and a tier is served straight from a cache hit without re-reading the source, so the second gallery gets the first gallery's photo. Keys are now scoped by photo id for every source type. The RAW branch passed proc.outputBasename, which would have dropped that scoping again; it now passes the scoped name. Tier selection used viewport WIDTH, but ?w= bounds the LONG edge (fit:'inside'). On a 390x844 phone at DPR 3 a 2:3 portrait is bound by height and renders ~1755 device px, so width-only picked 1280 and made portraits softer than today; landscape on the same phone needs ~1170. It now computes the rendered long edge from the photo's own dimensions and falls back to the top tier — today's behaviour — when they are unknown. Tiers live outside photos.preview_path, so nothing else knew they existed: delete, bulk-delete and archive left them orphaned in previews/ forever, and regenerate-previews refreshed only the canonical rendition while phones kept the stale copy. previewTierKeys derives them from the same deterministic scheme and all four paths clean up. Deliberately outside the preview_path guard — a tier can exist when the canonical rendition never did, so keying cleanup off preview_path would strand precisely the photos only ever viewed on a phone. The existing tier tests encoded the old width-only semantics and were updated rather than kept; that is a behaviour change, not a test fix. --------- Co-authored-by: Paul Nothaft <[email protected]>	2026-08-20 14:08:07 +02:00
Paul NothaftandPaul Nothaft	af7970b069	fix(preview): generate lightbox previews for external/reference photos (#1078) (#1079) ... * fix(preview): generate lightbox previews for external/reference photos (#1078) ensurePreviewImage() resolved its source only via resolvePhotoStorageKey(), which returns null for external/reference photos by design — those live on a media mount outside the managed storage tree. The null went straight into withLocalCopy(), which throws ("LocalFsStorage: invalid relative path: null"), so the preview route fell back to redirecting at the full-size original. A gallery whose photos are all external got no benefit from the preview tier (#492) at all: guests paid 5-12 MB on every lightbox open, with nothing surfaced in the admin UI. Add the external branch ensureThumbnail() has had since #423: resolve via resolvePhotoFilePath() and feed the mount path to generatePreviewImage() directly, with an ext<id>_ output basename so two events referencing the same NAS filename can't clobber each other's preview. Also close the adjacent hole that made the failure a throw rather than the documented null: a row with no source_origin in a reference-mode event takes its mode from the event, so resolvePhotoStorageKey returns null for it too. Return null instead of handing that to withLocalCopy. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * fix(preview): select the columns the external branch needs on bulk regenerate POST /api/admin/thumbnails/regenerate-previews selected only id, event_id, path, media_type, mime_type and preview_path, so photo.source_origin was undefined by the time ensurePreviewImage branched on it. Every external row in a reference gallery took the managed path, resolvePhotoStorageKey returned null for it, and the endpoint reported success while generating nothing. Add source_origin, external_relpath and filename to the select, plus a source-inspection test pinning the caller contract and a service-level test showing a column-starved row is indistinguishable from a managed one. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc * style(test): single-quote the source-inspection needles Matches the repo eslint quotes rule (no avoidEscape) by dropping the nested quotes from the search strings rather than escaping them. Claude-Session: https://claude.ai/code/session_01Ra4hcsYiKuQLbbRsg6EjAc --------- Co-authored-by: Paul Nothaft <[email protected]>	2026-08-19 10:17:44 +02:00
Paul Nothaft	8e3573788b	feat(downloads): per-gallery download resolutions (#858) (#1022) ... Clients who need smaller files no longer make the photographer re-export. Two capabilities, both off by default. STANDARD RESOLUTION — the size a gallery hands out for every ordinary download (single, selected, download-all). Global default in Settings, overridable per gallery with the NULL=inherit tri-state. The pre-built download-all zip is built AT the standard resolution, so changing it invalidates those archives, including a fan-out to inheriting galleries. RESOLUTION PICKER — opt-in modal letting guests choose a different size. Custom archives are built as a DB-backed job the client polls, never cached. The picker never offers a size above the standard, and Original reappears only when the admin explicitly allows it. Resize is fit:'inside' + withoutEnlargement — aspect preserved, never upscaled — applied before the watermark, since the mark is sized relative to its input. Three rounds of external review hardened this: job archives are bound to the requester's visibility scope and re-validated at delivery, the streamed download-all path applies the cap, queue admission is bounded, and rejected resolutions no longer inflate download stats. Closes #858.	2026-08-11 09:46:46 +02:00
Paul Nothaft	38b8d476d1	fix(security): bump backend deps to close all 14 open Trivy code-scanning alerts (#869) ... * fix(security): bump backend deps to close all open Trivy code-scanning alerts - axios 1.16.0 -> 1.18.1 (GHSA-gcfj-64vw-6mp9 high + 10 medium advisories) - sharp 0.34.3 -> 0.35.3 (GHSA-f88m-g3jw-g9cj, inherited libvips CVEs) - mailparser 3.9.9 -> 3.9.14 (pulls linkify-it 5.0.2, CVE-2026-59887) - brace-expansion override >=5.0.6 -> >=5.0.7 (CVE-2026-13149) - body-parser 1.20.4 -> 1.20.6 via lockfile refresh (CVE-2026-12590) * fix(images): migrate removed sharp failOnError option and enforce Node >=20.9 sharp 0.35 drops the deprecated failOnError constructor option, so recoverably corrupt images would start failing upload validation and thumbnail generation; use the failOn: 'none' equivalent instead. sharp 0.35 also requires Node >=20.9: declare it in engines and make picpeak-setup.sh compare the full version instead of only the major, so native installs on Node 20.3-20.8 upgrade instead of breaking. * fix(setup): align the Node floor with the whole dependency tree and gate native updates html-to-text@10 needs Node >=20.19 and the glob/minimatch family excludes Node 21, so declare engines as ^20.19.0 || >=22 and enforce the same range in picpeak-setup.sh. Also run install_nodejs at the start of update_native_installation so existing native installs on an old Node get upgraded before the service is stopped, instead of restarting broken. * fix(setup): make the update-path Node gate actually work --update dispatches before detect_os, so install_nodejs saw an empty PACKAGE_MANAGER, matched no install branch, and reported success on the old runtime. Detect the OS on demand and re-verify the installed version afterwards, failing loudly (before the service is stopped) when the runtime still misses the engines range, e.g. a Node 21 that package managers refuse to downgrade.	2026-07-26 20:38:05 +02:00
Paul Nothaft	be2ec0a4a1	feat(uploads): DNG / camera-RAW support via embedded-preview extraction (#821) ... Sharp's bundled libvips has no raw loader, so a DNG can't be thumbnailed directly. This adds a preview-extraction step so RAW/DNG uploads get a proper thumbnail + gallery preview while the original RAW is kept for download. - imageProcessor: isRawFilename() + extractRawPreview() (exiftool extracts the embedded full-res JPEG — JpgFromRaw → PreviewImage → ThumbnailImage, validated with sharp) + withProcessableImage() which is a pass-through for ordinary images and swaps in the extracted JPEG for RAW. Wired into ingest (photoProcessor) and all three on-demand generators (ensureThumbnail/Hero/ Preview). generateHeroImage/generatePreviewImage gained outputBasename so RAW-derived outputs stay named after the source. - Dockerfile: add exiftool (confirmed present in Alpine v3.24 community). - Format maps: dng → image/x-adobe-dng in uploadSettings.js and fileTypes.ts; ALLOWED_MEDIA_TYPES gains a DNG entry (TIFF magic numbers) so it passes the security file-validator. Strictly gated by extension: nothing in this path runs for jpg/png/webp/etc, so existing photos are unaffected. If extraction fails (corrupt RAW, no embedded preview), the photo is marked 'failed' with a clear error — same as any unreadable upload. Verification boundary (please validate on a real DNG after the image rebuilds): the exiftool extraction itself couldn't be exercised in the dev sandbox (exiftool isn't a dev dependency and there's no DNG fixture). Unit tests cover the gating (RAW detection + non-RAW pass-through + clean failure without exiftool); existing processPhoto tests still pass. Known limitation: a DNG is only accepted when the browser reports its MIME as image/x-adobe-dng (Chrome does); browsers that send an empty type reject it client- and server-side — a follow-up can add extension-based acceptance for the RAW set. Companion to the HEIC/dynamic-hint PR; targets main only.	2026-07-17 21:51:21 +02:00
Paul Nothaft	61f1d13210	feat(lightbox): medium-resolution preview tier (#492) ... Adds an opt-in lightbox preview tier so guests open photos against an aspect-preserved ~1920px JPEG (~200–500 KB) instead of the full original (often 5–12 MB). Originals are still served on Download. Backend: - imageProcessor: generatePreviewImage / isPreviewValid / ensurePreviewImage using fit:'inside' + withoutEnlargement (longEdge 1920, q85, mozjpeg) - migration 104: photos.preview_path + lightbox_preview_enabled setting (off by default, JSON-stringified for SQLite/Postgres parity) - GET /api/gallery/:slug/preview/:photoId — gallery-auth, lazy generation, ETag based on mtime+photoId+watermarkHash - preview_url surfaced in the photo response only when the toggle is on - admin /thumbnails/regenerate-previews mirrors regenerate-thumbnails, skipping videos - backup walk + archive cleanup + photo-delete now include previews/ Frontend: - PhotoLightbox uses photo.preview_url ?? photo.url (null-safe fallback) - ThumbnailsTab gets a Lightbox Preview Tier card: opt-in toggle + Regenerate All Previews button (gated until the toggle is on) - en/de locale strings; nl/pt/ru/fr fall back to en Tested end-to-end: 11 MB / 4000×3000 source → 985 KB / 1920×1440 preview, 381 ms first call, 7 ms cached, ~91% byte reduction.	2026-05-14 22:30:39 +02:00
Paul Nothaft	936a277eb8	fix(import): capture photo dimensions in fileWatcher + s3AutoImporter (#447) ... The aspect-aware gallery layouts (masonry / mosaic / justified) read photo.width and photo.height to size each card to the source's real proportions. Two import paths were inserting rows without those fields, which forced MasonryGalleryLayout to fall back to a hard-coded 800×600 default — every card came out the same shape, so users reported masonry as "always cropped to 1:1ish" no matter which thumbnail fit mode they chose. - fileWatcher.js: extract dims with sharp.metadata() before insert. - s3AutoImporter.js: same, materialising a tmp local copy via withLocalCopy so it works in S3 mode. - migration 090: backfill any pre-existing rows with NULL dims (skips videos, skips S3 deployments — those need the writer fix alone since migrations cannot reach the storage backend). - imageProcessor.js: change DEFAULT_THUMBNAIL_FIT from 'cover' to 'inside' (only kicks in when the seed setting is missing — existing installs keep their saved value). Add UI tooltip recommending 'inside' for masonry/mosaic/justified, 'cover' for uniform grids. i18n covers all six locales.	2026-05-11 09:58:08 +02:00
Paul Nothaft	f3d0f161c9	fix(external-media): pre-generate thumbnails so reference-mode galleries load fast (#423) ... External (source_origin='external') photos always had thumbnail_path=NULL, so the gallery returned thumbnail_url=null and every layout fell back to streaming the full original from the NAS via the secure-image route. With ~100 NAS-mounted photos that meant minutes of wall-clock load time, sequential per tile. Two halves: 1. import-external route generates the thumbnail right after each successful insert and writes thumbnail_path on the row. Best-effort: a single failure logs a warning and leaves thumbnail_path=NULL — ensureThumbnail will retry lazily on first view. Synchronous in the loop adds ~100-300ms per image; for the worst-case 1000-photo import that's still under the typical request timeout. 2. ensureThumbnail() in imageProcessor handles external photos too — resolves the local NAS mount path via resolvePhotoFilePath instead of the storage-backend key. This covers existing externals already in the database that were imported before this fix: first gallery view per photo regenerates the thumbnail, subsequent views are fast. Filename-collision protection: external thumbnails use `thumb_ext<photoId>_<basename>` so two events both referencing e.g. `IMG_0001.jpg` on different NAS subtrees can't clobber each other's thumbnail. generateThumbnail accepts a new options.outputBasename to support this without changing the managed-photo behaviour. Verified locally with a 3-photo external dir and a real NAS-style import: POST /api/admin/external-media/events/N/import-external → {imported:3, thumbnailsGenerated:3, thumbnailsFailed:0} /api/gallery/<slug>/photos returns thumbnail_url for every photo Lazy-regen path: clearing thumbnail_path + deleting the file, then hitting /thumbnail/N regenerates and repopulates the row in 42ms. Closes #423.	2026-05-08 19:17:15 +02:00
Paul Nothaft	1b717ce5ed	feat: native S3 storage backend (#328) + presigned download follow-up ... Lets PicPeak write photos, thumbnails, hero images, watermarks, and archive zips to any S3-compatible bucket (AWS S3, MinIO, Cloudflare R2, Backblaze B2, Wasabi, DigitalOcean Spaces) instead of the local filesystem. Selected via STORAGE_BACKEND=local|s3. Architecture - backend/src/services/storage/StorageBackend.js — abstract interface (put/get/exists/stat/delete/list/copy/rename/signedUrl/putFromFile/ getToFile) — typedef-only, documents the contract. - LocalFsStorage.js — wraps fs with atomic-write-via-tmp-rename, path traversal protection, list-as-walker. - S3StorageBackend.js — thin wrapper around the existing S3StorageAdapter (used by backupService) mapping it onto the canonical interface; supports optional STORAGE_S3_PREFIX namespace. - index.js — factory selected by STORAGE_BACKEND with startup ping (HEADs sentinel key on S3, fs.stat on local) so misconfig fails fast before the first request. Consumer refactors (~12 services + routes), each parametrized over the abstraction: - imageProcessor / videoProcessor — pipe Sharp/ffmpeg output through storage.put; expose withLocalCopy() helper for S3-mode regeneration paths that need a local file for sharp/ffmpeg. - archiveService / downloadZipService — finalize zip in tmp dir, then storage.putFromFile. Atomic-rename pattern preserved on local; S3 emulates via copy + delete (worker prunes orphaned .tmp.* on startup). - photoProcessor / photoReplacementService / adminPhotos upload+delete / routes/v1/events.js POST /events/:id/photos / routes/events.js — every upload path now goes storage.putFromFile(temp) → unlink temp. - gallery.js bulk-download (cached + on-the-fly + selected) — managed photos via storage.get, external-mode unchanged. - protectedImages / secureImages / photoResolver — read via storage.get; resolvePhotoStorageKey returns the canonical key. - watermarkService / watermarkGeneratorService — persistent watermarks via storage.put. - fileWatcher — bails out with a clear log warning when STORAGE_BACKEND=s3 (chokidar can't watch S3); auto-import lands via the S3 prefix walker introduced in the follow-up commit. - expirationChecker — small touch (event.expired webhook fire from #327 shipping in the next commit). Migration tooling - backend/scripts/migrate-storage.js — one-shot --dry-run capable script that walks photos.path, thumbnail_path, hero_path, watermark_path and events.archive_path/download_zip_path; streams local → S3; sha256 size-match skip for idempotent re-run; failures CSV. Presigned-URL "Download All" (#328 follow-up shipped in this commit) - routes/gallery.js — when STORAGE_BACKEND=s3 + event.allow_presigned_download + downloads enabled + watermark NOT enabled, /download-all returns a 302 redirect to a 5-minute presigned S3 URL. Per-event opt-in surface ships in the next commit's UI. Tests - backend/__tests__/integration/storageBackend.test.js — parametrized contract suite running against BOTH LocalFs AND MinIO (18 tests, both backends — 36 cases total). - backend/__tests__/integration/imageProcessor.storage.test.js — same parametrized pattern for the image processor (10 tests × 2 backends). - backend/__tests__/integration/backup-s3.test.js — bootstrap fix: drop the redundant initDb() (001_init handles it) and remove schema-drift in configureS3Backup (app_settings has no created_at anymore and the unique constraint is on setting_key alone, not composite). 0/12 → 7/12 (5 remaining are unrelated assertion drift). - backend/src/services/photoResolver.js — mixed-source events (reference mode with managed-uploaded photos) now fall back to managed when external_relpath is missing instead of throwing. - tests/e2e/s3-storage-roundtrip.spec.ts — Playwright spec that auto-skips against local backend; full upload → serve → delete round-trip when run against an S3-mode backend. Server wiring (server.js) - initStorage() called after database init, before rate limiters. - This commit's diff also includes the webhook delivery worker startup and the S3 auto-importer startup. Those features ship in the next two commits — co-located here for one bisectable diff per file. Docs + ops - README §"Storage Backends" — capability matrix, switching playbook, IAM policy snippet, MinIO/R2/B2 examples. - README §"Webhooks" — also added here (full diff bundled). - .env.example — STORAGE_BACKEND + STORAGE_S3_* + STORAGE_AUTO_IMPORT documented; WEBHOOK_* added in the same diff. - .gitignore — re-anchor the existing `storage/` rule to `/storage/` so backend/src/services/storage/ (the new abstraction code) is trackable. The runtime ./storage/ data dir stays ignored. Out of scope for v1 (per the issue): presigned URLs for individual photo display (always streamed for protection middleware), CDN integration, hybrid hot/cold tiers, S3 → local migration, multi-bucket per-event.	2026-04-28 10:06:36 +02:00
Paul Nothaft	2b25d81144	security: comprehensive hardening across frontend, backend, and infrastructure ... - Disable production source maps and hide nginx version - Reduce JSON body limit from 10gb to 50mb (uploads use multer, not JSON) - Strip database info and error details from health endpoint - Mask reCAPTCHA secret key in admin settings API responses - Whitelist sort/order query parameters in events and photos endpoints - Stop reflecting arbitrary origins in static file CORS headers - Align nginx security headers with backend Helmet CSP, remove deprecated X-XSS-Protection - Strip EXIF metadata from generated thumbnails and hero images - Bind postgres/redis dev ports to localhost in docker-compose configs - Add safeExec utility (spawn with shell:false) to prevent command injection - Convert all exec/execAsync calls in backup, restore, and database backup services to use safe spawn-based helpers	2026-02-16 22:33:20 +01:00
Paul Nothaft	50c09904a9	feat: add update instructions dialog, email notifications, and capture date sorting ... - Add Update Instructions Dialog with environment-specific commands (Docker/Git/Standalone) - Add email notification settings for new version alerts - Add "Sort by Capture Date" option using EXIF metadata extraction - Fix E2E tests by loading environment variables via dotenv - Add test-images/ and backend/*.db to .gitignore Closes #181	2026-02-16 16:23:57 +01:00
Paul Nothaft	5cef7fdd18	fix: restore aspect-ratio layouts and improve hero image quality (#180) ... - Fix masonry/mosaic layout regression where tiles displayed uniform heights instead of respecting image aspect ratios. Changed from fixed 150-500px height constraints to dynamic constraints based on column width. - Add hero image optimization pipeline generating 1920x1080 images for full-width hero sections instead of using low-quality thumbnails. - New /hero/:photoId endpoint serves optimized hero images with watermark support and automatic generation/caching. - Add hero_url field to photos API response for frontend consumption. - Migration 069 adds hero_path column to photos table.	2026-02-15 22:43:18 +01:00
paul	9a75f1c929	Add video support, media filters, and translations	2025-11-28 13:29:44 +01:00
Claude	d91ab436e8	Fix multiple bugs: thumbnail generation, branding settings, categories, theme, feedback icons, upload limit, email errors ... Bug fixes included: #52 - Thumbnail Generation: Added proper parsing of settings values and validation of Sharp fit parameter to handle JSON-encoded strings correctly #61 - Branding Settings Not Persisting: Added _parseBoolean helper for reliable boolean parsing, added hide_powered_by option for white-label support #55 - Categories Not Applied: Fixed category update logic to properly handle numeric category IDs, added updated_at timestamp, improved cache invalidation #59/#56 - Gallery Layout & Apply Theme: Set isPreviewMode=true so theme changes immediately propagate to parent state, hidden redundant Apply button #58 - Feedback Icons Show When Disabled: Added feedbackEnabled check to comment and like buttons in MasonryGalleryLayout and GridGalleryLayout #57 - Upload Limit 100MB: Increased body parser limit from 100MB to 500MB to support larger batch uploads #54 - Wrong Error Message: Enhanced email error handling with specific error codes and translation keys for better user feedback	2025-11-25 19:31:07 +00:00
paul	49c77785e7	feat(admin): external media import modal + thumbnail fixes for reference events\n\n- Photos tab: replace inline external folder picker with a modal opened via "Import from External Folder" button next to "Upload Photos"; add info that all pictures in the selected folder will be imported.\n- Admin thumbnails: align list endpoint to /api/admin/photos/:eventId/photos and always return thumbnail_url to trigger on-demand generation; normalize external paths to avoid duplicated folder segments (e.g., individual/individual) that broke resolver; improve thumbnail logging.\n- Use authenticated image fetching on admin feedback pages to prevent 401s in automation.\n- i18n: add backup.external.warning strings; complete German backup/restore coverage; add common keys (notSet, of, up, select, selected).\n- Docs: add Local (npm) setup for EXTERNAL_MEDIA_ROOT in deployment guide.\n\nRefs #17 – gallery feature request: https://github.com/the-luap/picpeak/issues/17	2025-09-05 23:44:30 +02:00
paulandClaude	1b4b497fdf	chore: clean up codebase for production readiness ... - Remove all console.log/debug statements from production code - Add NODE_ENV checks for development-only logging - Remove test scripts (test-feedback, test-image-security, test-backup-*, test-restore) - Remove one-time fix scripts (fix-temp-photos, fix-migration-state, mark-migration-applied) - Remove sensitive files (.env.backup, ADMIN_CREDENTIALS.txt) - Update package.json to remove references to deleted scripts - Replace console statements with logger utility in backend - Secure error boundaries to not expose stack traces in production This makes the codebase production-ready with no debug output or test scripts. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>	2025-08-24 23:19:30 +02:00
paulandClaude	1773ed5f95	Initial commit - Project start (July 17, 2025) ... Original: feat: enhance security logging and ensure rate limit blocks are properly tracked - Add comprehensive logging for rate limit blocks with full request details - IP address (with proper proxy detection), user agent, headers, timestamps - Rate limit info (current count, limit, remaining, reset time) - Separate tracking for auth vs general endpoints - Enhance authentication failure logging - JWT validation failures with detailed error info - Admin auth attempts without token - Failed token validation with user context - All events include IP, path, method, user agent - Improve Winston logger configuration for production - Add automatic log rotation (10MB errors, 50MB combined) - Create separate security.log for auth/rate limit events - Ensure logs directory exists automatically - Add structured JSON format for log aggregation - Support container logging with LOG_TO_CONSOLE env var - Create comprehensive documentation - Security logging guide with examples - Monitoring recommendations - Configuration reference - Add test script to verify logging functionality All rate limit settings remain configurable via admin panel: - Window duration, max requests, auth limits - Skip authenticated requests option - Public endpoints only option 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>	2025-07-24 16:57:07 +02:00