* fix(images): backfill orientation for libraries that predate the fix (#1198)
#1194 corrected the generators and every ingest path, but did nothing for
photos already in the database. Those rows end up worse than untouched ones:
before the fix a rotated photo was CONSISTENTLY wrong — a sideways image in a
tile shaped to match — and afterwards the regenerated thumbnail is correct
while photos.width/height still describe the raw sensor order, so masonry and
justified size a portrait photo with a landscape ratio. The dimension repair
cannot reach them: it only selects rows with a NULL dimension, and an affected
row has both, just transposed.
Its own job rather than a mode of that one. They look alike but are not the
same operation: the repair FILLS missing values and touches nothing else,
while this RECOMPUTES and invalidates the derived data generated against the
old orientation. Sharing a lease would also mean one blocks the other.
A first attempt at this was reverted from #1194 after review found five
problems. All five are addressed here:
- Originals are read through resolvePhotoStorageKey + withLocalCopy +
withProcessableImage, so the job works on S3 installs and on RAW/DNG. The
dimension repair's direct fs read does neither, which stops being an edge
case in a job that walks the whole library.
- The canonical preview is cleared BEFORE faces are requeued.
ensurePreviewImage returns a cached preview whenever it is still a valid
image, and a pre-fix unrotated one is perfectly valid — so requeueing alone
made the rescan read unrotated pixels and scale those boxes by the corrected
dimensions, which is worse than leaving the data alone.
- Invalidation keys off the EXIF transform, not a dimension delta. Orientations
2, 3 and 4 move every pixel while leaving width and height unchanged, as does
5-8 on a square image; a delta check skips exactly those rows.
- Archived events are excluded — archiving deletes the originals and keeps the
rows, so every one of them would fail its read.
- The dimension write and the invalidation share a transaction. Split, a
failure between them leaves stale face data that no retry can fix, because
the retry computes "already correct".
Tier deletion stays outside the transaction on purpose: it touches storage, and
a failed object delete must not roll back a correct database write. A leftover
tier regenerates on next read; a rolled-back write is silent corruption.
* fix(images): invalidate every stale rendition, fence the writes, and give the job a button (#1198)
Three things from review, one of which mattered a lot.
The invalidation was too narrow. Clearing only preview_path fixed the face
data and left the gallery worse off: ensureThumbnail and ensureHeroImage
return their cached file whenever it is merely VALID, and a pre-fix sideways
thumbnail is perfectly valid — so a corrected row rendered the old sideways
image inside a newly-corrected portrait tile. All three canonical renditions
are cleared now, their stored objects deleted, and both responsive tier sets
with them.
The responsive tiers also needed handling rather than a hopeful catch. Their
helpers swallow delete errors, and ensurePreviewImageAtWidth treats
storage.stat(key) as a cache hit — so a tier that survived deletion keeps
serving unrotated forever and never regenerates. The keys are re-checked after
deletion and survivors are counted into the result, so a run that could not
clear them does not report itself as clean.
Writes are fenced on the identity that was measured, not just the id.
replacePhoto swaps a new file under an existing row and rewrites
path/filename, and it IS reachable — from the replace_by_name upload path in
adminPhotos.js. A replacement landing while this job read the old original
would otherwise have had the previous file's dimensions written over it and
its fresh renditions cleared.
And the job had no way to start it: the endpoint existed with no caller, so an
upgrade would have left every affected library untouched unless an operator
found the API themselves. It gets a Status card like its two neighbours, with
strings in en/de/fr/sl. No backlog counter, because unlike the other two it
cannot know how many rows need it without doing the work.
* fix(images): make the backfill idempotent, and stop it lying about what it did (#1198)
Six things from review round 2.
The job was not idempotent, and the way it failed was expensive. Its trigger is
the EXIF tag on the ORIGINAL, which correcting a photo never changes — so every
re-run threw away the renditions it had just regenerated and requeued every
completed face scan. On a face-enabled install, running it twice meant
re-detecting the whole library for nothing. Migration 191 adds
photos.orientation_checked_at, written in the same transaction as the work it
records, with `force` as the escape hatch for an interrupted run.
The candidate query selected preview_path but not thumbnail_path or hero_path,
which the deletion loop reads — so those two pointers were cleared in the
database while the objects stayed in storage, still reachable through
previously issued URLs.
watermark_path was missed entirely. gallery.js serves it ahead of the original
when branding watermarking is on, which makes it the most visible rendition of
the lot. (Its generator needed rotating too — that went into #1185, where the
other three live.)
storage.stat() RESOLVES with null for a missing key rather than rejecting, so
counting "the promise settled" marked every deleted — and every never-created —
tier as a survivor. A perfectly clean run told the operator to re-run. Now a
null means gone, and a rejection counts as stuck, since a storage error is not
proof the object went away.
Face data is invalidated whenever the stored dimensions change, not only when
the change came from rotation: boxes are scaled by photo.width at read time, so
any dimension change strands them.
And `corrected` now comes from the affected-row count. If the fence rejected the
write because the file was replaced mid-run, the photo was not corrected and
the run must not claim it was.
* fix(images): stop the backfill doing unnecessary work, and make its retry advice true (#1198)
Round 3, four points, all narrower than the last two rounds.
It re-processed photos that were already correct. A 5-8 rotation changes the
dimensions, so a tagged photo whose stored dimensions are ALREADY oriented must
have been ingested after #1185 — its renditions are fine and clearing them
deletes valid files and rescans a completed face detection for nothing. Those
are now skipped and simply marked. Orientations 2, 3 and 4 (and 5-8 on a square
image) leave the dimensions identical either way, so they carry no such
evidence and are still done once.
The retry advice was impossible to follow. When a responsive tier could not be
deleted the row was still marked, so the ordinary re-run the UI recommends
found nothing and the stale tier kept serving unrotated forever. The marker is
withheld when a tier survives, which is what makes that message honest.
Storage cleanup now only runs when a fenced write actually landed. If the file
was replaced mid-run every update matched zero rows, but the deletion went
ahead anyway and could destroy renditions belonging to the REPLACEMENT —
watermarks especially, which are keyed by photo id and alias straight onto the
new file.
And the full-photo ETag includes the backfill's timestamp. It was built from
the ORIGINAL's mtime plus the watermark settings hash, neither of which this
job touches — so a guest holding a pre-fix ETag would go on getting 304 and
their cached sideways image no matter how many times the backfill succeeded.
---------
Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
Paul Nothaft
edef4d7365