fix(images): single-flight lazy rendition generation and keep the old rendition during replacement (#1350)

* fix(images): single-flight lazy rendition generation and keep the old rendition during replacement

The lazy generators in imageProcessor are check-then-generate, and the
check reads the path off the photo row the route already fetched. N
concurrent cold requests for one photo all held a snapshot with the path
still null, all missed, and all ran the same sharp pipeline. Only the
thumbnail tier path had a guard; the canonical thumbnail it falls back
to, heroes, previews and preview tiers had none.

One process-local map now covers every rendition, keyed by photo id and
rendition (`thumbnail:<id>`, `thumbnail:<id>:w<width>`, `hero:<id>`,
`preview:<id>`, `preview:<id>:w<width>`). Concurrent callers share one
promise; the entry is cleared in a finally on success and failure alike
so a rejection cannot poison the key. The tier stat moved inside the
flight so a request arriving as the previous flight clears finds the
written tier instead of missing on a stale probe.

Heroes and previews also deleted the existing object before generating
its replacement, and again in the catch. Both are gone, mirroring what
the thumbnail generator already does: put is the last statement in the
try and replaces atomically on local storage and by key on S3, so the
delete only ever opened a window with no rendition at all, and a source
that failed to read stripped the old rendition with the row still
pointing at it.

No re-read of the photo row inside the flight: the admin regenerate
endpoints force a rebuild by passing a row with the path nulled, and a
re-read would hand back the persisted rendition untouched.

Fixes the single-flight half of issue 1020. The preview cache-key
extension mismatch and any server-wide work queue remain separate.

* fix(images): keep the snapshot validity check outside the single-flight

With the check inside the flight, an admin regeneration (row passed with
the path nulled) could join a viewer's flight for the same photo that was
merely confirming an already-good rendition, and be handed back the very
file it was asked to replace while the endpoint counted a success. Only a
miss enters the flight now; inside it everything is a regeneration.

* fix(images): forced rebuilds run after an in-flight lazy generation instead of adopting it

The admin regenerate endpoints could still join a lazy flight that was
already generating for the same photo. That flight read the thumbnail
settings when it started, so after a settings change it produces exactly
the rendition the regenerate was invoked to replace; adopting it counted
a success while the old size stayed cached.

ensureThumbnail and ensurePreviewImage take `{ force: true }`: skip the
snapshot check and, if a flight is pending, start after it settles. Lazy
misses arriving meanwhile join the forced flight, and an older flight
settling late no longer evicts the newer entry from the map.

* fix(images): key rendition flights by source as well as photo id

replacePhoto keeps the photo id and changes path and filename. Keyed by
id and width alone, a request carrying the replacement row joined a
flight still rendering the file it replaced and was handed the old
image, which the gallery caches for 30 minutes. The tier map this
replaced was keyed by storage key and so already told the two apart.

* test(thumbnails): wait for the regenerate loop's completion line instead of a fixed 150 ms

The loop runs in setImmediate after the response. Under a loaded machine
(fifteen suites in parallel, each booting a migrated SQLite) it took
longer than 150 ms once and the assertions ran against a half-finished
mock call list. Poll the logger spy for the "regeneration complete" line
with a 10 s deadline; the suite also finishes sooner because the wait
ends as soon as the loop does.

---------

Co-authored-by: Paul Nothaft <[email protected]>

backend/__tests__/integration/adminThumbnails.regenerate.test.js

@@ -23,7 +23,7 @@ const express = require('express');
 const request = require('supertest');
 
 describe('admin thumbnail regeneration (#1129)', () => {
-  let tmpDir; let db; let cleanup; let app; let imageProcessor; let storage;
+  let tmpDir; let db; let cleanup; let app; let imageProcessor; let storage; let logInfo;
 
   beforeAll(async () => {
     tmpDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'picpeak-regen-'));
@@ -54,6 +54,10 @@ describe('admin thumbnail regeneration (#1129)', () => {
       deletePreviewTiers: jest.fn().mockResolvedValue(undefined),
     }));
 
+    // Same module registry as the route, so the spy sees its calls. The
+    // completion line is what drain() below waits for.
+    logInfo = jest.spyOn(require('../../src/utils/logger'), 'info');
+
     // bootCrmDb, not run-migrations: the latter calls process.exit(0) on
     // success, which ends the jest worker mid-suite.
     ({ db, cleanup } = await require('./helpers/crmDb').bootCrmDb());
@@ -94,8 +98,20 @@ describe('admin thumbnail regeneration (#1129)', () => {
     return typeof row === 'object' ? row.id : row;
   }
 
-  /** The work runs in setImmediate; give it room to finish. */
-  const drain = () => new Promise((resolve) => setTimeout(resolve, 150));
+  /**
+   * The work runs in setImmediate, after the response. Wait for the loop's
+   * "regeneration complete" log line rather than a fixed 150 ms: under a
+   * loaded machine (fifteen suites in parallel, each booting a migrated
+   * SQLite) the loop occasionally took longer than that, and the assertions
+   * then ran against a half-finished mock call list.
+   */
+  const drain = async () => {
+    const deadline = Date.now() + 10000;
+    const done = () => logInfo.mock.calls.some((c) => /regeneration complete/.test(String(c[0])));
+    while (!done() && Date.now() < deadline) {
+      await new Promise((resolve) => setTimeout(resolve, 10));
+    }
+  };
 
   it('rebuilds the canonical thumbnail for an external photo instead of erroring', async () => {
     const eventId = await seedEvent();