fix(external-media): one row per external file per event (#1162) (#1173)

* fix(external-media): one row per external file per event (#1162)

Stable twin of #1167.

Two overlapping import-external runs against the same event inserted every
file twice. The route checked for an existing external_relpath and then
inserted, with an fs.stat and a sharp().metadata() read sitting in between — a
window wide enough for both runs to see "not there". A reporter's event held
8004 rows for 6012 distinct paths. Nothing at the storage layer stopped it:
migration 041 created only a NON-unique (event_id, source_origin) index.

- migration 176 removes the existing duplicates and adds a partial unique
  index on (event_id, external_relpath), verified against the catalog
  afterwards — a failed CREATE INDEX raises 23505 on Postgres, which
  run-migrations-safe treats as "schema already exists" and would record as
  applied on an install that never got the index.
- dependent rows are removed explicitly rather than by cascade: PicPeak never
  sets `PRAGMA foreign_keys = ON`, so on SQLite the declared CASCADE is inert
  and a bare delete strands feedback and access-log rows. Guest feedback moves
  to the survivor instead of being discarded, keyed on guest identity the way
  feedbackService defines it, and the survivor's denormalized counters are
  recomputed.
- the route treats a unique violation as a skip, so a writer this process
  cannot see converges instead of duplicating, and a second import while one
  is running gets a 409.
- a .picpeak taken before migration 176 carries exactly these duplicates, and
  suspending FK enforcement does not suspend a unique index — so the restore
  drops the index for the load and rebuilds it after running the same dedupe.

Divergences from the main twin, both because the feature is absent here:
faces (no faceProcessor, so no purgePhotoFaces reconciliation — the rows are
still deleted so nothing dangles), admin marks, transfer membership, and
photos.view_count/download_count. The service guards each on hasTable /
hasColumn, so those branches simply do not fire.

Verified on this branch: 36 new tests pass; full suite leaves the same 5
pre-existing failures as origin/stable, unchanged.

* fix(external-media): invalidate the download zip when duplicates are removed (#1162)

External review. Same fix as the main twin.

The pre-built "download everything" archive still contained the duplicate rows
the dedupe had just deleted, so guests kept receiving them. Every ordinary
photo-deletion path calls downloadZipService.invalidate for exactly this
reason.

The columns are cleared rather than the service being called: that service
carries debounce timers and a regeneration queue, which a migration should not
start. getZipInfo already treats a cleared record as a cache miss and rebuilds
on the next request. The stale object is left in storage, as elsewhere.

---------

Co-authored-by: Paul Nothaft <[email protected]>

backend/src/services/picpeakImportService.js

@@ -26,6 +26,11 @@ const { getStoragePath } = require('../config/storage');
 const { hasColumnCached } = require('../utils/schemaCache');
 const logger = require('../utils/logger');
 const { PICPEAK_FORMAT_VERSION, EXCLUDED_TABLES, listDataTables } = require('./picpeakExportService');
+const {
+  dedupeExternalPhotos,
+  createExternalRelpathIndex,
+  dropExternalRelpathIndex,
+} = require('./externalPhotoDedupe');
 
 const isPostgres = () => knexConfig.client === 'pg';
 
@@ -288,6 +293,19 @@ async function replaceAllTables(tables, dataDir, currentAdmin, { crossEngine = f
       await trx.raw('PRAGMA defer_foreign_keys = ON');
     }
 
+    // Suspending FK enforcement does not suspend UNIQUE indexes on either
+    // engine (#1162). A backup taken before migration 176 carries the
+    // duplicate photo rows that migration exists to remove, so batchInsert
+    // below would hit photos_event_external_relpath_uniq and roll the whole
+    // restore back — after every table had already been emptied. Drop it for
+    // the load and rebuild it once the rows are deduped, which is the same
+    // repair the migration performs.
+    let hadRelpathIndex = false;
+    if (await trx.schema.hasColumn('photos', 'external_relpath')) {
+      hadRelpathIndex = true;
+      await dropExternalRelpathIndex(trx);
+    }
+
     for (const table of tables) {
       await trx(table).del();
     }
@@ -309,6 +327,18 @@ async function replaceAllTables(tables, dataDir, currentAdmin, { crossEngine = f
       await trx.batchInsert(table, prepared, 100);
     }
 
+    // Restore the constraint the load ran without. Deduping first because the
+    // incoming rows may be exactly the duplicates migration 176 removes; the
+    // index creation then also proves the repair worked, inside the same
+    // transaction that would otherwise leave the target unprotected.
+    if (hadRelpathIndex) {
+      const removed = await dedupeExternalPhotos(trx);
+      if (removed) {
+        logger.info(`picpeakImport: removed ${removed} duplicate external photo row(s) from the archive (#1162)`);
+      }
+      await createExternalRelpathIndex(trx);
+    }
+
     await reinjectCurrentAdmin(trx, currentAdmin);
 
     // Reset the pg session flag BEFORE the connection returns to the pool.