From 410b8f8f6f7258c285446b3454164c9a20f2280f Mon Sep 17 00:00:00 2001 From: Paul Nothaft <53005142+the-luap@users.noreply.github.com> Date: Wed, 26 Aug 2026 08:43:13 +0200 Subject: [PATCH] fix(external-media): record capture dates on import, and backfill existing libraries (#1172) (#1179) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(external-media): record capture dates on import, and backfill existing libraries (#1172) External imports never read EXIF, so photos.captured_at stayed NULL for every row they created. The gallery sorts "Date Taken" with COALESCE(captured_at, uploaded_at), which on a bulk import is the import timestamp — so the sort silently degraded into "order by import batch" with no error and nothing in the UI to say the sort key was missing. The reporter's 12-day trip came back with its first two days at positions 4204-5296 of 5555, because those folders happened to be imported second. - the import reads the capture date next to the sharp().metadata() call that already opens the file, so this costs one more read of the same source rather than a second pass over the mount. Best-effort like the dimensions: a source without EXIF imports with captured_at NULL, as before. - POST /api/admin/photos/repair-capture-dates backfills existing libraries, modelled on the dimension repair beside it — background pass, in-flight guard, status endpoint, and resolvePhotoFilePath, which is what reaches an external row at all. Not a migration: the originals sit on a mount that may be down at upgrade time, reading 8000+ of them would block the boot, and a run that found nothing has to be repeatable. - "no EXIF date" is counted separately from "could not read the file". An operator needs to tell "these files carry no date" from "the mount is broken" before deciding to re-run. - the update is guarded whereNull, so an import finishing mid-run is not overwritten by a slower pass. - every sort branch now carries photos.id as a tiebreaker, not just capture_date. A bulk import writes hundreds of rows inside one second, so uploaded_at and the COALESCE fallback both collapse and the grid reshuffles between loads. id is insertion order, which makes the fallback meaningful. Not addressed: extractCaptureDate reads no OffsetTimeOriginal, and exifr resolves a naive EXIF timestamp against the HOST timezone — so captured_at is not a true instant, and the same file imported on two machines yields two values. That predates this and applies to managed uploads equally; the tests here deliberately assert ordering rather than an absolute instant so they do not encode the bug. Worth its own issue. * fix(capture-dates): read managed originals through storage, skip archived, claim the run flag (#1172) Four holes in the backfill endpoint, all found in review: - Managed photos were resolved with resolvePhotoFilePath, which builds a STORAGE_PATH filesystem path. On an S3 install nothing is there, so every managed row failed. Now split the way the thumbnail regenerator does: external rows read from the mount directly, managed rows go through resolvePhotoStorageKey + withLocalCopy. - Archived events keep their photos rows but their originals are deleted on archive, so those rows failed every run and kept the button lit forever. Excluded from both the job and the status counts. - isRunning was claimed after the candidate query, so two concurrent POSTs could both pass the guard and start a pass. Claimed before the await, with every early exit releasing it. - The noExif comment promised a distinction extractCaptureDate does not make (it returns null for unreadable files too). Reworded to what it is. * chore: drop a stray node_modules symlink committed by mistake The .gitignore pattern is `node_modules/`, which matches a directory and not a symlink of the same name, so a local convenience link slipped past it. It pointed at an absolute path on one machine and would dangle everywhere else, breaking `cd backend && npm install`. * fix(capture-dates): gate the backfill as system maintenance, stop overstating the counters (#1172) The endpoint walks every event in the install and rewrites their metadata, but required only photos.edit — which the built-in team_photographer preset holds (175_granular_permissions_and_presets.js:106). That role exists for a contributing shooter, who should not be able to start a whole-library S3/NAS scan or touch another owner's photos. Now system.manage, with the status endpoint on system.view so the panel simply stays hidden for everyone else. The "without EXIF date" wording also promised a distinction the code does not draw: extractCaptureDate returns null for an unparseable file as well as for one that genuinely carries no date, so both land in that bucket. Reworded to "no date found" / "unreachable" in en, de and fr, which is what the two numbers actually separate. * docs: point the permission note at the follow-up PR (#1172) The dimension repair's matching gate landed in #1182, so the comment no longer needs to describe it as unaddressed. * fix(i18n): align the Slovenian capture-date wording with the other locales (#1172) sl was missed when the counters were reworded from 'without EXIF date' / 'unreadable' to what they actually measure. * fix(capture-dates): gate the status card on the permission the button needs (#1172) system.view and system.manage are independent grants, and StatusTab has no permission gate of its own — a successful status payload is what renders the card and its enabled button (StatusTab.tsx:637). Gating the status endpoint on system.view therefore handed a system.view-only role a live Backfill button whose every click 403s, with no error surfaced by the mutation. The comment above it already claimed this endpoint matched the POST. Now it does. * fix(gallery): make the Date Taken sort correct on SQLite (#1172) photos.captured_at does not hold one type on SQLite. Three writers put three different things in it: integer managed uploads — photoProcessor.js:488 hands knex a Date, which the sqlite3 binding stores as epoch milliseconds text external imports and the backfill, which write ISO-8601 null no capture date, so the sort falls through to uploaded_at, itself text in knex's 'YYYY-MM-DD HH:MM:SS' default shape A plain COALESCE over that is not an ordering. SQLite sorts INTEGER before TEXT unconditionally, so every managed photo carrying EXIF came back ahead of every photo that did not, whatever the dates said — a 2027 capture landing before a 2020 one. Among the text values 'T' (0x54) also outranks the space (0x20), so a same-day ISO 01:15 sorted behind a fallback 23:00. Both failures predate this branch — the first needs only two managed photos — but making that sort correct is what #1172 is about, so it is fixed here rather than left for the issue it belongs to. Normalised in the ORDER BY rather than by rewriting the column: the data fix would have to touch every existing row and every writer, which is a far heavier change than the sort it corrects. The cost is that this sort no longer uses idx_photos_captured_at on SQLite — an acceptable trade on the fallback engine, where the alternative is an index-assisted wrong answer. Postgres is untouched: captured_at is a real timestamp there and COALESCE already compares correctly. The regression tests drive the real gallery route on real SQLite. They write the epoch-millisecond integer directly, because the Date that produces it in production cannot be reproduced inside jest — there the binding's type dispatch misses sandbox Dates and stores "[object Object]" (CLAUDE.md). All four behavioural tests fail on the unfixed ORDER BY; verified by reverting it. * fix(gallery): normalise epoch-integer uploaded_at too, and stop polling a 403 (#1172) Two follow-ups from review. uploaded_at is not always text on SQLite either. A legacy archive restore leaves epoch milliseconds in it — there is a test pinning exactly that (__tests__/integration/sqliteEpochTimestamps.test.js) — and the fallback branch read it with substr(), so '1830297600000' was compared against '2020-01-01 00:00:00' as text and a 2028 upload sorted first. Both columns now get the integer/real branch. The status card also polled every ten seconds regardless of permission. With the endpoint correctly requiring system.manage, anyone who can open the Status tab but cannot run the job would have had a 403 and a logged denial every ten seconds for a panel they were never shown. The query is now gated on the same permission the endpoint requires, so it never starts. * style: quote convention in the capture-sort test (#1172) * fix(capture-dates): skip watcher-imported videos, and make the status counts consistent (#1172) Three follow-ups from review. fileWatcher.processNewPhoto sets type='video' and a video/* mime but never media_type (fileWatcher.js:128-130), so those rows keep the 'image' default from migration 048. Filtering on media_type alone queued every such video on every run — extractCaptureDate returns null for a video, captured_at stays null, and the backlog never cleared. Candidate query and status scope now check all three markers. The status counts were two separate queries, so an import committing a dated photo between them could be counted by the second and not the first: the card then showed withCaptureDate > total and a negative backlog, with the button enabled to "fix" it. One aggregate now. And the card's render checked only the cached payload. TanStack keeps that after `enabled` flips false, so a lower-privileged admin logging in behind a system.manage user inside the cache lifetime would still have seen the card and a button whose POST 403s. The permission is part of the render condition now. --------- Co-authored-by: Paul Nothaft --- .../integration/captureDateBackfill.test.js | 202 +++++++++++++++ .../externalImportCaptureDate.test.js | 161 ++++++++++++ .../routes/gallerySqliteCaptureSort.test.js | 170 +++++++++++++ backend/src/routes/adminExternalMedia.js | 30 ++- backend/src/routes/adminPhotoDimensions.js | 239 ++++++++++++++++++ backend/src/routes/gallery.js | 61 ++++- .../src/features/settings/tabs/StatusTab.tsx | 98 +++++++ frontend/src/i18n/locales/de.json | 11 + frontend/src/i18n/locales/en.json | 11 + frontend/src/i18n/locales/fr.json | 11 + frontend/src/i18n/locales/sl.json | 11 + 11 files changed, 998 insertions(+), 7 deletions(-) create mode 100644 backend/__tests__/integration/captureDateBackfill.test.js create mode 100644 backend/__tests__/integration/externalImportCaptureDate.test.js create mode 100644 backend/__tests__/routes/gallerySqliteCaptureSort.test.js diff --git a/backend/__tests__/integration/captureDateBackfill.test.js b/backend/__tests__/integration/captureDateBackfill.test.js new file mode 100644 index 00000000..38074878 --- /dev/null +++ b/backend/__tests__/integration/captureDateBackfill.test.js @@ -0,0 +1,202 @@ +/** + * Backfilling captured_at on a library imported before #1172. + * + * The point of the endpoint, rather than a migration: it resolves originals + * through resolvePhotoFilePath, which is the only path that reaches an + * external row. The thumbnail regenerator resolves under + * storage/events/active/, which never exists for those (#1129) — + * so it cannot be the model. + */ + +const fs = require('fs'); +const path = require('path'); +const os = require('os'); +const express = require('express'); +const request = require('supertest'); +const sharp = require('sharp'); + +describe('capture date backfill (#1172)', () => { + let tmpDir; let db; let app; let mediaRoot; + + const writeJpegWithExif = async (abs, iso) => { + await fs.promises.mkdir(path.dirname(abs), { recursive: true }); + const d = new Date(iso); + const pad = (n) => String(n).padStart(2, '0'); + const exifDate = `${d.getUTCFullYear()}:${pad(d.getUTCMonth() + 1)}:${pad(d.getUTCDate())} ` + + `${pad(d.getUTCHours())}:${pad(d.getUTCMinutes())}:${pad(d.getUTCSeconds())}`; + await sharp({ create: { width: 60, height: 40, channels: 3, background: { r: 9, g: 9, b: 9 } } }) + .withExif({ IFD2: { DateTimeOriginal: exifDate } }).jpeg().toFile(abs); + }; + + const settle = async () => { for (let i = 0; i < 60; i++) { await new Promise((r) => setTimeout(r, 50)); const s = await status(); if (!s.body.isRunning) return s; } throw new Error('backfill did not settle'); }; + const status = () => request(app).get('/api/admin/photos/repair-capture-dates/status'); + + beforeAll(async () => { + tmpDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'picpeak-capfill-')); + mediaRoot = path.join(tmpDir, 'media'); + await fs.promises.mkdir(mediaRoot, { recursive: true }); + + process.env.NODE_ENV = 'test'; + process.env.TEST_DATABASE_PATH = path.join(tmpDir, 'data', 'db.sqlite'); + await fs.promises.mkdir(path.dirname(process.env.TEST_DATABASE_PATH), { recursive: true }); + process.env.STORAGE_PATH = path.join(tmpDir, 'storage'); + process.env.EXTERNAL_MEDIA_ROOT = mediaRoot; + process.env.JWT_SECRET = process.env.JWT_SECRET || 'capfill-secret'; + + jest.resetModules(); + jest.doMock('../../src/middleware/auth', () => ({ + adminAuth: (req, _res, next) => { req.admin = { id: 1, username: 'tester', roleName: 'admin' }; next(); }, + })); + jest.doMock('../../src/middleware/permissions', () => ({ + requirePermission: () => (_req, _res, next) => next(), + })); + jest.doMock('../../src/utils/logger', () => ({ + debug: jest.fn(), info: jest.fn(), warn: jest.fn(), error: jest.fn(), + })); + + ({ db } = await require('./helpers/crmDb').bootCrmDb()); + + app = express(); + app.use(express.json()); + app.use('/api/admin/photos', require('../../src/routes/adminPhotoDimensions')); + }, 180000); + + afterAll(async () => { + if (db) await db.destroy?.(); + await fs.promises.rm(tmpDir, { recursive: true, force: true }).catch(() => {}); + }); + + async function seed({ relpath, exifIso, writeFile = true, archived = false }) { + await db('photos').del(); + await db('events').del(); + const [e] = await db('events').insert({ + slug: 'capfill', event_type: 'wedding', event_name: 'capfill', event_date: '2026-01-01', + host_email: 'h@example.com', admin_email: 'a@example.com', password_hash: 'x', + share_link: `capfill-${Math.random()}`, expires_at: new Date().toISOString(), + source_mode: 'reference', external_path: 'trip', is_archived: archived, + }).returning('id'); + const eventId = typeof e === 'object' ? e.id : e; + if (writeFile) await writeJpegWithExif(path.join(mediaRoot, 'trip', relpath), exifIso); + const [p] = await db('photos').insert({ + event_id: eventId, filename: path.basename(relpath), path: `capfill/${path.basename(relpath)}`, + // Root-relative, as this branch stores it (#1163) — the file lives at + // /trip/. + type: 'individual', source_origin: 'external', external_relpath: `trip/${relpath}`, + uploaded_at: new Date().toISOString(), captured_at: null, + }).returning('id'); + return { eventId, photoId: typeof p === 'object' ? p.id : p }; + } + + it('fills captured_at for an external photo the thumbnail regenerator cannot reach', async () => { + const { photoId } = await seed({ relpath: 'a.jpg', exifIso: '2026-06-01T09:45:03Z' }); + + const res = await request(app).post('/api/admin/photos/repair-capture-dates'); + expect(res.status).toBe(200); + expect(res.body.count).toBe(1); + const done = await settle(); + + expect(done.body.lastResult.success).toBe(1); + expect((await db('photos').where({ id: photoId }).first()).captured_at).toBeTruthy(); + }); + + it('counts a photo with no EXIF separately from a failure', async () => { + // "The mount is broken" and "these files carry no date" need different + // answers from an operator, so they are not the same number. + await db('photos').del(); await db('events').del(); + const { photoId } = await seed({ relpath: 'plain.jpg', exifIso: '2026-06-01T09:45:03Z', writeFile: false }); + await sharp({ create: { width: 40, height: 30, channels: 3, background: { r: 1, g: 1, b: 1 } } }) + .jpeg().toFile(path.join(mediaRoot, 'trip', 'plain.jpg')); + + await request(app).post('/api/admin/photos/repair-capture-dates'); + const done = await settle(); + + expect(done.body.lastResult).toMatchObject({ success: 0, noExif: 1, failed: 0 }); + expect((await db('photos').where({ id: photoId }).first()).captured_at).toBeNull(); + }); + + it('counts an unreachable original as a failure, not as missing EXIF', async () => { + await seed({ relpath: 'gone.jpg', exifIso: '2026-06-01T09:45:03Z', writeFile: false }); + + await request(app).post('/api/admin/photos/repair-capture-dates'); + const done = await settle(); + + expect(done.body.lastResult).toMatchObject({ success: 0, noExif: 0, failed: 1 }); + }); + + it('reports nothing to do once every photo has a date', async () => { + const { photoId } = await seed({ relpath: 'b.jpg', exifIso: '2026-06-02T09:00:00Z' }); + await db('photos').where({ id: photoId }).update({ captured_at: new Date().toISOString() }); + + const res = await request(app).post('/api/admin/photos/repair-capture-dates'); + + expect(res.body.count).toBe(0); + expect((await status()).body.withoutCaptureDate).toBe(0); + }); + + it('skips a watcher-imported video, which carries media_type "image"', async () => { + // fileWatcher.processNewPhoto sets type='video' and a video/* mime but + // never media_type (fileWatcher.js:128-130), so the row keeps the 'image' + // default from migration 048. Filtering on media_type alone queued it every + // run: extractCaptureDate returns null for a video, captured_at stays null, + // and the backlog never cleared. + const { eventId } = await seed({ relpath: 'clip.jpg', exifIso: '2026-06-01T09:45:03Z', writeFile: false }); + await db('photos').del(); + await db('photos').insert({ + event_id: eventId, filename: 'clip.mp4', path: 'capfill/clip.mp4', + type: 'video', media_type: 'image', mime_type: 'video/mp4', + source_origin: 'external', external_relpath: 'trip/clip.mp4', + uploaded_at: new Date().toISOString(), captured_at: null, + }); + + const res = await request(app).post('/api/admin/photos/repair-capture-dates'); + expect(res.body.count).toBe(0); + + const s = await status(); + // And it is not counted as a permanent backlog either. + expect(s.body.total).toBe(0); + expect(s.body.withoutCaptureDate).toBe(0); + }); + + it('never reports more dated photos than it has photos', async () => { + // Both counts come from one aggregate; as two queries an import committing + // between them produced withCaptureDate > total and a negative backlog. + const { photoId } = await seed({ relpath: 'counted.jpg', exifIso: '2026-06-05T08:00:00Z' }); + await db('photos').where({ id: photoId }).update({ captured_at: new Date().toISOString() }); + + const s = await status(); + expect(s.body.total).toBe(1); + expect(s.body.withCaptureDate).toBe(1); + expect(s.body.withoutCaptureDate).toBe(0); + expect(s.body.withoutCaptureDate).toBeGreaterThanOrEqual(0); + }); + + it('skips archived events instead of failing them on every run', async () => { + // Archiving deletes the originals and keeps the rows, so an archived photo + // can never get a date. Counting it would fail it every pass and leave the + // status endpoint permanently reporting a backlog. + await seed({ relpath: 'archived.jpg', exifIso: '2026-06-04T09:00:00Z', archived: true }); + + const res = await request(app).post('/api/admin/photos/repair-capture-dates'); + + expect(res.body.count).toBe(0); + const s = await status(); + expect(s.body.total).toBe(0); + expect(s.body.withoutCaptureDate).toBe(0); + expect(s.body.isRunning).toBe(false); + }); + + it('does not overwrite a date written while it was running', async () => { + // whereNull on the update: an import or a replacement finishing mid-run has + // already written a better value than this pass would. + const { photoId } = await seed({ relpath: 'c.jpg', exifIso: '2026-06-03T09:00:00Z' }); + const claimed = '2020-01-01T00:00:00.000Z'; + + const res = await request(app).post('/api/admin/photos/repair-capture-dates'); + expect(res.body.count).toBe(1); + await db('photos').where({ id: photoId }).update({ captured_at: claimed }); + const done = await settle(); + + expect(new Date((await db('photos').where({ id: photoId }).first()).captured_at).toISOString()).toBe(claimed); + expect(done.body.lastResult.success).toBe(0); + }); +}); diff --git a/backend/__tests__/integration/externalImportCaptureDate.test.js b/backend/__tests__/integration/externalImportCaptureDate.test.js new file mode 100644 index 00000000..e141c70c --- /dev/null +++ b/backend/__tests__/integration/externalImportCaptureDate.test.js @@ -0,0 +1,161 @@ +/** + * External imports must record captured_at (#1172). + * + * Managed uploads get it from photoProcessor, which external media never goes + * through — so every externally imported photo carried captured_at NULL, and + * the gallery's "Date Taken" sort fell back to uploaded_at through its + * COALESCE. On a library imported in two batches that ordered a 12-day trip by + * which folder was imported first: the reporter's first two days landed at + * positions 4204-5296 of 5555. + * + * Driven through the real route against real files carrying real EXIF, because + * the whole question is whether the import reads the file it already has open. + */ + +const fs = require('fs'); +const path = require('path'); +const os = require('os'); +const express = require('express'); +const request = require('supertest'); +const sharp = require('sharp'); + +describe('external import capture dates (#1172)', () => { + let tmpDir; let db; let app; let mediaRoot; + + /** + * A real JPEG carrying DateTimeOriginal. + * + * IFD2, not IFD0 — DateTimeOriginal lives in the Exif IFD, and exifr does not + * see it anywhere else (IFD0 takes plain DateTime, which surfaces as + * ModifyDate instead). + */ + const writeJpegWithExif = async (rel, iso) => { + const full = path.join(mediaRoot, rel); + await fs.promises.mkdir(path.dirname(full), { recursive: true }); + const d = new Date(iso); + const pad = (n) => String(n).padStart(2, '0'); + const exifDate = `${d.getUTCFullYear()}:${pad(d.getUTCMonth() + 1)}:${pad(d.getUTCDate())} ` + + `${pad(d.getUTCHours())}:${pad(d.getUTCMinutes())}:${pad(d.getUTCSeconds())}`; + await sharp({ create: { width: 60, height: 40, channels: 3, background: { r: 10, g: 20, b: 30 } } }) + .withExif({ IFD2: { DateTimeOriginal: exifDate } }) + .jpeg() + .toFile(full); + return full; + }; + + const writeJpegNoExif = async (rel) => { + const full = path.join(mediaRoot, rel); + await fs.promises.mkdir(path.dirname(full), { recursive: true }); + await sharp({ create: { width: 60, height: 40, channels: 3, background: { r: 200, g: 10, b: 10 } } }) + .jpeg().toFile(full); + }; + + beforeAll(async () => { + tmpDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'picpeak-capdate-')); + mediaRoot = path.join(tmpDir, 'media'); + await fs.promises.mkdir(mediaRoot, { recursive: true }); + + process.env.NODE_ENV = 'test'; + process.env.TEST_DATABASE_PATH = path.join(tmpDir, 'data', 'db.sqlite'); + await fs.promises.mkdir(path.dirname(process.env.TEST_DATABASE_PATH), { recursive: true }); + process.env.STORAGE_PATH = path.join(tmpDir, 'storage'); + process.env.EXTERNAL_MEDIA_ROOT = mediaRoot; + process.env.JWT_SECRET = process.env.JWT_SECRET || 'capdate-secret'; + + jest.resetModules(); + jest.doMock('../../src/middleware/auth', () => ({ + adminAuth: (req, _res, next) => { req.admin = { id: 1, username: 'tester', roleName: 'admin' }; next(); }, + })); + jest.doMock('../../src/middleware/permissions', () => ({ + requirePermission: () => (_req, _res, next) => next(), + })); + jest.doMock('../../src/middleware/ownership', () => ({ + requireEventOwnership: (_req, _res, next) => next(), + })); + jest.doMock('../../src/services/imageProcessor', () => { + const actual = jest.requireActual('../../src/services/imageProcessor'); + return { ...actual, generateThumbnail: jest.fn(async () => 'thumbnails/mock.jpg'), ensureThumbnail: jest.fn() }; + }); + jest.doMock('../../src/utils/logger', () => ({ + debug: jest.fn(), info: jest.fn(), warn: jest.fn(), error: jest.fn(), + })); + + ({ db } = await require('./helpers/crmDb').bootCrmDb()); + + app = express(); + app.use(express.json()); + app.use('/api/admin/external-media', require('../../src/routes/adminExternalMedia')); + }, 180000); + + afterAll(async () => { + if (db) await db.destroy?.(); + await fs.promises.rm(tmpDir, { recursive: true, force: true }).catch(() => {}); + }); + + async function seedEvent() { + await db('photos').del(); + await db('events').del(); + await fs.promises.rm(mediaRoot, { recursive: true, force: true }); + await fs.promises.mkdir(mediaRoot, { recursive: true }); + const [e] = await db('events').insert({ + slug: `capdate-${Math.random().toString(36).slice(2, 8)}`, + event_type: 'wedding', event_name: 'capdate', event_date: '2026-01-01', + host_email: 'h@example.com', admin_email: 'a@example.com', password_hash: 'x', + share_link: `capdate-${Math.random()}`, expires_at: new Date().toISOString(), + source_mode: 'reference', + }).returning('id'); + return typeof e === 'object' ? e.id : e; + } + + const runImport = (eventId, external_path) => request(app) + .post(`/api/admin/external-media/events/${eventId}/import-external`) + .send({ external_path, recursive: true }); + + it('records the EXIF capture date on import', async () => { + const eventId = await seedEvent(); + await writeJpegWithExif('trip/a.jpg', '2026-06-01T09:45:03Z'); + + await runImport(eventId, 'trip'); + + const photo = await db('photos').where({ event_id: eventId }).first(); + expect(photo.captured_at).toBeTruthy(); + // NOT asserted as an absolute instant. EXIF carries a naive wall-clock + // time and exifr resolves it against the HOST timezone, so the stored UTC + // value differs between a CEST developer machine and a UTC runner. What + // this fix is about is that the field is populated and orders correctly; + // that captured_at is not a true instant is a separate, pre-existing + // problem shared with managed uploads (#1172's own footnote). + expect(new Date(photo.captured_at).getUTCFullYear()).toBe(2026); + expect(new Date(photo.captured_at).getUTCMonth()).toBe(5); // June + }); + + it('imports a photo with no EXIF date rather than failing it', async () => { + // Plenty of sources carry none; that must stay an import, not an error. + const eventId = await seedEvent(); + await writeJpegNoExif('trip/plain.jpg'); + + const res = await runImport(eventId, 'trip'); + + expect(res.body.imported).toBe(1); + const photo = await db('photos').where({ event_id: eventId }).first(); + expect(photo.captured_at).toBeNull(); + }); + + it('orders a two-batch import by capture time, not by batch', async () => { + // The reported shape: the FIRST days of the trip imported second. Sorting + // on COALESCE(captured_at, uploaded_at) put them after the last days, + // because uploaded_at is the import timestamp. + const eventId = await seedEvent(); + await writeJpegWithExif('late/day12.jpg', '2026-06-12T10:00:00Z'); + await runImport(eventId, 'late'); + await writeJpegWithExif('early/day01.jpg', '2026-06-01T10:00:00Z'); + await runImport(eventId, 'early'); + + const rows = await db('photos') + .where({ event_id: eventId }) + .orderByRaw('COALESCE(captured_at, uploaded_at) asc') + .select('filename'); + + expect(rows.map((r) => r.filename)).toEqual(['day01.jpg', 'day12.jpg']); + }); +}); diff --git a/backend/__tests__/routes/gallerySqliteCaptureSort.test.js b/backend/__tests__/routes/gallerySqliteCaptureSort.test.js new file mode 100644 index 00000000..6024d7ed --- /dev/null +++ b/backend/__tests__/routes/gallerySqliteCaptureSort.test.js @@ -0,0 +1,170 @@ +/** + * "Date Taken" ordering across SQLite's storage classes (#1172). + * + * photos.captured_at does not hold one type on SQLite. Three writers put three + * different things in it: + * + * integer managed uploads — photoProcessor.js:488 hands knex a Date, which + * the sqlite3 binding stores as epoch milliseconds + * text external imports and the capture-date backfill, which write + * ISO-8601 ('2026-06-03T01:15:00.000Z') + * null no capture date, so the sort falls through to uploaded_at — + * itself text, in knex's 'YYYY-MM-DD HH:MM:SS' shape + * + * A plain COALESCE over that mixture is not an ordering. SQLite sorts INTEGER + * before TEXT unconditionally, so every managed photo carrying EXIF came back + * ahead of every photo that did not, whatever the dates said. And among the + * text values 'T' (0x54) outranks the space (0x20), so a same-day ISO 01:15 + * sorted behind a fallback 23:00. + * + * Both failures predate #1172 — the first needs only two managed photos — but + * the sort is what that issue is about, so they are fixed and pinned here. + * Every test below fails on the unfixed ORDER BY. + */ + +const path = require('path'); +const fs = require('fs'); +const os = require('os'); + +process.env.NODE_ENV = 'test'; +process.env.TEST_DATABASE_PATH = path.join( + fs.mkdtempSync(path.join(os.tmpdir(), 'picpeak-capsort-')), 'db.sqlite', +); +process.env.JWT_SECRET = process.env.JWT_SECRET || 'capsort-test-secret'; +process.env.STORAGE_PATH = fs.mkdtempSync(path.join(os.tmpdir(), 'picpeak-capsort-storage-')); + +const request = require('supertest'); +const express = require('express'); +const cookieParser = require('cookie-parser'); +const { bootCrmDb, seedMinimal } = require('../integration/helpers/crmDb'); + +const SLUG = 'capsort-gallery'; + +describe('capture-date ordering on SQLite (#1172)', () => { + let db; let cleanup; let app; let eventId; + + // Managed uploads store an epoch-millisecond INTEGER, because + // photoProcessor.js:488 hands knex a Date and the sqlite3 binding converts + // it. That conversion cannot be reproduced from inside jest — there the + // binding's type dispatch misses sandbox-created Dates and writes the string + // "[object Object]" instead (CLAUDE.md). Verified outside jest: a Date lands + // as {"c":1830211200000,"ty":"integer"}. So these tests write the integer + // production would have written, rather than a Date that jest mangles. + const managed = (iso) => new Date(iso).getTime(); + + const addPhoto = async (filename, capturedAt, uploadedAt) => { + const row = await db('photos').insert({ + event_id: eventId, + filename, + path: `${SLUG}/${filename}`, + type: 'individual', + captured_at: capturedAt, + uploaded_at: uploadedAt, + }).returning('id'); + return row[0]?.id ?? row[0]; + }; + + const orderedFilenames = async (order = 'asc') => { + const res = await request(app).get(`/api/gallery/${SLUG}/photos?sort=capture_date&order=${order}`); + expect(res.status).toBe(200); + return res.body.photos.map((p) => p.filename); + }; + + beforeAll(async () => { + ({ db, cleanup } = await bootCrmDb()); + await seedMinimal(db); + + const ev = await db('events').insert({ + slug: SLUG, + event_type: 'wedding', + event_name: 'Capture Sort', + event_date: '2026-08-01', + host_email: 'h@example.com', + admin_email: 'a@example.com', + password_hash: 'x', + share_link: `/gallery/${SLUG}/s`, + share_token: 'capsort-share', + expires_at: new Date(Date.now() + 7 * 864e5).toISOString(), + is_active: 1, + is_archived: 0, + is_draft: 0, + require_password: 0, + created_at: new Date().toISOString(), + }).returning('id'); + eventId = ev[0]?.id ?? ev[0]; + + app = express(); + app.use(express.json()); + app.use(cookieParser()); + app.use('/api/gallery', require('../../src/routes/gallery')); + }, 120000); + + afterAll(async () => { if (cleanup) await cleanup(); }); + + beforeEach(async () => { await db('photos').where({ event_id: eventId }).del(); }); + + test('the fixture really does put three storage classes in one column', async () => { + expect(['sqlite3', 'better-sqlite3']).toContain(db.client.config.client); + await addPhoto('m.jpg', managed('2026-06-03T01:15:00Z'), '2026-01-01 00:00:00'); + await addPhoto('e.jpg', '2020-01-01T00:00:00.000Z', '2026-01-01 00:00:00'); + await addPhoto('n.jpg', null, '2026-01-01 00:00:00'); + + const rows = await db.raw('select filename, typeof(captured_at) as t from photos order by filename'); + const byName = Object.fromEntries((rows.rows || rows).map((r) => [r.filename, r.t])); + // Exactly the mixture that made COALESCE meaningless. + expect(byName).toEqual({ 'm.jpg': 'integer', 'e.jpg': 'text', 'n.jpg': 'null' }); + }); + + test('a managed EXIF date does not outrank an earlier one stored as text', async () => { + // The pre-existing failure, reachable with managed photos alone: integer + // beat text regardless of the dates, so this came back exactly reversed. + await addPhoto('managed-2027.jpg', managed('2027-12-31T00:00:00Z'), '2026-01-01 00:00:00'); + await addPhoto('external-2020.jpg', '2020-01-01T00:00:00.000Z', '2026-01-01 00:00:00'); + + expect(await orderedFilenames('asc')).toEqual(['external-2020.jpg', 'managed-2027.jpg']); + expect(await orderedFilenames('desc')).toEqual(['managed-2027.jpg', 'external-2020.jpg']); + }); + + test('a photo with no capture date sorts by its upload time, not ahead of everything', async () => { + await addPhoto('has-exif-2027.jpg', managed('2027-12-31T00:00:00Z'), '2027-12-31 00:00:00'); + await addPhoto('no-exif-2020.jpg', null, '2020-01-01 00:00:00'); + + expect(await orderedFilenames('asc')).toEqual(['no-exif-2020.jpg', 'has-exif-2027.jpg']); + }); + + test('an ISO capture time and a fallback upload time compare by clock, not by separator', async () => { + // Same day: 'T' vs ' ' decided this before, so 01:15 sorted after 23:00. + await addPhoto('iso-0115.jpg', '2026-06-03T01:15:00.000Z', '2026-06-03 05:00:00'); + await addPhoto('fallback-2300.jpg', null, '2026-06-03 23:00:00'); + + expect(await orderedFilenames('asc')).toEqual(['iso-0115.jpg', 'fallback-2300.jpg']); + }); + + test('an epoch-integer uploaded_at is compared as a date, not as its digits', async () => { + // uploaded_at is not always text either: a legacy archive restore leaves + // epoch milliseconds in it (__tests__/integration/sqliteEpochTimestamps.js). + // Reading that with substr() would have compared the string '1830297600000' + // against '2020-01-01 00:00:00', putting the 2028 row first. + await addPhoto('epoch-upload-2028.jpg', null, new Date('2028-01-01T00:00:00Z').getTime()); + await addPhoto('captured-2020.jpg', managed('2020-01-01T00:00:00Z'), '2020-01-01 00:00:00'); + + const [row] = await db.raw('select typeof(uploaded_at) as t from photos where filename = \'epoch-upload-2028.jpg\''); + expect((row.t || row).toString()).toBe('integer'); + + expect(await orderedFilenames('asc')).toEqual(['captured-2020.jpg', 'epoch-upload-2028.jpg']); + }); + + test('all three storage classes order together correctly', async () => { + await addPhoto('c-managed-2026-08.jpg', managed('2026-08-15T12:00:00Z'), '2026-09-01 00:00:00'); + await addPhoto('a-external-2026-06.jpg', '2026-06-03T01:15:00.000Z', '2026-09-01 00:00:00'); + await addPhoto('d-fallback-2026-09.jpg', null, '2026-09-01 00:00:00'); + await addPhoto('b-managed-2026-07.jpg', managed('2026-07-04T09:30:00Z'), '2026-09-01 00:00:00'); + + expect(await orderedFilenames('asc')).toEqual([ + 'a-external-2026-06.jpg', + 'b-managed-2026-07.jpg', + 'c-managed-2026-08.jpg', + 'd-fallback-2026-09.jpg', + ]); + }); +}); diff --git a/backend/src/routes/adminExternalMedia.js b/backend/src/routes/adminExternalMedia.js index c78bd037..c2b03b33 100644 --- a/backend/src/routes/adminExternalMedia.js +++ b/backend/src/routes/adminExternalMedia.js @@ -8,7 +8,7 @@ const { list, resolveExternalPath, getExternalMediaRoot } = require('../services const { db, logActivity } = require('../database/db'); const sharp = require('sharp'); const logger = require('../utils/logger'); -const { generateThumbnail } = require('../services/imageProcessor'); +const { generateThumbnail, extractCaptureDate } = require('../services/imageProcessor'); const { isUniqueViolation } = require('../utils/dbErrors'); const router = express.Router(); @@ -208,6 +208,27 @@ router.post('/events/:id/import-external', adminAuth, requirePermission('photos. logger.warn(`Could not extract dimensions for ${f.rel}: ${dimErr.message}`); } + // Capture date from EXIF (#1172). Managed uploads get this from + // photoProcessor, which external media never goes through — so + // captured_at stayed NULL for every externally imported photo, and the + // gallery's "Date Taken" sort silently degraded into import order via + // its COALESCE fallback. On a library imported in two batches that put + // the first days of a trip after the last ones. + // + // Read here because the file is already open a few lines above for the + // dimensions, so this costs one more read of the same source rather + // than a second pass over the mount. + // + // Best-effort, exactly like the dimensions: a source without EXIF, or + // one Sharp/exifr cannot parse, imports with captured_at NULL and + // falls back to uploaded_at as before. + let capturedAt = null; + try { + capturedAt = await extractCaptureDate(f.full); + } catch (dateErr) { + logger.warn(`Could not extract capture date for ${f.rel}: ${dateErr.message}`); + } + let inserted; try { inserted = await db('photos') @@ -222,7 +243,12 @@ router.post('/events/:id/import-external', adminAuth, requirePermission('photos. width, height, source_origin: 'external', - external_relpath: relFromRoot + external_relpath: relFromRoot, + // .toISOString() rather than the Date: inside jest, Dates handed + // to the sqlite3 binding land as the literal string + // "[object Object]" (see CLAUDE.md). Strings round-trip on both + // engines. + captured_at: capturedAt ? capturedAt.toISOString() : null }) .returning('id'); } catch (insertErr) { diff --git a/backend/src/routes/adminPhotoDimensions.js b/backend/src/routes/adminPhotoDimensions.js index 0c630a0d..5003e2dc 100644 --- a/backend/src/routes/adminPhotoDimensions.js +++ b/backend/src/routes/adminPhotoDimensions.js @@ -14,6 +14,14 @@ let repairProgress = { lastResult: null }; +// Same shape, separate state: the two jobs walk the same photos but read +// different things out of them, and one running must not block or report for +// the other. +let captureDateProgress = { + isRunning: false, + lastResult: null +}; + // Repair photo dimensions (background job) router.post('/repair-dimensions', adminAuth, requirePermission('photos.edit'), async (req, res) => { try { @@ -152,4 +160,235 @@ router.get('/repair-dimensions/status', adminAuth, requirePermission('photos.vie } }); +/** + * Backfill captured_at from EXIF (#1172). + * + * External imports never read EXIF before this release, so every + * externally-imported photo carries captured_at NULL — and the gallery's + * "Date Taken" sort falls back to uploaded_at, which on a bulk import is the + * import timestamp. A 5555-photo trip came back ordered by which folder was + * imported first. + * + * Deliberately an endpoint rather than a migration: the originals live on a + * mount that may be unavailable at upgrade time, reading 8000+ of them blocks + * the boot, and a run that found nothing needs to be repeatable once the mount + * is back. Same reasoning, and the same shape, as the dimension repair above — + * including resolvePhotoFilePath, which is what makes it work for external + * rows at all (the thumbnail regenerator resolves under + * storage/events/active/, which never exists for them). + */ +// system.manage, not photos.edit: this walks every event in the install and +// rewrites their metadata. photos.edit is held by the team_photographer preset +// (175_granular_permissions_and_presets.js:106), which exists precisely for a +// contributing shooter who should not be able to start a whole-library S3/NAS +// scan or touch another owner's photos. The dimension repair above had the same +// exposure and predates this; it is brought in line separately in #1182. +router.post('/repair-capture-dates', adminAuth, requirePermission('system.manage'), async (req, res) => { + try { + if (captureDateProgress.isRunning) { + return res.status(409).json({ error: 'Capture date backfill is already running' }); + } + // Claimed here, not after the candidate query: that query is an await, and + // two POSTs arriving inside it would both read isRunning === false and both + // start a pass over the same rows. Every early exit below has to release it + // again, hence the try/catch around the query. + captureDateProgress.isRunning = true; + captureDateProgress.lastResult = null; + + let photos; + try { + photos = await db('photos') + .join('events', 'photos.event_id', 'events.id') + .whereNull('photos.captured_at') + // Three markers, because no single one is reliable. fileWatcher's + // auto-import sets type='video' and a video/* mime but never + // media_type (fileWatcher.js:128-130), so those rows keep the 'image' + // default from migration 048 and a media_type-only filter queues them + // forever: extractCaptureDate returns null for a video, captured_at + // stays null, and every run picks it up again. + .where(function () { + this.where('photos.media_type', '!=', 'video').orWhereNull('photos.media_type'); + }) + .where(function () { + this.where('photos.type', '!=', 'video').orWhereNull('photos.type'); + }) + .where(function () { + this.whereNull('photos.mime_type').orWhere('photos.mime_type', 'not like', 'video/%'); + }) + // Archiving deletes the originals from storage but keeps the photos + // rows (archiveService.js:166,199). Those files are inside the zip and + // nothing here can read them, so including them would fail every row + // on every run and leave the button permanently lit. + .where(function () { + this.where('events.is_archived', false).orWhereNull('events.is_archived'); + }) + .select( + 'photos.id', 'photos.path', 'photos.filename', + 'photos.source_origin', 'photos.external_relpath', 'photos.event_id', + 'events.source_mode', 'events.external_path', 'events.slug' + ); + } catch (err) { + captureDateProgress.isRunning = false; + throw err; + } + + if (photos.length === 0) { + captureDateProgress.isRunning = false; + return res.json({ message: 'No photos need a capture date', count: 0 }); + } + + res.json({ + message: `Started backfilling capture dates for ${photos.length} photos`, + count: photos.length + }); + + setImmediate(async () => { + const { extractCaptureDate, withLocalCopy } = require('../services/imageProcessor'); + const { resolvePhotoStorageKey } = require('../services/photoResolver'); + let successCount = 0; + let missingCount = 0; + let errorCount = 0; + + for (const photo of photos) { + try { + const event = { source_mode: photo.source_mode, external_path: photo.external_path, slug: photo.slug }; + const isExternal = photo.source_origin === 'external' || photo.source_origin === 'reference'; + + // Two source shapes, same split the thumbnail regenerator uses + // (imageProcessor.js:391-420). External rows live on a local mount + // and are read directly; managed rows live behind the storage + // backend, which on an S3 install is not a filesystem at all — going + // through resolvePhotoFilePath there would build a STORAGE_PATH that + // holds nothing and fail every managed photo. + let captured; + if (isExternal) { + let fullPath; + try { + fullPath = resolvePhotoFilePath(event, photo); + } catch (err) { + logger.warn(`Photo ${photo.id} has no resolvable path, skipping capture date: ${err.message}`); + errorCount++; + continue; + } + + try { + await fs.access(fullPath); + } catch (err) { + logger.warn(`File not found for photo ${photo.id}: ${fullPath}`); + errorCount++; + continue; + } + + captured = await extractCaptureDate(fullPath); + } else { + let sourceKey; + try { + sourceKey = resolvePhotoStorageKey(event, photo); + } catch (err) { + logger.warn(`Photo ${photo.id} has no resolvable storage key, skipping capture date: ${err.message}`); + errorCount++; + continue; + } + + // In local-fs mode withLocalCopy hands back the resolved path + // without checking it exists, so the access probe stays. In S3 + // mode a missing object throws out of getToFile and lands in the + // outer catch — both end up counted as failures, which is what a + // missing original is. + captured = await withLocalCopy(sourceKey, async (localPath) => { + await fs.access(localPath); + return extractCaptureDate(localPath); + }); + } + + if (!captured) { + // No date recovered. Usually genuine — plenty of sources carry no + // EXIF — but extractCaptureDate also returns null when the file is + // unreadable as an image, so this bucket is "nothing to write", + // not "definitely has no EXIF". The failure counter above is the + // one that means the storage is broken. + missingCount++; + continue; + } + + // whereNull, not a blanket set: the job can run for a long time on a + // large library, and an import or a replacement finishing meanwhile + // has already written a date this pass would otherwise overwrite + // with the same-or-worse value. + const updated = await db('photos') + .where({ id: photo.id }) + .whereNull('captured_at') + .update({ captured_at: captured.toISOString() }); + if (updated) successCount++; + + if (successCount % 50 === 0 && successCount > 0) { + logger.info(`Capture date backfill progress: ${successCount} updated...`); + } + } catch (error) { + logger.error(`Error backfilling capture date for photo ${photo.id}:`, error); + errorCount++; + } + } + + captureDateProgress.isRunning = false; + captureDateProgress.lastResult = { success: successCount, noExif: missingCount, failed: errorCount }; + logger.info(`Capture date backfill complete: ${successCount} updated, ${missingCount} without EXIF, ${errorCount} errors`); + }); + } catch (error) { + logger.error('Error starting capture date backfill:', error); + res.status(500).json({ error: 'Failed to start capture date backfill' }); + } +}); + +// The same permission as the POST, not the read-only system.view. system.view +// and system.manage are independent grants, and StatusTab has no permission +// gate of its own — a successful status payload is what renders the card and +// its enabled button (StatusTab.tsx:637). Gating this on system.view therefore +// handed a system.view-only role a live button whose every click 403s with no +// error surfaced. Requiring system.manage makes the card appear only for +// someone who can actually run it, which is what this comment always claimed. +router.get('/repair-capture-dates/status', adminAuth, requirePermission('system.manage'), async (req, res) => { + try { + // Same scope as the job itself — counting archived photos here would show + // a permanent backlog the button can never clear. + const scoped = () => db('photos') + .join('events', 'photos.event_id', 'events.id') + .where(function () { + this.where('photos.media_type', '!=', 'video').orWhereNull('photos.media_type'); + }) + .where(function () { + this.where('photos.type', '!=', 'video').orWhereNull('photos.type'); + }) + .where(function () { + this.whereNull('photos.mime_type').orWhere('photos.mime_type', 'not like', 'video/%'); + }) + .where(function () { + this.where('events.is_archived', false).orWhereNull('events.is_archived'); + }); + + // One query, two aggregates. As two separate counts an import committing a + // dated photo between them could be counted by the second and not the + // first, so withCaptureDate came out larger than total and the card showed + // a negative backlog — with the button enabled to "fix" it. + const counts = await scoped() + .count('photos.id as total') + .count({ dated: db.raw('CASE WHEN photos.captured_at IS NOT NULL THEN 1 END') }) + .first(); + + const total = Number(counts.total); + const withCaptureDate = Number(counts.dated); + + res.json({ + total, + withCaptureDate, + withoutCaptureDate: total - withCaptureDate, + isRunning: captureDateProgress.isRunning, + lastResult: captureDateProgress.lastResult + }); + } catch (error) { + logger.error('Error fetching capture date backfill status:', error); + res.status(500).json({ error: 'Failed to fetch capture date backfill status' }); + } +}); + module.exports = router; diff --git a/backend/src/routes/gallery.js b/backend/src/routes/gallery.js index 9ece85cd..54ec4d19 100644 --- a/backend/src/routes/gallery.js +++ b/backend/src/routes/gallery.js @@ -674,15 +674,66 @@ router.get('/:slug/photos', verifyGalleryAccess, resolveGuest, async (req, res) photosQuery = photosQuery.where('photos.category_id', req.event.show_category_id); } - // Apply sort option + // Apply sort option. + // + // Every branch carries photos.id as a tiebreaker (#1172). Without one the + // order within a tie is whatever the engine happens to return, and ties are + // the normal case rather than the exception: a bulk import writes hundreds + // of rows inside the same second, so uploaded_at collapses — and with + // captured_at NULL the COALESCE below collapses onto it too. The visible + // symptom is a grid that reshuffles between page loads. id is insertion + // order, so it also makes the fallback ordering meaningful rather than + // arbitrary. if (sort === 'capture_date') { - // Sort by capture date, falling back to uploaded_at if capture date is null - photosQuery = photosQuery.orderByRaw('COALESCE(photos.captured_at, photos.uploaded_at) ' + sortOrder); + // Sort by capture date, falling back to uploaded_at if capture date is null. + // + // On SQLite that fallback cannot be a plain COALESCE, because the two + // columns do not hold one type. photos.captured_at ends up carrying three + // different storage classes: + // + // integer managed uploads — photoProcessor.js:488 writes a Date, which + // the sqlite3 binding stores as epoch milliseconds + // text external imports and the backfill, which write ISO-8601 + // ('2026-06-03T01:15:00.000Z') per the CLAUDE.md rule that + // Dates must not be handed to the binding in tests + // null no capture date, so the sort falls through to uploaded_at — + // usually text in knex's 'YYYY-MM-DD HH:MM:SS' default shape, + // but epoch milliseconds on rows written by a legacy archive + // restore (see __tests__/integration/sqliteEpochTimestamps.js), + // so that column needs the same two branches + // + // SQLite orders INTEGER before TEXT unconditionally, so every managed + // photo carrying EXIF sorted ahead of every photo that did not, whatever + // the actual dates — a 2027 capture landing before a 2020 one. Among the + // text values the 'T' separator (0x54) also outranks the space (0x20), so + // a same-day ISO 01:15 sorted after a fallback 23:00. + // + // Normalising in the ORDER BY rather than rewriting the column: the data + // fix would have to touch every existing row and every writer, which is a + // much heavier change than the sort it is meant to correct. The cost here + // is that this sort stops using idx_photos_captured_at on SQLite — an + // acceptable trade on the fallback engine, where the alternative is an + // index-assisted wrong answer. + // + // Postgres is untouched: captured_at is a real timestamp there, so + // COALESCE already compares correctly. + if (db.client.config.client === 'pg') { + photosQuery = photosQuery + .orderByRaw('COALESCE(photos.captured_at, photos.uploaded_at) ' + sortOrder); + } else { + photosQuery = photosQuery.orderByRaw(`CASE + WHEN typeof(photos.captured_at) IN ('integer', 'real') THEN datetime(photos.captured_at / 1000, 'unixepoch') + WHEN photos.captured_at IS NOT NULL THEN replace(replace(substr(photos.captured_at, 1, 19), 'T', ' '), 'Z', '') + WHEN typeof(photos.uploaded_at) IN ('integer', 'real') THEN datetime(photos.uploaded_at / 1000, 'unixepoch') + ELSE substr(photos.uploaded_at, 1, 19) + END ${sortOrder}`); + } + photosQuery = photosQuery.orderBy('photos.id', sortOrder); } else if (sort === 'filename') { - photosQuery = photosQuery.orderBy('photos.filename', sortOrder); + photosQuery = photosQuery.orderBy('photos.filename', sortOrder).orderBy('photos.id', sortOrder); } else { // Default: sort by upload date - photosQuery = photosQuery.orderBy('photos.uploaded_at', sortOrder); + photosQuery = photosQuery.orderBy('photos.uploaded_at', sortOrder).orderBy('photos.id', sortOrder); } // Reveal mode (#838): while the gallery is hidden, plain guests get diff --git a/frontend/src/features/settings/tabs/StatusTab.tsx b/frontend/src/features/settings/tabs/StatusTab.tsx index d37b9cc0..6f2d630a 100644 --- a/frontend/src/features/settings/tabs/StatusTab.tsx +++ b/frontend/src/features/settings/tabs/StatusTab.tsx @@ -8,6 +8,7 @@ import { HardDrive, Activity, Ruler, + CalendarClock, } from 'lucide-react'; import { Button, Card, Input } from '../../../components/common'; import { useTranslation } from 'react-i18next'; @@ -17,6 +18,7 @@ import { settingsService } from '../../../services/settings.service'; import { useStatusTab } from '../hooks/useStatusTab'; import { UpdateNotificationSettings } from '../components/UpdateNotificationSettings'; import { useLocalizedDate } from '../../../hooks/useLocalizedDate'; +import { usePermission } from '../../../hooks/usePermission'; const BYTES_PER_GB = 1024 * 1024 * 1024; @@ -80,6 +82,35 @@ export const StatusTab: React.FC = ({ }, }); + // Capture dates (#1172). Same shape as the dimension repair above — a + // background pass over originals that resolves external rows properly — so + // it gets the same status/poll/mutation treatment. + // Gated on the same permission the endpoint requires, so a role without it + // never starts the poll. Without this the card would poll a 403 every ten + // seconds for anyone who can open the Status tab but cannot run the job, + // filling the logs with denials for a panel they were never shown. + const canManageSystem = usePermission('system.manage'); + + const { data: captureDateStatus } = useQuery({ + queryKey: ['photo-capture-date-status'], + queryFn: async () => { + const res = await api.get('/admin/photos/repair-capture-dates/status'); + return res.data; + }, + enabled: isActive && canManageSystem, + refetchInterval: 10000, + }); + + const captureDateMutation = useMutation({ + mutationFn: async () => { + const res = await api.post('/admin/photos/repair-capture-dates'); + return res.data; + }, + onSuccess: () => { + queryClient.invalidateQueries({ queryKey: ['photo-capture-date-status'] }); + }, + }); + // Sync soft limit from storage info useEffect(() => { if (!storageInfo || softLimitDirty) return; @@ -609,6 +640,73 @@ export const StatusTab: React.FC = ({ )} + {/* Capture Dates (#1172) */} + {/* canManageSystem as well as the payload: TanStack keeps the cached + status after `enabled` flips false, so without it a lower-privileged + admin logging in behind a system.manage user inside the cache lifetime + would still be shown the card and a button whose POST 403s. */} + {captureDateStatus && canManageSystem && ( + +

+ + {t('settings.captureDates.title', 'Capture Dates')} +

+ +

+ {t('settings.captureDates.description', 'Backfill "Date Taken" from EXIF for photos imported before capture dates were read. External/reference imports never recorded one, so their galleries sort by import order instead of when the photos were taken.')} +

+ +
+
+

{captureDateStatus.total}

+

{t('settings.captureDates.totalPhotos', 'Total Photos')}

+
+
+

{captureDateStatus.withCaptureDate}

+

{t('settings.captureDates.withDates', 'With Capture Date')}

+
+
0 ? 'bg-amber-50 dark:bg-amber-900/30' : 'bg-neutral-50 dark:bg-neutral-800'}`}> +

0 ? 'text-amber-600 dark:text-amber-400' : 'text-neutral-900 dark:text-neutral-100'}`}>{captureDateStatus.withoutCaptureDate}

+

{t('settings.captureDates.missingDates', 'Missing Capture Date')}

+
+
+ + {captureDateStatus.lastResult && ( +

+ {/* Two buckets on purpose: "the mount is gone" and "these files + carry no date" need different reactions. The middle number is + worded as "no date found" rather than "without EXIF" because + it also absorbs files whose metadata could not be parsed — + extractCaptureDate returns null for those too. The last number + is the one that means the storage could not be reached. */} + {t('settings.captureDates.resultSuccess', { + success: captureDateStatus.lastResult.success, + noExif: captureDateStatus.lastResult.noExif, + failed: captureDateStatus.lastResult.failed, + defaultValue: 'Last run: {{success}} updated, {{noExif}} with no date found, {{failed}} unreachable', + })} +

+ )} + +
+ +
+ + )} + {/* Update Notification Settings */} diff --git a/frontend/src/i18n/locales/de.json b/frontend/src/i18n/locales/de.json index fa8c4711..0924eda1 100644 --- a/frontend/src/i18n/locales/de.json +++ b/frontend/src/i18n/locales/de.json @@ -2352,6 +2352,17 @@ "inheritWith": "Erben ({{value}})", "effective": "Gibt derzeit heraus: {{standard}}", "pickerOn": "Gäste dürfen eine andere Größe wählen" + }, + "captureDates": { + "title": "Aufnahmedaten", + "totalPhotos": "Fotos gesamt", + "withDates": "Mit Aufnahmedatum", + "missingDates": "Ohne Aufnahmedatum", + "button": "Aufnahmedaten nachtragen", + "running": "Wird nachgetragen...", + "noneToFill": "Alle Fotos haben bereits ein Aufnahmedatum", + "resultSuccess": "Letzter Lauf: {{success}} aktualisiert, {{noExif}} ohne gefundenes Datum, {{failed}} nicht erreichbar", + "description": "Trägt „Aufnahmedatum\" aus den EXIF-Daten nach, für Fotos die vor dieser Auswertung importiert wurden. Externe Importe haben nie eines gespeichert, dadurch sortieren diese Galerien nach Importreihenfolge statt nach Aufnahmezeit." } }, "branding": { diff --git a/frontend/src/i18n/locales/en.json b/frontend/src/i18n/locales/en.json index 05f545ee..f93bd93e 100644 --- a/frontend/src/i18n/locales/en.json +++ b/frontend/src/i18n/locales/en.json @@ -1893,6 +1893,17 @@ "inheritWith": "Inherit ({{value}})", "effective": "Currently hands out: {{standard}}", "pickerOn": "guests may choose another size" + }, + "captureDates": { + "title": "Capture Dates", + "totalPhotos": "Total Photos", + "withDates": "With Capture Date", + "missingDates": "Missing Capture Date", + "button": "Backfill Capture Dates", + "running": "Backfilling...", + "noneToFill": "All photos already have a capture date", + "resultSuccess": "Last run: {{success}} updated, {{noExif}} with no date found, {{failed}} unreachable", + "description": "Backfill \"Date Taken\" from EXIF for photos imported before capture dates were read. External/reference imports never recorded one, so their galleries sort by import order instead of when the photos were taken." } }, "analytics": { diff --git a/frontend/src/i18n/locales/fr.json b/frontend/src/i18n/locales/fr.json index 1d9a7baf..f9b1c4c6 100644 --- a/frontend/src/i18n/locales/fr.json +++ b/frontend/src/i18n/locales/fr.json @@ -1201,6 +1201,17 @@ "save": "Enregistrer les modifications", "saved": "Marque du tableau de bord client enregistrée", "error": "Impossible d'enregistrer les paramètres" + }, + "captureDates": { + "title": "Dates de prise de vue", + "totalPhotos": "Photos au total", + "withDates": "Avec date de prise de vue", + "missingDates": "Sans date de prise de vue", + "button": "Compléter les dates", + "running": "Traitement...", + "noneToFill": "Toutes les photos ont déjà une date de prise de vue", + "resultSuccess": "Dernier passage : {{success}} mises à jour, {{noExif}} sans date trouvée, {{failed}} inaccessibles", + "description": "Complète la « date de prise de vue » depuis les EXIF pour les photos importées avant sa lecture. Les imports externes n'en enregistraient aucune, si bien que ces galeries se trient par ordre d'import plutôt que par date de prise de vue." } }, "analytics": { diff --git a/frontend/src/i18n/locales/sl.json b/frontend/src/i18n/locales/sl.json index 1f441de2..90ade1d2 100644 --- a/frontend/src/i18n/locales/sl.json +++ b/frontend/src/i18n/locales/sl.json @@ -1201,6 +1201,17 @@ "save": "Shrani spremembe", "saved": "Blagovna znamka nadzorne plošče stranke je shranjena", "error": "Nastavitev ni bilo mogoče shraniti" + }, + "captureDates": { + "title": "Datumi zajema", + "totalPhotos": "Skupaj fotografij", + "withDates": "Z datumom zajema", + "missingDates": "Brez datuma zajema", + "button": "Dopolni datume zajema", + "running": "Dopolnjevanje...", + "noneToFill": "Vse fotografije že imajo datum zajema", + "resultSuccess": "Zadnji zagon: {{success}} posodobljenih, {{noExif}} brez najdenega datuma, {{failed}} nedosegljivih", + "description": "Dopolni »datum zajema« iz EXIF za fotografije, uvožene pred njegovim branjem. Zunanji uvozi ga niso zabeležili, zato se te galerije razvrščajo po vrstnem redu uvoza namesto po času zajema." } }, "analytics": {