fix(external-media): record captured_at on import and add a backfill (stable) (#1183)
* fix(external-media): record captured_at on import and add a backfill (#1172) Stable twin of #1179. External media never went through photoProcessor, so captured_at stayed NULL for every externally imported photo. The gallery's "Date Taken" sort then degraded into import order through its own COALESCE fallback — a library imported in two batches showed the first days of a trip after the last ones. Ported whole: - adminExternalMedia.js reads the capture date at import, off the file it has already opened for the dimensions. Best-effort, like the dimensions. - A backfill endpoint for photos imported before this, so existing installs can fix historical rows rather than only new imports. Managed originals go through resolvePhotoStorageKey + withLocalCopy so S3 installs work; archived events are excluded because archiving deletes their originals; the run flag is claimed before the candidate query so two POSTs cannot both start. - gallery.js carries photos.id as a tiebreaker on all three sorts. A bulk import writes hundreds of rows inside the same second, so uploaded_at ties are the normal case and the grid reshuffled between page loads. One deliberate difference from main: the backfill is gated on settings.edit / settings.view rather than system.manage / system.view, which do not exist on this branch. They are what settings.edit was later split into, and main's migration 175 projects every settings.edit holder forward onto system.manage, so both branches let exactly the same people through. * fix(external-media): gate the status card on the permission the button needs (#1172) The built-in admin role holds settings.view but not settings.edit (056_add_role_permissions_table.js:63), and StatusTab renders its card and enabled button purely on a successful status payload. Gating the status endpoint on settings.view therefore showed every admin a Backfill button whose every click 403s with no error surfaced. * fix(gallery): make the Date Taken sort correct on SQLite (#1172) Same defect as the main twin: photos.captured_at holds three storage classes on SQLite — an epoch-millisecond integer from managed uploads (photoProcessor.js:441 hands knex a Date), ISO text from external imports and the backfill, and null falling through to uploaded_at's 'YYYY-MM-DD HH:MM:SS' text. SQLite sorts INTEGER before TEXT unconditionally, so a 2027 capture came back before a 2020 one, and within the text values the 'T' separator outranked the space. Normalised in the ORDER BY; Postgres keeps the plain COALESCE, its column being a real timestamp. Regression tests drive the real gallery route on real SQLite. * fix(gallery): normalise epoch-integer uploaded_at too, and stop polling a 403 (#1172) Both follow-ups from the main twin's review, ported. uploaded_at is not always text on SQLite: a .picpeak restore can carry epoch milliseconds in from an install that stored them that way, and the fallback branch read it with substr(), comparing '1830297600000' against '2020-01-01 00:00:00' as text. Both columns now get the integer/real branch. The status card also polled every ten seconds regardless of permission. On this branch that hits every built-in admin — they hold settings.view but not settings.edit — so each would have had a 403 and a logged denial every ten seconds for a panel they were never shown. * style: quote convention in the capture-sort test (#1172) * fix(capture-dates): skip watcher-imported videos, and make the status counts consistent (#1172) All three follow-ups from the main twin, ported: the three-marker video filter (fileWatcher sets type/mime but not media_type, so those rows sat in the backlog forever), the single-aggregate status counts (two queries could report a negative backlog mid-import), and the card's render gated on settings.edit as well as the cached payload. --------- Co-authored-by: Paul Nothaft <[email protected]>
This commit is contained in:
Paul Nothaft
co-authored by
Paul Nothaft
Paul Nothaft
parent
2b1c3588ae
commit
7f0ed23ea4
@@ -8,6 +8,7 @@ import {
|
||||
HardDrive,
|
||||
Activity,
|
||||
Ruler,
|
||||
CalendarClock,
|
||||
} from 'lucide-react';
|
||||
import { Button, Card, Input } from '../../../components/common';
|
||||
import { useTranslation } from 'react-i18next';
|
||||
@@ -17,6 +18,7 @@ import { settingsService } from '../../../services/settings.service';
|
||||
import { useStatusTab } from '../hooks/useStatusTab';
|
||||
import { UpdateNotificationSettings } from '../components/UpdateNotificationSettings';
|
||||
import { useLocalizedDate } from '../../../hooks/useLocalizedDate';
|
||||
import { usePermission } from '../../../hooks/usePermission';
|
||||
|
||||
const BYTES_PER_GB = 1024 * 1024 * 1024;
|
||||
|
||||
@@ -80,6 +82,36 @@ export const StatusTab: React.FC<StatusTabProps> = ({
|
||||
},
|
||||
});
|
||||
|
||||
// Capture dates (#1172). Same shape as the dimension repair above — a
|
||||
// background pass over originals that resolves external rows properly — so
|
||||
// it gets the same status/poll/mutation treatment.
|
||||
// Gated on the same permission the endpoint requires, so a role without it
|
||||
// never starts the poll. Without this the card would poll a 403 every ten
|
||||
// seconds for anyone who can open the Status tab but cannot run the job —
|
||||
// which on this branch is every built-in admin, since they hold settings.view
|
||||
// but not settings.edit.
|
||||
const canEditSettings = usePermission('settings.edit');
|
||||
|
||||
const { data: captureDateStatus } = useQuery({
|
||||
queryKey: ['photo-capture-date-status'],
|
||||
queryFn: async () => {
|
||||
const res = await api.get('/admin/photos/repair-capture-dates/status');
|
||||
return res.data;
|
||||
},
|
||||
enabled: isActive && canEditSettings,
|
||||
refetchInterval: 10000,
|
||||
});
|
||||
|
||||
const captureDateMutation = useMutation({
|
||||
mutationFn: async () => {
|
||||
const res = await api.post('/admin/photos/repair-capture-dates');
|
||||
return res.data;
|
||||
},
|
||||
onSuccess: () => {
|
||||
queryClient.invalidateQueries({ queryKey: ['photo-capture-date-status'] });
|
||||
},
|
||||
});
|
||||
|
||||
// Sync soft limit from storage info
|
||||
useEffect(() => {
|
||||
if (!storageInfo || softLimitDirty) return;
|
||||
@@ -609,6 +641,73 @@ export const StatusTab: React.FC<StatusTabProps> = ({
|
||||
</Card>
|
||||
)}
|
||||
|
||||
{/* Capture Dates (#1172) */}
|
||||
{/* canEditSettings as well as the payload: TanStack keeps the cached
|
||||
status after `enabled` flips false, so without it a built-in admin
|
||||
logging in behind a settings.edit user inside the cache lifetime would
|
||||
still be shown the card and a button whose POST 403s. */}
|
||||
{captureDateStatus && canEditSettings && (
|
||||
<Card padding="md">
|
||||
<h2 className="text-lg font-semibold text-neutral-900 dark:text-neutral-100 mb-4 flex items-center gap-2">
|
||||
<CalendarClock className="w-5 h-5" />
|
||||
{t('settings.captureDates.title', 'Capture Dates')}
|
||||
</h2>
|
||||
|
||||
<p className="text-sm text-neutral-600 dark:text-neutral-400 mb-4">
|
||||
{t('settings.captureDates.description', 'Backfill "Date Taken" from EXIF for photos imported before capture dates were read. External/reference imports never recorded one, so their galleries sort by import order instead of when the photos were taken.')}
|
||||
</p>
|
||||
|
||||
<div className="grid grid-cols-1 md:grid-cols-3 gap-4 mb-4">
|
||||
<div className="bg-neutral-50 dark:bg-neutral-800 rounded-lg p-3 text-center">
|
||||
<p className="text-2xl font-bold text-neutral-900 dark:text-neutral-100">{captureDateStatus.total}</p>
|
||||
<p className="text-xs text-neutral-600 dark:text-neutral-400">{t('settings.captureDates.totalPhotos', 'Total Photos')}</p>
|
||||
</div>
|
||||
<div className="bg-neutral-50 dark:bg-neutral-800 rounded-lg p-3 text-center">
|
||||
<p className="text-2xl font-bold text-green-600 dark:text-green-400">{captureDateStatus.withCaptureDate}</p>
|
||||
<p className="text-xs text-neutral-600 dark:text-neutral-400">{t('settings.captureDates.withDates', 'With Capture Date')}</p>
|
||||
</div>
|
||||
<div className={`rounded-lg p-3 text-center ${Number(captureDateStatus.withoutCaptureDate) > 0 ? 'bg-amber-50 dark:bg-amber-900/30' : 'bg-neutral-50 dark:bg-neutral-800'}`}>
|
||||
<p className={`text-2xl font-bold ${Number(captureDateStatus.withoutCaptureDate) > 0 ? 'text-amber-600 dark:text-amber-400' : 'text-neutral-900 dark:text-neutral-100'}`}>{captureDateStatus.withoutCaptureDate}</p>
|
||||
<p className="text-xs text-neutral-600 dark:text-neutral-400">{t('settings.captureDates.missingDates', 'Missing Capture Date')}</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{captureDateStatus.lastResult && (
|
||||
<p className="text-sm text-neutral-600 dark:text-neutral-400 mb-4">
|
||||
{/* Two buckets on purpose: "the mount is gone" and "these files
|
||||
carry no date" need different reactions. The middle number is
|
||||
worded as "no date found" rather than "without EXIF" because
|
||||
it also absorbs files whose metadata could not be parsed —
|
||||
extractCaptureDate returns null for those too. The last number
|
||||
is the one that means the storage could not be reached. */}
|
||||
{t('settings.captureDates.resultSuccess', {
|
||||
success: captureDateStatus.lastResult.success,
|
||||
noExif: captureDateStatus.lastResult.noExif,
|
||||
failed: captureDateStatus.lastResult.failed,
|
||||
defaultValue: 'Last run: {{success}} updated, {{noExif}} with no date found, {{failed}} unreachable',
|
||||
})}
|
||||
</p>
|
||||
)}
|
||||
|
||||
<div className="flex justify-end">
|
||||
<Button
|
||||
variant="secondary"
|
||||
size="sm"
|
||||
onClick={() => captureDateMutation.mutate()}
|
||||
isLoading={captureDateMutation.isPending || captureDateStatus.isRunning}
|
||||
disabled={Number(captureDateStatus.withoutCaptureDate) === 0 || captureDateStatus.isRunning}
|
||||
leftIcon={<CalendarClock className="w-4 h-4" />}
|
||||
>
|
||||
{captureDateStatus.isRunning
|
||||
? t('settings.captureDates.running', 'Backfilling...')
|
||||
: Number(captureDateStatus.withoutCaptureDate) === 0
|
||||
? t('settings.captureDates.noneToFill', 'All photos already have a capture date')
|
||||
: t('settings.captureDates.button', 'Backfill Capture Dates')}
|
||||
</Button>
|
||||
</div>
|
||||
</Card>
|
||||
)}
|
||||
|
||||
{/* Update Notification Settings */}
|
||||
<UpdateNotificationSettings />
|
||||
|
||||
|
||||
@@ -2067,6 +2067,17 @@
|
||||
"enableFailed": "Zwei-Faktor-Authentifizierung konnte nicht aktiviert werden. Prüfe den Code und versuche es erneut.",
|
||||
"disableFailed": "Zwei-Faktor-Authentifizierung konnte nicht deaktiviert werden. Prüfe den Code und versuche es erneut.",
|
||||
"regenerateFailed": "Wiederherstellungscodes konnten nicht neu erzeugt werden. Prüfe den Code und versuche es erneut."
|
||||
},
|
||||
"captureDates": {
|
||||
"title": "Aufnahmedaten",
|
||||
"totalPhotos": "Fotos gesamt",
|
||||
"withDates": "Mit Aufnahmedatum",
|
||||
"missingDates": "Ohne Aufnahmedatum",
|
||||
"button": "Aufnahmedaten nachtragen",
|
||||
"running": "Wird nachgetragen...",
|
||||
"noneToFill": "Alle Fotos haben bereits ein Aufnahmedatum",
|
||||
"resultSuccess": "Letzter Lauf: {{success}} aktualisiert, {{noExif}} ohne gefundenes Datum, {{failed}} nicht erreichbar",
|
||||
"description": "Trägt „Aufnahmedatum\" aus den EXIF-Daten nach, für Fotos die vor dieser Auswertung importiert wurden. Externe Importe haben nie eines gespeichert, dadurch sortieren diese Galerien nach Importreihenfolge statt nach Aufnahmezeit."
|
||||
}
|
||||
},
|
||||
"branding": {
|
||||
|
||||
@@ -1608,6 +1608,17 @@
|
||||
"enableFailed": "Could not enable two-factor authentication. Check the code and try again.",
|
||||
"disableFailed": "Could not disable two-factor authentication. Check the code and try again.",
|
||||
"regenerateFailed": "Could not regenerate recovery codes. Check the code and try again."
|
||||
},
|
||||
"captureDates": {
|
||||
"title": "Capture Dates",
|
||||
"totalPhotos": "Total Photos",
|
||||
"withDates": "With Capture Date",
|
||||
"missingDates": "Missing Capture Date",
|
||||
"button": "Backfill Capture Dates",
|
||||
"running": "Backfilling...",
|
||||
"noneToFill": "All photos already have a capture date",
|
||||
"resultSuccess": "Last run: {{success}} updated, {{noExif}} with no date found, {{failed}} unreachable",
|
||||
"description": "Backfill \"Date Taken\" from EXIF for photos imported before capture dates were read. External/reference imports never recorded one, so their galleries sort by import order instead of when the photos were taken."
|
||||
}
|
||||
},
|
||||
"analytics": {
|
||||
|
||||
@@ -1171,6 +1171,17 @@
|
||||
"save": "Enregistrer les modifications",
|
||||
"saved": "Marque du tableau de bord client enregistrée",
|
||||
"error": "Impossible d'enregistrer les paramètres"
|
||||
},
|
||||
"captureDates": {
|
||||
"title": "Dates de prise de vue",
|
||||
"totalPhotos": "Photos au total",
|
||||
"withDates": "Avec date de prise de vue",
|
||||
"missingDates": "Sans date de prise de vue",
|
||||
"button": "Compléter les dates",
|
||||
"running": "Traitement...",
|
||||
"noneToFill": "Toutes les photos ont déjà une date de prise de vue",
|
||||
"resultSuccess": "Dernier passage : {{success}} mises à jour, {{noExif}} sans date trouvée, {{failed}} inaccessibles",
|
||||
"description": "Complète la « date de prise de vue » depuis les EXIF pour les photos importées avant sa lecture. Les imports externes n'en enregistraient aucune, si bien que ces galeries se trient par ordre d'import plutôt que par date de prise de vue."
|
||||
}
|
||||
},
|
||||
"analytics": {
|
||||
|
||||
@@ -1171,6 +1171,17 @@
|
||||
"save": "Shrani spremembe",
|
||||
"saved": "Blagovna znamka nadzorne plošče stranke je shranjena",
|
||||
"error": "Nastavitev ni bilo mogoče shraniti"
|
||||
},
|
||||
"captureDates": {
|
||||
"title": "Datumi zajema",
|
||||
"totalPhotos": "Skupaj fotografij",
|
||||
"withDates": "Z datumom zajema",
|
||||
"missingDates": "Brez datuma zajema",
|
||||
"button": "Dopolni datume zajema",
|
||||
"running": "Dopolnjevanje...",
|
||||
"noneToFill": "Vse fotografije že imajo datum zajema",
|
||||
"resultSuccess": "Zadnji zagon: {{success}} posodobljenih, {{noExif}} brez najdenega datuma, {{failed}} nedosegljivih",
|
||||
"description": "Dopolni »datum zajema« iz EXIF za fotografije, uvožene pred njegovim branjem. Zunanji uvozi ga niso zabeležili, zato se te galerije razvrščajo po vrstnem redu uvoza namesto po času zajema."
|
||||
}
|
||||
},
|
||||
"analytics": {
|
||||
|
||||
Reference in New Issue
Block a user