fix(external-media): record captured_at on import and add a backfill (stable) (#1183)
* fix(external-media): record captured_at on import and add a backfill (#1172) Stable twin of #1179. External media never went through photoProcessor, so captured_at stayed NULL for every externally imported photo. The gallery's "Date Taken" sort then degraded into import order through its own COALESCE fallback — a library imported in two batches showed the first days of a trip after the last ones. Ported whole: - adminExternalMedia.js reads the capture date at import, off the file it has already opened for the dimensions. Best-effort, like the dimensions. - A backfill endpoint for photos imported before this, so existing installs can fix historical rows rather than only new imports. Managed originals go through resolvePhotoStorageKey + withLocalCopy so S3 installs work; archived events are excluded because archiving deletes their originals; the run flag is claimed before the candidate query so two POSTs cannot both start. - gallery.js carries photos.id as a tiebreaker on all three sorts. A bulk import writes hundreds of rows inside the same second, so uploaded_at ties are the normal case and the grid reshuffled between page loads. One deliberate difference from main: the backfill is gated on settings.edit / settings.view rather than system.manage / system.view, which do not exist on this branch. They are what settings.edit was later split into, and main's migration 175 projects every settings.edit holder forward onto system.manage, so both branches let exactly the same people through. * fix(external-media): gate the status card on the permission the button needs (#1172) The built-in admin role holds settings.view but not settings.edit (056_add_role_permissions_table.js:63), and StatusTab renders its card and enabled button purely on a successful status payload. Gating the status endpoint on settings.view therefore showed every admin a Backfill button whose every click 403s with no error surfaced. * fix(gallery): make the Date Taken sort correct on SQLite (#1172) Same defect as the main twin: photos.captured_at holds three storage classes on SQLite — an epoch-millisecond integer from managed uploads (photoProcessor.js:441 hands knex a Date), ISO text from external imports and the backfill, and null falling through to uploaded_at's 'YYYY-MM-DD HH:MM:SS' text. SQLite sorts INTEGER before TEXT unconditionally, so a 2027 capture came back before a 2020 one, and within the text values the 'T' separator outranked the space. Normalised in the ORDER BY; Postgres keeps the plain COALESCE, its column being a real timestamp. Regression tests drive the real gallery route on real SQLite. * fix(gallery): normalise epoch-integer uploaded_at too, and stop polling a 403 (#1172) Both follow-ups from the main twin's review, ported. uploaded_at is not always text on SQLite: a .picpeak restore can carry epoch milliseconds in from an install that stored them that way, and the fallback branch read it with substr(), comparing '1830297600000' against '2020-01-01 00:00:00' as text. Both columns now get the integer/real branch. The status card also polled every ten seconds regardless of permission. On this branch that hits every built-in admin — they hold settings.view but not settings.edit — so each would have had a 403 and a logged denial every ten seconds for a panel they were never shown. * style: quote convention in the capture-sort test (#1172) * fix(capture-dates): skip watcher-imported videos, and make the status counts consistent (#1172) All three follow-ups from the main twin, ported: the three-marker video filter (fileWatcher sets type/mime but not media_type, so those rows sat in the backlog forever), the single-aggregate status counts (two queries could report a negative backlog mid-import), and the card's render gated on settings.edit as well as the cached payload. --------- Co-authored-by: Paul Nothaft <[email protected]>
This commit is contained in:
co-authored by
Paul Nothaft
parent
2b1c3588ae
commit
7f0ed23ea4
@@ -8,6 +8,7 @@ import {
|
||||
HardDrive,
|
||||
Activity,
|
||||
Ruler,
|
||||
CalendarClock,
|
||||
} from 'lucide-react';
|
||||
import { Button, Card, Input } from '../../../components/common';
|
||||
import { useTranslation } from 'react-i18next';
|
||||
@@ -17,6 +18,7 @@ import { settingsService } from '../../../services/settings.service';
|
||||
import { useStatusTab } from '../hooks/useStatusTab';
|
||||
import { UpdateNotificationSettings } from '../components/UpdateNotificationSettings';
|
||||
import { useLocalizedDate } from '../../../hooks/useLocalizedDate';
|
||||
import { usePermission } from '../../../hooks/usePermission';
|
||||
|
||||
const BYTES_PER_GB = 1024 * 1024 * 1024;
|
||||
|
||||
@@ -80,6 +82,36 @@ export const StatusTab: React.FC<StatusTabProps> = ({
|
||||
},
|
||||
});
|
||||
|
||||
// Capture dates (#1172). Same shape as the dimension repair above — a
|
||||
// background pass over originals that resolves external rows properly — so
|
||||
// it gets the same status/poll/mutation treatment.
|
||||
// Gated on the same permission the endpoint requires, so a role without it
|
||||
// never starts the poll. Without this the card would poll a 403 every ten
|
||||
// seconds for anyone who can open the Status tab but cannot run the job —
|
||||
// which on this branch is every built-in admin, since they hold settings.view
|
||||
// but not settings.edit.
|
||||
const canEditSettings = usePermission('settings.edit');
|
||||
|
||||
const { data: captureDateStatus } = useQuery({
|
||||
queryKey: ['photo-capture-date-status'],
|
||||
queryFn: async () => {
|
||||
const res = await api.get('/admin/photos/repair-capture-dates/status');
|
||||
return res.data;
|
||||
},
|
||||
enabled: isActive && canEditSettings,
|
||||
refetchInterval: 10000,
|
||||
});
|
||||
|
||||
const captureDateMutation = useMutation({
|
||||
mutationFn: async () => {
|
||||
const res = await api.post('/admin/photos/repair-capture-dates');
|
||||
return res.data;
|
||||
},
|
||||
onSuccess: () => {
|
||||
queryClient.invalidateQueries({ queryKey: ['photo-capture-date-status'] });
|
||||
},
|
||||
});
|
||||
|
||||
// Sync soft limit from storage info
|
||||
useEffect(() => {
|
||||
if (!storageInfo || softLimitDirty) return;
|
||||
@@ -609,6 +641,73 @@ export const StatusTab: React.FC<StatusTabProps> = ({
|
||||
</Card>
|
||||
)}
|
||||
|
||||
{/* Capture Dates (#1172) */}
|
||||
{/* canEditSettings as well as the payload: TanStack keeps the cached
|
||||
status after `enabled` flips false, so without it a built-in admin
|
||||
logging in behind a settings.edit user inside the cache lifetime would
|
||||
still be shown the card and a button whose POST 403s. */}
|
||||
{captureDateStatus && canEditSettings && (
|
||||
<Card padding="md">
|
||||
<h2 className="text-lg font-semibold text-neutral-900 dark:text-neutral-100 mb-4 flex items-center gap-2">
|
||||
<CalendarClock className="w-5 h-5" />
|
||||
{t('settings.captureDates.title', 'Capture Dates')}
|
||||
</h2>
|
||||
|
||||
<p className="text-sm text-neutral-600 dark:text-neutral-400 mb-4">
|
||||
{t('settings.captureDates.description', 'Backfill "Date Taken" from EXIF for photos imported before capture dates were read. External/reference imports never recorded one, so their galleries sort by import order instead of when the photos were taken.')}
|
||||
</p>
|
||||
|
||||
<div className="grid grid-cols-1 md:grid-cols-3 gap-4 mb-4">
|
||||
<div className="bg-neutral-50 dark:bg-neutral-800 rounded-lg p-3 text-center">
|
||||
<p className="text-2xl font-bold text-neutral-900 dark:text-neutral-100">{captureDateStatus.total}</p>
|
||||
<p className="text-xs text-neutral-600 dark:text-neutral-400">{t('settings.captureDates.totalPhotos', 'Total Photos')}</p>
|
||||
</div>
|
||||
<div className="bg-neutral-50 dark:bg-neutral-800 rounded-lg p-3 text-center">
|
||||
<p className="text-2xl font-bold text-green-600 dark:text-green-400">{captureDateStatus.withCaptureDate}</p>
|
||||
<p className="text-xs text-neutral-600 dark:text-neutral-400">{t('settings.captureDates.withDates', 'With Capture Date')}</p>
|
||||
</div>
|
||||
<div className={`rounded-lg p-3 text-center ${Number(captureDateStatus.withoutCaptureDate) > 0 ? 'bg-amber-50 dark:bg-amber-900/30' : 'bg-neutral-50 dark:bg-neutral-800'}`}>
|
||||
<p className={`text-2xl font-bold ${Number(captureDateStatus.withoutCaptureDate) > 0 ? 'text-amber-600 dark:text-amber-400' : 'text-neutral-900 dark:text-neutral-100'}`}>{captureDateStatus.withoutCaptureDate}</p>
|
||||
<p className="text-xs text-neutral-600 dark:text-neutral-400">{t('settings.captureDates.missingDates', 'Missing Capture Date')}</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{captureDateStatus.lastResult && (
|
||||
<p className="text-sm text-neutral-600 dark:text-neutral-400 mb-4">
|
||||
{/* Two buckets on purpose: "the mount is gone" and "these files
|
||||
carry no date" need different reactions. The middle number is
|
||||
worded as "no date found" rather than "without EXIF" because
|
||||
it also absorbs files whose metadata could not be parsed —
|
||||
extractCaptureDate returns null for those too. The last number
|
||||
is the one that means the storage could not be reached. */}
|
||||
{t('settings.captureDates.resultSuccess', {
|
||||
success: captureDateStatus.lastResult.success,
|
||||
noExif: captureDateStatus.lastResult.noExif,
|
||||
failed: captureDateStatus.lastResult.failed,
|
||||
defaultValue: 'Last run: {{success}} updated, {{noExif}} with no date found, {{failed}} unreachable',
|
||||
})}
|
||||
</p>
|
||||
)}
|
||||
|
||||
<div className="flex justify-end">
|
||||
<Button
|
||||
variant="secondary"
|
||||
size="sm"
|
||||
onClick={() => captureDateMutation.mutate()}
|
||||
isLoading={captureDateMutation.isPending || captureDateStatus.isRunning}
|
||||
disabled={Number(captureDateStatus.withoutCaptureDate) === 0 || captureDateStatus.isRunning}
|
||||
leftIcon={<CalendarClock className="w-4 h-4" />}
|
||||
>
|
||||
{captureDateStatus.isRunning
|
||||
? t('settings.captureDates.running', 'Backfilling...')
|
||||
: Number(captureDateStatus.withoutCaptureDate) === 0
|
||||
? t('settings.captureDates.noneToFill', 'All photos already have a capture date')
|
||||
: t('settings.captureDates.button', 'Backfill Capture Dates')}
|
||||
</Button>
|
||||
</div>
|
||||
</Card>
|
||||
)}
|
||||
|
||||
{/* Update Notification Settings */}
|
||||
<UpdateNotificationSettings />
|
||||
|
||||
|
||||
Reference in New Issue
Block a user