feat(faces): consolidate look-alike clusters after a scan, and suggest the rest (#1107)
consolidate() has existed since #1074 and described this exact symptom in its own comment, but its only caller was recluster() — i.e. when an admin pressed Re-group people. After a normal background scan the centroids converged and nobody looked, so a gallery settled with 14 people that should have been 8. It now runs when a scan drains. There is no scan-finished event to hook, so an idle worker asks whether the events it touched have actually drained — 'a worker went idle' is deliberately not treated as sufficient, because with concurrency above one the others may still be working. The uncertain band asks instead of acting: pairs between the assignment threshold and the stricter auto-merge one surface as accept/dismiss suggestions, with sticky dismissals. Nothing merges silently — a pass that merged anything reports it and points at Split. Review rounds hardened it against overruling explicit decisions: it no longer absorbs ignored clusters (mergePeople ORs is_ignored onto the survivor, which would have hidden a real person), no longer merges dismissed pairs, no longer undoes a manual Split (which now records a separation), and no longer runs after detection is switched off. The dismissal read fails closed, a failed pass is retried with backoff rather than lost or hot-looped, and the new table follows event_people out of exports and backups. Name autocomplete needs no endpoint — the people list already open is the source, and it is event-scoped on purpose. Known limitation, tracked in #1132: separations are keyed on person ids, so a full re-scan loses them. Reported by @BraynArts.
This commit is contained in:
Paul Nothaft
@@ -29,6 +29,12 @@ interface FacesPayload {
|
||||
enabled: boolean;
|
||||
visible_to_guests: boolean;
|
||||
last_scan_at: string | null;
|
||||
// What the automatic consolidation pass merged when this gallery last
|
||||
// finished scanning (#1107).
|
||||
consolidation?: {
|
||||
merged: number;
|
||||
at: string | null;
|
||||
};
|
||||
status: {
|
||||
scanned: number;
|
||||
total: number;
|
||||
@@ -156,6 +162,29 @@ export const FaceRecognitionCard: React.FC<FaceRecognitionCardProps> = ({ eventI
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [healthAt, shouldProbe, scanRunning, health]);
|
||||
|
||||
// Consolidation happens AFTER the last photo is marked done: the worker
|
||||
// records the event, goes idle, then drains. So the poll that first sees
|
||||
// `in_progress: false` also stops polling, and it read the consolidation
|
||||
// count a moment too early — leaving the card silent about merges that did
|
||||
// happen until the admin navigates back.
|
||||
//
|
||||
// Two catch-up refetches rather than one, because a consolidation that hits
|
||||
// a transient error is retried by the queue a minute later
|
||||
// (FACE_CONSOLIDATE_RETRY_MS): the first covers the normal case, the second
|
||||
// covers one retry. Deliberately a fixed pair and not a poll — an event
|
||||
// whose photos all failed never consolidates at all, and a condition-based
|
||||
// poll would spin on it forever.
|
||||
const wasScanning = useRef(false);
|
||||
useEffect(() => {
|
||||
const scanning = !!data?.status?.in_progress;
|
||||
const justFinished = wasScanning.current && !scanning;
|
||||
wasScanning.current = scanning;
|
||||
if (!justFinished) return;
|
||||
|
||||
const timers = [8000, 70000].map((ms) => setTimeout(() => { refetch(); }, ms));
|
||||
return () => timers.forEach(clearTimeout);
|
||||
}, [data?.status?.in_progress, refetch]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!data?.enabled) return;
|
||||
api.get('/admin/events/faces/auto-categories')
|
||||
@@ -451,6 +480,24 @@ export const FaceRecognitionCard: React.FC<FaceRecognitionCardProps> = ({ eventI
|
||||
misconfiguration or from corrupt images at some earlier point.
|
||||
Attributing them properly would mean reading stored face_error
|
||||
rows — worth doing, but a bigger change than this. */}
|
||||
{/* Automatic consolidation (#1107). Clustering merged look-alike
|
||||
groups on its own once the scan drained, and doing that to
|
||||
biometric clusters without saying so is the wrong default —
|
||||
even at the stricter-than-assignment threshold it uses. Points
|
||||
at the tool for undoing it rather than claiming an undo we do
|
||||
not have: Split is how a wrong merge gets unpicked. */}
|
||||
{!status.in_progress && (data.consolidation?.merged ?? 0) > 0 && (
|
||||
<p className="mt-2 flex items-start gap-2 text-xs text-neutral-600 dark:text-neutral-400">
|
||||
<Users size={14} className="mt-0.5 shrink-0 text-neutral-400" />
|
||||
<span>
|
||||
{t('admin.faces.consolidated', {
|
||||
count: data.consolidation!.merged,
|
||||
defaultValue_one: 'Grouping merged {{count}} look-alike pair automatically after the last scan. Open Manage people to check it — anything merged wrongly can be separated again with Split.',
|
||||
defaultValue_other: 'Grouping merged {{count}} look-alike pairs automatically after the last scan. Open Manage people to check them — anything merged wrongly can be separated again with Split.',
|
||||
})}
|
||||
</span>
|
||||
</p>
|
||||
)}
|
||||
{!status.in_progress && sidecarNotice && (
|
||||
<div className="mt-2">
|
||||
<p className="text-xs text-neutral-500 dark:text-neutral-400 mb-1">
|
||||
|
||||
@@ -56,6 +56,13 @@ interface PersonFace {
|
||||
blur: number | null;
|
||||
}
|
||||
|
||||
/** A pair scoring between the assignment threshold and the auto-merge one. */
|
||||
interface MergeSuggestion {
|
||||
person_a_id: number;
|
||||
person_b_id: number;
|
||||
score: number;
|
||||
}
|
||||
|
||||
interface PeopleManagerModalProps {
|
||||
eventId: number;
|
||||
open: boolean;
|
||||
@@ -199,6 +206,14 @@ export const PeopleManagerModal: React.FC<PeopleManagerModalProps> = ({
|
||||
// differs. Keyed by person id so switching between them reuses the cache.
|
||||
const facesFor = splitting || coverFor || viewing?.person || null;
|
||||
|
||||
// Look-alike pairs the automatic pass deliberately did NOT merge (#1107):
|
||||
// similar enough to ask about, not similar enough to act on unasked.
|
||||
const { data: suggestionData } = useQuery<{ suggestions: MergeSuggestion[] }>({
|
||||
queryKey: ['admin-people-suggestions', eventId],
|
||||
queryFn: async () => (await api.get(`/admin/events/${eventId}/people/suggestions`)).data,
|
||||
enabled: open,
|
||||
});
|
||||
|
||||
const { data: faceData, isLoading: facesLoading } = useQuery<{ faces: PersonFace[] }>({
|
||||
queryKey: ['admin-person-faces', eventId, facesFor?.id],
|
||||
queryFn: async () =>
|
||||
@@ -208,12 +223,53 @@ export const PeopleManagerModal: React.FC<PeopleManagerModalProps> = ({
|
||||
|
||||
const people = useMemo(() => data?.people || [], [data]);
|
||||
|
||||
/**
|
||||
* Suggestions resolved against the loaded people.
|
||||
*
|
||||
* The endpoint returns ids and a score only — the covers are already here.
|
||||
* Pairs whose people are missing are dropped rather than rendered blank: the
|
||||
* two queries are invalidated together, but a suggestion computed just before
|
||||
* a merge can name a person that no longer exists.
|
||||
*/
|
||||
const suggestionPairs = useMemo(() => {
|
||||
const byId = new Map(people.map((p) => [p.id, p]));
|
||||
return (suggestionData?.suggestions || [])
|
||||
.map((s) => ({
|
||||
a: byId.get(s.person_a_id),
|
||||
b: byId.get(s.person_b_id),
|
||||
score: s.score,
|
||||
}))
|
||||
.filter((p): p is { a: AdminPerson; b: AdminPerson; score: number } => !!p.a && !!p.b);
|
||||
}, [suggestionData, people]);
|
||||
|
||||
/**
|
||||
* Names already used in THIS gallery, for the rename input's datalist
|
||||
* (#1107). Naming a wedding means typing the same surname into a dozen
|
||||
* fresh empty inputs; the second occurrence should be a keystroke.
|
||||
*
|
||||
* Deliberately event-scoped. Names from other events would be more useful —
|
||||
* the same family recurs across shoots — but that would surface client names
|
||||
* from galleries the current admin may not be allowed to open, which is a
|
||||
* permissions decision (#743), not an implementation detail.
|
||||
*
|
||||
* No fetch: the people list already in front of the user IS the source.
|
||||
*/
|
||||
const knownNames = useMemo(() => {
|
||||
const names = people
|
||||
.map((p) => (p.label || '').trim())
|
||||
.filter(Boolean);
|
||||
return [...new Set(names)].sort((a, b) => a.localeCompare(b));
|
||||
}, [people]);
|
||||
|
||||
const after = async (message: string) => {
|
||||
// The face list is cached per person and split/merge move faces between
|
||||
// them. Until the cover picker landed, the only reader closed itself after
|
||||
// acting so nobody saw the stale copy; now a second surface reads the same
|
||||
// key and would offer faces that are no longer this person's.
|
||||
await queryClient.invalidateQueries({ queryKey: ['admin-person-faces'] });
|
||||
// Merging or splitting changes which pairs are still worth suggesting, and
|
||||
// a suggestion naming a person that no longer exists is worse than none.
|
||||
await queryClient.invalidateQueries({ queryKey: ['admin-people-suggestions'] });
|
||||
await refetch();
|
||||
onChanged?.();
|
||||
setSelected([]);
|
||||
@@ -262,6 +318,36 @@ export const PeopleManagerModal: React.FC<PeopleManagerModalProps> = ({
|
||||
);
|
||||
};
|
||||
|
||||
/**
|
||||
* Accept a suggestion. The named side is the target so the name survives —
|
||||
* the same rule doMerge applies, but here the order is ours to choose rather
|
||||
* than the click order's, so it can be chosen correctly. Falling back to the
|
||||
* larger cluster keeps the bigger centroid as the survivor.
|
||||
*/
|
||||
const acceptSuggestion = (a: AdminPerson, b: AdminPerson) => {
|
||||
const target = a.label ? a : b.label ? b
|
||||
: ((a.total_face_count ?? a.face_count) >= (b.total_face_count ?? b.face_count) ? a : b);
|
||||
const source = target.id === a.id ? b : a;
|
||||
run(
|
||||
async () => {
|
||||
await api.post(`/admin/events/${eventId}/people/merge`, {
|
||||
source_ids: [source.id], target_id: target.id,
|
||||
});
|
||||
},
|
||||
t('admin.people.merged', { count: 1, defaultValue: 'People merged' })
|
||||
);
|
||||
};
|
||||
|
||||
const dismissSuggestion = (a: AdminPerson, b: AdminPerson) =>
|
||||
run(
|
||||
async () => {
|
||||
await api.post(`/admin/events/${eventId}/people/suggestions/dismiss`, {
|
||||
person_a_id: a.id, person_b_id: b.id,
|
||||
});
|
||||
},
|
||||
t('admin.people.suggestionDismissed', { defaultValue: 'Kept separate' })
|
||||
);
|
||||
|
||||
const doSplit = () => {
|
||||
if (!splitting || !splitFaceIds.length) return;
|
||||
const personId = splitting.id;
|
||||
@@ -307,6 +393,13 @@ export const PeopleManagerModal: React.FC<PeopleManagerModalProps> = ({
|
||||
aria-modal="true"
|
||||
className="relative bg-white dark:bg-neutral-900 text-neutral-900 dark:text-neutral-100 rounded-xl shadow-xl w-full max-w-4xl max-h-[88vh] flex flex-col"
|
||||
>
|
||||
{/* One datalist for every row's rename input — a per-row copy would
|
||||
duplicate the whole name list once per person. */}
|
||||
{knownNames.length > 0 && (
|
||||
<datalist id="picpeak-people-names">
|
||||
{knownNames.map((name) => <option key={name} value={name} />)}
|
||||
</datalist>
|
||||
)}
|
||||
<div className="flex items-center justify-between px-5 py-4 border-b border-neutral-100 dark:border-neutral-700">
|
||||
<div>
|
||||
<h2 className="text-lg font-medium text-neutral-900 dark:text-neutral-100">
|
||||
@@ -539,6 +632,71 @@ export const PeopleManagerModal: React.FC<PeopleManagerModalProps> = ({
|
||||
</p>
|
||||
) : (
|
||||
<div className="space-y-1">
|
||||
{/* --- merge suggestions (#1107) -------------------------
|
||||
The band below the auto-merge threshold. These are asked
|
||||
rather than done: an over-eager merge of two people is
|
||||
much harder to unpick than a missed one, and this is
|
||||
biometric grouping, so the uncertain cases get a human.
|
||||
Dismissal is sticky — a pair told "not the same" does not
|
||||
come back after the next scan. */}
|
||||
{suggestionPairs.length > 0 && (
|
||||
<div className="mb-4 rounded-lg border border-amber-200 dark:border-amber-800 bg-amber-50 dark:bg-amber-900/20 overflow-hidden">
|
||||
<p className="px-3 py-2 text-xs text-amber-900 dark:text-amber-200 border-b border-amber-200 dark:border-amber-800">
|
||||
{t('admin.people.suggestionsHeading', {
|
||||
count: suggestionPairs.length,
|
||||
defaultValue: 'These might be the same person. Grouping was not confident enough to merge them on its own.',
|
||||
})}
|
||||
</p>
|
||||
<div className="divide-y divide-amber-200 dark:divide-amber-800">
|
||||
{suggestionPairs.map(({ a, b, score }) => (
|
||||
<div key={`${a.id}-${b.id}`} className="flex items-center gap-3 p-3 flex-wrap">
|
||||
<div className="flex items-center gap-2">
|
||||
{[a, b].map((person) => (
|
||||
<div key={person.id} className="flex items-center gap-2">
|
||||
<FaceThumb
|
||||
eventId={eventId}
|
||||
photoId={person.cover?.photo_id ?? 0}
|
||||
bbox={person.cover?.bbox}
|
||||
photoWidth={person.cover?.photo_width}
|
||||
photoHeight={person.cover?.photo_height}
|
||||
size={48}
|
||||
/>
|
||||
<span className="text-xs text-neutral-700 dark:text-neutral-300">
|
||||
{person.label || t('admin.people.photoCount', {
|
||||
count: person.total_face_count ?? person.face_count,
|
||||
defaultValue: `${person.total_face_count ?? person.face_count} photos`,
|
||||
})}
|
||||
</span>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
<span className="text-xs text-neutral-500 dark:text-neutral-400 tabular-nums">
|
||||
{t('admin.people.suggestionScore', {
|
||||
percent: Math.round(score * 100),
|
||||
defaultValue: `${Math.round(score * 100)}% alike`,
|
||||
})}
|
||||
</span>
|
||||
<div className="flex gap-2 ml-auto">
|
||||
<Button
|
||||
variant="outline" size="sm" disabled={busy}
|
||||
onClick={() => dismissSuggestion(a, b)}
|
||||
>
|
||||
{t('admin.people.suggestionReject', { defaultValue: 'Not the same' })}
|
||||
</Button>
|
||||
<Button
|
||||
variant="primary" size="sm" disabled={busy}
|
||||
onClick={() => acceptSuggestion(a, b)}
|
||||
leftIcon={<Merge className="w-3.5 h-3.5" />}
|
||||
>
|
||||
{t('admin.people.suggestionAccept', { defaultValue: 'Same person' })}
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{people.map((person) => {
|
||||
const isSelected = selected.includes(person.id);
|
||||
return (
|
||||
@@ -577,6 +735,7 @@ export const PeopleManagerModal: React.FC<PeopleManagerModalProps> = ({
|
||||
if (e.key === 'Escape') setRenaming(null);
|
||||
}}
|
||||
placeholder={t('admin.people.namePlaceholder', { defaultValue: 'Add a name' })}
|
||||
list={knownNames.length ? 'picpeak-people-names' : undefined}
|
||||
className="w-full max-w-xs px-2 py-1 text-sm border border-neutral-300 dark:border-neutral-600 bg-white dark:bg-neutral-800 text-neutral-900 dark:text-neutral-100 rounded focus:outline-none focus:ring-2 focus:ring-primary-500"
|
||||
/>
|
||||
) : (
|
||||
|
||||
@@ -3150,7 +3150,14 @@
|
||||
"splitHelp": "Wählen Sie die Fotos aus, die NICHT diese Person zeigen. Sie werden zu einem neuen Eintrag, alles andere bleibt.",
|
||||
"splitSelected": "{{count}} ausgewählt",
|
||||
"doSplit": "Abtrennen",
|
||||
"empty": "Noch keine Personen erkannt."
|
||||
"empty": "Noch keine Personen erkannt.",
|
||||
"photoCount_one": "{{count}} Foto",
|
||||
"photoCount_other": "{{count}} Fotos",
|
||||
"suggestionsHeading": "Das könnte dieselbe Person sein. Die Gruppierung war sich nicht sicher genug, um sie von selbst zusammenzuführen.",
|
||||
"suggestionScore": "{{percent}}% ähnlich",
|
||||
"suggestionAccept": "Dieselbe Person",
|
||||
"suggestionReject": "Nicht dieselbe",
|
||||
"suggestionDismissed": "Getrennt gelassen"
|
||||
},
|
||||
"faces": {
|
||||
"manage": "Personen verwalten",
|
||||
@@ -3180,7 +3187,9 @@
|
||||
"sidecarUnreachable": "Der Gesichtserkennungs-Dienst unter {{url}} ist nicht erreichbar, deshalb werden die {{pending}} eingereihten Fotos nicht verarbeitet. Es geht nichts verloren — der Scan läuft von selbst weiter, sobald der Dienst wieder da ist. Starte ihn mit `docker compose --profile faces up -d`; er beendet sich sofort, wenn FACE_ML_TOKEN nicht auf denselben Wert wie im Backend gesetzt ist — einen Standardwert gibt es nicht.",
|
||||
"sidecarUnauthorized": "Der Gesichtserkennungs-Dienst unter {{url}} weist unser Token zurück; Fotos werden als fehlgeschlagen markiert statt erneut versucht. Setze FACE_ML_TOKEN im Backend und im picpeak-ml-Container identisch, starte beide neu und nutze dann Neu scannen — das Token allein zu korrigieren verarbeitet die bereits fehlgeschlagenen Fotos nicht erneut.",
|
||||
"sidecarRejected": "{{url}} antwortet, aber nicht wie der Gesichtserkennungs-Dienst — Fotos werden als fehlgeschlagen markiert statt erneut versucht. Prüfe, ob FACE_ML_URL auf den picpeak-ml-Container zeigt und kein Proxy dazwischenliegt, und nutze dann Neu scannen für die bereits fehlgeschlagenen Fotos.",
|
||||
"sidecarStateNow": "Aktueller Zustand des Dienstes — einige der Fehler oben können eine andere Ursache haben, aber ein erneuter Scan wird erst nach der Behebung erfolgreich sein:"
|
||||
"sidecarStateNow": "Aktueller Zustand des Dienstes — einige der Fehler oben können eine andere Ursache haben, aber ein erneuter Scan wird erst nach der Behebung erfolgreich sein:",
|
||||
"consolidated_one": "Beim letzten Scan wurde {{count}} ähnliches Paar automatisch zusammengeführt. Prüfen Sie es unter „Personen verwalten“ — falsch Zusammengeführtes lässt sich mit „Trennen“ wieder aufteilen.",
|
||||
"consolidated_other": "Beim letzten Scan wurden {{count}} ähnliche Paare automatisch zusammengeführt. Prüfen Sie sie unter „Personen verwalten“ — falsch Zusammengeführtes lässt sich mit „Trennen“ wieder aufteilen."
|
||||
}
|
||||
},
|
||||
"acceptInvitation": {
|
||||
|
||||
@@ -2722,7 +2722,14 @@
|
||||
"splitHelp": "Pick the photos that are NOT this person. They become a new entry, and everything else stays.",
|
||||
"splitSelected": "{{count}} selected",
|
||||
"doSplit": "Split out",
|
||||
"empty": "No people detected yet."
|
||||
"empty": "No people detected yet.",
|
||||
"photoCount_one": "{{count}} photo",
|
||||
"photoCount_other": "{{count}} photos",
|
||||
"suggestionsHeading": "These might be the same person. Grouping was not confident enough to merge them on its own.",
|
||||
"suggestionScore": "{{percent}}% alike",
|
||||
"suggestionAccept": "Same person",
|
||||
"suggestionReject": "Not the same",
|
||||
"suggestionDismissed": "Kept separate"
|
||||
},
|
||||
"faces": {
|
||||
"manage": "Manage people",
|
||||
@@ -2752,7 +2759,9 @@
|
||||
"sidecarUnreachable": "Can't reach the face-detection service at {{url}}, so the {{pending}} queued photos aren't being processed. Nothing is lost — the scan resumes on its own once the service is up. Start it with `docker compose --profile faces up -d`, and note it exits immediately unless FACE_ML_TOKEN is set to the same value as the backend — there is no default.",
|
||||
"sidecarUnauthorized": "The face-detection service at {{url}} is rejecting our token, and photos are being marked failed rather than retried. Make FACE_ML_TOKEN identical on the backend and the picpeak-ml container, restart both, then use Re-scan — fixing the token alone will not reprocess the photos that already failed.",
|
||||
"sidecarRejected": "{{url}} answered, but not like the face-detection service — photos are being marked failed rather than retried. Check FACE_ML_URL points at the picpeak-ml container and that nothing is proxying that address, then use Re-scan for the photos that already failed.",
|
||||
"sidecarStateNow": "Service state right now — some of the failures above may have a different cause, but a re-scan will not succeed until this is fixed:"
|
||||
"sidecarStateNow": "Service state right now — some of the failures above may have a different cause, but a re-scan will not succeed until this is fixed:",
|
||||
"consolidated_one": "Grouping merged {{count}} look-alike pair automatically after the last scan. Open Manage people to check it — anything merged wrongly can be separated again with Split.",
|
||||
"consolidated_other": "Grouping merged {{count}} look-alike pairs automatically after the last scan. Open Manage people to check them — anything merged wrongly can be separated again with Split."
|
||||
}
|
||||
},
|
||||
"acceptInvitation": {
|
||||
|
||||
Reference in New Issue
Block a user