Files
picpeak/.github/workflows
Paul Nothaft 37a15e3d49 fix(faces): restore the :beta image tag and surface sidecar health (#1087)
* fix(faces): restore the :beta image tag and surface sidecar health

Both halves of what a user hit on discussions/1069: the People card sat
at "Scanning… 0 of 227" for 30 minutes with no explanation, because the
sidecar container could never have started.

docker-build.yml — republish `:beta`. It used to come for free via
`type=ref,event=branch` when the active development branch was literally
named `beta`; the rename to `main` silently retired it. backend:beta has
been frozen at 2026-06-29 (448da950) ever since while :main moved on, so
PICPEAK_CHANNEL=beta has been serving a seven-week-old build across every
image. The ml sidecar was added after the rename and so never had a
`:beta` at all, which left docker-compose.production.yml:158 unable to
resolve ghcr.io/picpeak/picpeak/ml for any documented channel — the
image simply does not exist as :beta or :stable, only as :main and
pinned versions. Tag added to all four merge jobs, gated on main.

`:stable` stays absent for ml on purpose: it is gated on refs/heads/stable
and the sidecar does not exist there. stable's docker-compose.production.yml
carries no picpeak-ml service, so nothing can reference the missing tag.

FaceRecognitionCard — show when the sidecar is unreachable. An
unreachable sidecar is not an error by design: faceQueue.js:132-136
releases the photo back to `pending` and retries forever so a restart
does not burn the queue. The cost was that a stopped container looked
exactly like a slow scan, indefinitely, and the only signal was a
backend log line rate-limited to once per five minutes.
/admin/events/faces/health already existed and nothing in the frontend
called it. It is now polled while a scan is in progress, and a failing
check replaces the spinner with the sidecar URL, the underlying error
(which distinguishes a stopped container from a token mismatch) and the
command to start it.

Health is only polled while a scan is running — an idle card has no
reason to care whether the sidecar is up.

* fix(faces): tell the three sidecar failure modes apart

Follow-up to the health surface in this branch, from an external review
pass. The original warning was right about "the sidecar is not working"
and wrong about almost everything after that.

faceClient.checkHealth now returns a `reason` rather than only a message,
because the caller has to know whether photos survive:
  - 'unauthorized' (401) and 'rejected' (any other 4xx) both become
    SidecarRejectedError in classify(), which workerLoop does NOT retry —
    every claimed photo is marked 'failed'. Telling the admin the scan
    resumes on its own was simply untrue there; both now say to fix the
    cause and Re-scan.
  - 'unreachable' (refused/DNS/timeout/5xx) is the retryable one.

The card also no longer cries wolf. /faces runs inference synchronously
inside an `async def`, so one slow photo blocks the event loop and stalls
/info past its 5s timeout — a healthy sidecar can fail a probe. Verified
with an isolated uvicorn repro: a blocking call in an async handler
stalled the sync /info endpoint to 5.01s. The warning now needs three
consecutive failures AND no drop in `pending`. Three because a single
/faces call may legitimately run to FACE_ML_TIMEOUT_MS (30s) and two
probes 15s apart both fit inside that window; `pending` rather than
`scanned` because scanned counts only 'done', so a run producing
skipped/failed photos is progress that counter misses.

A 4xx burns the queue with no backoff, so it can empty before anyone
opens the card — in_progress goes false and only "227 failed" is left.
The probe therefore also runs when a finished scan has failures, and the
notice renders under the counts instead of replacing them. It is worded
as present-tense service state, not as a claim about those specific
failures: a live probe cannot know whether they came from this
misconfiguration or from corrupt images earlier. Attributing them exactly
needs stored face_error rows, which is a bigger change than this.

Also adds the missing-token case to the unreachable text: FACE_ML_TOKEN
has no default and the container refuses to start without it, so the most
likely first run fails as a plain connection refusal that "just start it"
does not fix.

---------

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
2026-08-19 16:49:47 +02:00
..