8cc7d7d14a
* feat(external-media): watch reference folders and import new files automatically Managed uploads dropped into storage/events/active are picked up by the chokidar watcher; external media had no equivalent, so a NAS folder that keeps growing needed an admin to open the event and press Import every time. Relates to issue 1187. - The import pass moves out of the route into services/externalImportService.js. The watcher and the Import button now run the identical function; the route only validates and maps errors to status codes. - Mutual exclusion is the per-event claim from maintenanceJobState (`external_import:<id>`, seeded on demand by the new ensure()) instead of the in-process Set. The Set stopped a double-click in one process; the claim also stops the watcher on a second replica, or an admin clicking while the watcher is mid-run elsewhere. The run heartbeats so a claim from a dead process is taken over. - services/externalMediaWatcher.js: per-event opt-in via the new events.external_watch column (migration 208), chokidar with awaitWriteFinish so a copy in flight is not imported half-written, debounced full pass per change, a timer sweep every 15 minutes as the fallback for NFS/SMB mounts that deliver no inotify events, optional stat-polling via EXTERNAL_MEDIA_WATCH_POLLING. The set of watched events is re-read every minute, so the toggle works from any replica. A watcher that just started runs one pass immediately. - Deletions are ignored on purpose: a file vanishing from a NAS is at least as likely to be a reorganisation or a dropped mount as an intentional removal, and acting on it would delete a guest-visible photo. Rows whose file is gone stay, as they do today. - Not gated on STORAGE_BACKEND: EXTERNAL_MEDIA_ROOT is always local. - Quiet system passes stay out of the activity log; runs that imported something are logged with actor external-media-watcher. - Frontend: "Watch folder for new files" checkbox under the external folder picker, status line in view mode, EN/DE strings. * fix(external-media): close the review gaps in the folder watcher Codex review of the watcher, round 1. All six findings were real: - Enabling the watcher, or pointing an enabled one at another folder, now requires photos.upload — the permission the manual Import already requires. events.edit alone was a way around it. Only the transition is checked, so a role without photos.upload can still edit an already-watched event. The checkbox is disabled for such roles. - Automatic passes defer files that are still changing: anything modified inside the stability window, or whose size moves across one wait of that window, is left for the next pass. chokidar's awaitWriteFinish only settles the file that fired the event, and the sweep sees no events at all, so a sibling still being copied could be inserted half-written and then skipped forever. - Photos an admin deleted are not brought back by the sweep. The delete routes record the file in external_import_exclusions (migration 209); automatic passes skip the list, the manual Import ignores it and clears it for what it imports. - The six EXTERNAL_MEDIA_WATCH* variables are forwarded in all three compose files; they were documented but the backend services use explicit environment lists, so the kill switch did nothing. - A pass re-checks is_active / is_archived at run time, not only in the minutely reconcile. - The lease is renewed on a timer for the whole run, walk included, and ownership is checked before the event row is touched. * fix(external-media): make automatic passes follow the row, not rewrite it Codex review round 2, four findings, all applied: - The event update route drops non-canonical spellings of external_watch and external_path before the permission guard. SQLite resolves column names case-insensitively, so `External_Watch` reached the column while the guard only looked at the lowercase key. - Exclusions are checked per file at insert time, not against a snapshot taken before the settle wait. A photo deleted during the wait was present in the snapshot and got re-inserted by the loop. - An automatic pass no longer writes source_mode / external_path. It re-reads the row after the walk and the settle wait and stops if the folder changed or the event went managed; the manual Import is the only writer. The options are now `automatic` + `settleMs`. - A pass that deferred files re-arms the debounced import, so a file copied just before the watcher started is not stranded when the sweep is disabled. * fix(external-media): keep exclusions for replaced photos, stop a pass whose event stopped qualifying Codex review round 3, both findings applied: - recordExclusions keys on external_relpath alone. A replaced external photo becomes managed but keeps its relpath on purpose, and deleting that replacement must not republish the NAS original. - An automatic pass checks the full watcher predicate (reference mode, same folder, watch on, active, not archived) before it inserts and on every heartbeat tick during the loop, and stops as soon as the event no longer qualifies. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
288 lines
12 KiB
YAML
288 lines
12 KiB
YAML
services:
|
|
# Generates machine secrets (JWT/DB/Redis) on first run when they aren't set
|
|
# in .env (seeds from the env var when provided, else a random value).
|
|
# Idempotent — never overwrites an existing file. See docker-compose.production.yml.
|
|
secrets-init:
|
|
image: alpine:3.20
|
|
container_name: picpeak-secrets-init
|
|
env_file: .env
|
|
entrypoint:
|
|
- sh
|
|
- -c
|
|
- |
|
|
set -e
|
|
mkdir -p /run/secrets
|
|
if [ ! -s /run/secrets/jwt_secret ]; then
|
|
if [ -n "$$JWT_SECRET" ]; then printf '%s' "$$JWT_SECRET" > /run/secrets/jwt_secret;
|
|
else tr -dc A-Za-z0-9 < /dev/urandom | head -c 48 > /run/secrets/jwt_secret; fi
|
|
fi
|
|
if [ ! -s /run/secrets/db_password ]; then
|
|
if [ -n "$$DB_PASSWORD" ]; then printf '%s' "$$DB_PASSWORD" > /run/secrets/db_password;
|
|
else tr -dc A-Za-z0-9 < /dev/urandom | head -c 48 > /run/secrets/db_password; fi
|
|
fi
|
|
if [ ! -s /run/secrets/redis_password ]; then
|
|
if [ -n "$$REDIS_PASSWORD" ]; then printf '%s' "$$REDIS_PASSWORD" > /run/secrets/redis_password;
|
|
else tr -dc A-Za-z0-9 < /dev/urandom | head -c 48 > /run/secrets/redis_password; fi
|
|
fi
|
|
# 644: the readers run as three different users (postgres, redis, nodejs),
|
|
# so a non-root reader must be able to read them. The volume is private to
|
|
# these containers and never host-exposed.
|
|
chmod 644 /run/secrets/jwt_secret /run/secrets/db_password /run/secrets/redis_password
|
|
volumes:
|
|
- picpeak-secrets:/run/secrets
|
|
restart: "no"
|
|
|
|
backend:
|
|
build:
|
|
context: ./backend
|
|
dockerfile: Dockerfile
|
|
container_name: picpeak-backend
|
|
restart: unless-stopped
|
|
environment:
|
|
- NODE_ENV=${NODE_ENV:-production}
|
|
- PORT=3000
|
|
- JWT_SECRET=${JWT_SECRET:-}
|
|
- ADMIN_USERNAME=${ADMIN_USERNAME:-admin}
|
|
- ADMIN_EMAIL=${ADMIN_EMAIL:-admin@example.com}
|
|
- ADMIN_PASSWORD=${ADMIN_PASSWORD:-}
|
|
# How long a gallery guest stays recognised (#1210). Unset = 30d.
|
|
# Listed explicitly because this service takes an environment list, so a
|
|
# value in .env that is not named here never reaches the container.
|
|
- GUEST_TOKEN_TTL=${GUEST_TOKEN_TTL:-}
|
|
- DATABASE_CLIENT=pg
|
|
- DB_TYPE=postgresql
|
|
- DB_HOST=postgres
|
|
- DB_PORT=5432
|
|
- DB_USER=${DB_USER}
|
|
- DB_PASSWORD=${DB_PASSWORD:-}
|
|
- DB_NAME=${DB_NAME}
|
|
- EXTERNAL_MEDIA_ROOT=${EXTERNAL_MEDIA_ROOT:-/app/storage/external-media}
|
|
- SMTP_HOST=${SMTP_HOST}
|
|
- SMTP_PORT=${SMTP_PORT}
|
|
- SMTP_SECURE=${SMTP_SECURE:-false}
|
|
- SMTP_USER=${SMTP_USER}
|
|
- SMTP_PASS=${SMTP_PASS}
|
|
- EMAIL_FROM=${EMAIL_FROM:-noreply@picpeak.local}
|
|
# Webhook email transport (#1225). Listed here because this service
|
|
# declares an explicit `environment:` block — a variable only present in
|
|
# .env is NOT passed through, so without these three the documented
|
|
# "uncomment in .env and restart" flow silently leaves the transport off.
|
|
- EMAIL_WEBHOOK_URL=${EMAIL_WEBHOOK_URL:-}
|
|
- EMAIL_WEBHOOK_SECRET=${EMAIL_WEBHOOK_SECRET:-}
|
|
- EMAIL_WEBHOOK_ALLOW_PRIVATE_URLS=${EMAIL_WEBHOOK_ALLOW_PRIVATE_URLS:-false}
|
|
# Unset by default (#705): an injected value would always win over the
|
|
# `general_site_url` admin setting, so the setup wizard could never
|
|
# take effect. Set this only to pin the origin from config-as-code.
|
|
- FRONTEND_URL=${FRONTEND_URL:-}
|
|
# Public API origin for split-origin deployments (#798 SSO redirect_uri).
|
|
# Empty = same origin as FRONTEND_URL (the standard proxied setup).
|
|
- API_URL=${API_URL:-}
|
|
# OIDC SSO (#798): key for the client secret at rest (falls back to
|
|
# JWT_SECRET) and the break-glass override that re-enables local
|
|
# password login when the IdP is down while SSO-only mode is active.
|
|
- OIDC_ENCRYPTION_KEY=${OIDC_ENCRYPTION_KEY:-}
|
|
# Optional product usage (#1110); remains off until explicit in-app consent.
|
|
- USAGE_COLLECTOR_URL=${USAGE_COLLECTOR_URL:-https://usage.picpeak.app}
|
|
- USAGE_ENCRYPTION_KEY=${USAGE_ENCRYPTION_KEY:-}
|
|
- OIDC_BREAK_GLASS=${OIDC_BREAK_GLASS:-}
|
|
- ADMIN_URL=${ADMIN_URL:-}
|
|
- TZ=${TZ:-UTC}
|
|
- STORAGE_PATH=/app/storage
|
|
# Watch-folder auto-import: max photos processed in parallel (default 2).
|
|
- FILE_WATCHER_CONCURRENCY=${FILE_WATCHER_CONCURRENCY:-2}
|
|
# External-media folder watcher (issue 1187); all optional, see .env.example.
|
|
- EXTERNAL_MEDIA_WATCH=${EXTERNAL_MEDIA_WATCH:-true}
|
|
- EXTERNAL_MEDIA_WATCH_POLLING=${EXTERNAL_MEDIA_WATCH_POLLING:-false}
|
|
- EXTERNAL_MEDIA_WATCH_POLL_INTERVAL_MS=${EXTERNAL_MEDIA_WATCH_POLL_INTERVAL_MS:-5000}
|
|
- EXTERNAL_MEDIA_WATCH_SWEEP_INTERVAL_MS=${EXTERNAL_MEDIA_WATCH_SWEEP_INTERVAL_MS:-900000}
|
|
- EXTERNAL_MEDIA_WATCH_DEBOUNCE_MS=${EXTERNAL_MEDIA_WATCH_DEBOUNCE_MS:-10000}
|
|
- EXTERNAL_MEDIA_WATCH_STABILITY_MS=${EXTERNAL_MEDIA_WATCH_STABILITY_MS:-5000}
|
|
# Face recognition (#1074). The URL defaults to the sidecar's compose
|
|
# service name, so the common case needs no configuration. None of this
|
|
# is touched until the `faces` feature flag is enabled in admin
|
|
# settings — an install without the picpeak-ml container never attempts
|
|
# a connection.
|
|
- FACE_ML_URL=${FACE_ML_URL:-http://picpeak-ml:8000}
|
|
- FACE_ML_TOKEN=${FACE_ML_TOKEN:-}
|
|
- FACE_PROCESSOR_CONCURRENCY=${FACE_PROCESSOR_CONCURRENCY:-}
|
|
# No `user:` directive — as of #484, the container starts as root,
|
|
# chowns the bind mounts to nodejs (UID 1001), then drops privileges
|
|
# via su-exec. PUID/PGID env vars are no longer read; if you need
|
|
# a different runtime UID, pre-chown the host dirs and pin
|
|
# `user: "<uid>:<gid>"` here.
|
|
volumes:
|
|
- ./events:/app/events
|
|
- ./data:/app/data
|
|
- ./logs:/app/logs
|
|
- ./backup:/backup
|
|
- ./storage:/app/storage
|
|
- picpeak-secrets:/run/secrets:ro
|
|
ports:
|
|
- "${BACKEND_PORT:-3001}:3000"
|
|
depends_on:
|
|
secrets-init:
|
|
condition: service_completed_successfully
|
|
postgres:
|
|
condition: service_healthy
|
|
healthcheck:
|
|
test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://127.0.0.1:3000/health"]
|
|
interval: 30s
|
|
timeout: 10s
|
|
retries: 3
|
|
start_period: 40s
|
|
networks:
|
|
- picpeak-network
|
|
|
|
postgres:
|
|
image: postgres:15-alpine
|
|
container_name: picpeak-postgres
|
|
restart: unless-stopped
|
|
userns_mode: "host"
|
|
environment:
|
|
- POSTGRES_USER=${DB_USER}
|
|
- POSTGRES_PASSWORD_FILE=/run/secrets/db_password
|
|
- POSTGRES_DB=${DB_NAME}
|
|
- PGDATA=/var/lib/postgresql/data/pgdata
|
|
- TZ=${TZ:-UTC}
|
|
volumes:
|
|
- postgres-data:/var/lib/postgresql/data
|
|
- picpeak-secrets:/run/secrets:ro
|
|
depends_on:
|
|
secrets-init:
|
|
condition: service_completed_successfully
|
|
ports:
|
|
- "127.0.0.1:${DB_PORT:-5432}:5432"
|
|
healthcheck:
|
|
test: ["CMD-SHELL", "pg_isready -U ${DB_USER} -d ${DB_NAME}"]
|
|
interval: 10s
|
|
timeout: 5s
|
|
retries: 5
|
|
start_period: 20s
|
|
networks:
|
|
- picpeak-network
|
|
|
|
redis:
|
|
image: redis:7-alpine
|
|
container_name: picpeak-redis
|
|
restart: unless-stopped
|
|
userns_mode: "host"
|
|
command: sh -c 'exec redis-server --appendonly yes --requirepass "$$(cat /run/secrets/redis_password)"'
|
|
volumes:
|
|
- redis-data:/data
|
|
- picpeak-secrets:/run/secrets:ro
|
|
depends_on:
|
|
secrets-init:
|
|
condition: service_completed_successfully
|
|
ports:
|
|
- "127.0.0.1:${REDIS_PORT:-6379}:6379"
|
|
healthcheck:
|
|
test: ["CMD", "redis-cli", "--raw", "incr", "ping"]
|
|
interval: 10s
|
|
timeout: 5s
|
|
retries: 3
|
|
networks:
|
|
- picpeak-network
|
|
|
|
# Local mail catcher for development/testing only — never wanted in a real
|
|
# deployment. Gated behind the `dev` profile so a plain `docker compose up -d`
|
|
# does NOT start it; opt in with `docker compose --profile dev up -d`. Nothing
|
|
# depends on it (SMTP_HOST comes from .env), so gating is safe.
|
|
mailhog:
|
|
image: mailhog/mailhog:latest
|
|
container_name: picpeak-mailhog
|
|
restart: unless-stopped
|
|
profiles:
|
|
- dev
|
|
ports:
|
|
- "${MAILHOG_SMTP_PORT:-1025}:1025"
|
|
- "${MAILHOG_UI_PORT:-8025}:8025"
|
|
networks:
|
|
- picpeak-network
|
|
|
|
# Optional face-detection sidecar (#1074). Gated behind the `faces` profile
|
|
# so a plain `docker compose up -d` does NOT start it — opt in with
|
|
# `docker compose --profile faces up -d`. Nothing depends on it: the backend
|
|
# only ever calls it when the `faces` feature flag is on, so an install that
|
|
# skips this service behaves exactly as it did before the feature existed.
|
|
#
|
|
# The service name is `picpeak-ml` (not `ml`) because it doubles as the
|
|
# hostname in FACE_ML_URL's default, `http://picpeak-ml:8000`. Renaming this
|
|
# service silently breaks that default for every install that never set the
|
|
# variable.
|
|
#
|
|
# Requires FACENET_ONNX_URL / FACENET_ONNX_SHA256 at build time — see
|
|
# ml/README.md. The image publishes no host port and mounts no volumes; it
|
|
# is reachable only from the backend on picpeak-network.
|
|
picpeak-ml:
|
|
build:
|
|
context: ./ml
|
|
dockerfile: Dockerfile
|
|
args:
|
|
# Defaults live in ml/Dockerfile and point at the canonical published
|
|
# model. These pass an override through from .env when set; an empty
|
|
# value here would BLANK the Dockerfile default and fail the build,
|
|
# so the fallbacks repeat it deliberately.
|
|
- FACENET_ONNX_URL=${FACENET_ONNX_URL:-https://github.com/PicPeak/picpeak/releases/download/ml-models-v1/facenet512.onnx}
|
|
- FACENET_ONNX_SHA256=${FACENET_ONNX_SHA256:-a1c06dcb79dc17a42af01d5bcbce4822caa148b9c24bf7eb8b8e556b4fd0d5db}
|
|
container_name: picpeak-ml
|
|
restart: unless-stopped
|
|
profiles:
|
|
- faces
|
|
environment:
|
|
# Shared secret with the backend. The container refuses to start
|
|
# without it rather than serving anonymously.
|
|
- FACE_ML_TOKEN=${FACE_ML_TOKEN:-}
|
|
- FACE_ORT_THREADS=${FACE_ORT_THREADS:-1}
|
|
- TZ=${TZ:-UTC}
|
|
healthcheck:
|
|
test: ["CMD", "python", "-c", "import urllib.request,sys; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=4).status == 200 else 1)"]
|
|
interval: 30s
|
|
timeout: 10s
|
|
retries: 3
|
|
start_period: 40s
|
|
networks:
|
|
- picpeak-network
|
|
|
|
frontend:
|
|
build:
|
|
context: ./frontend
|
|
dockerfile: Dockerfile
|
|
args:
|
|
- VITE_API_URL=${VITE_API_URL:-/api}
|
|
- VITE_UMAMI_URL=${VITE_UMAMI_URL:-}
|
|
- VITE_UMAMI_WEBSITE_ID=${VITE_UMAMI_WEBSITE_ID:-}
|
|
- VITE_UMAMI_SHARE_URL=${VITE_UMAMI_SHARE_URL:-}
|
|
container_name: picpeak-frontend
|
|
restart: unless-stopped
|
|
environment:
|
|
- NODE_ENV=${NODE_ENV:-production}
|
|
# Static social-preview brand (#521) — substituted into
|
|
# index.html at container start; see frontend/docker-entrypoint.sh.
|
|
- BRAND_TITLE=${BRAND_TITLE:-PicPeak}
|
|
- BRAND_DESCRIPTION=${BRAND_DESCRIPTION:-Photo gallery shared with PicPeak.}
|
|
ports:
|
|
- "${FRONTEND_PORT:-3000}:80"
|
|
depends_on:
|
|
- backend
|
|
healthcheck:
|
|
test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://127.0.0.1/health"]
|
|
interval: 30s
|
|
timeout: 10s
|
|
retries: 3
|
|
networks:
|
|
- picpeak-network
|
|
|
|
volumes:
|
|
postgres-data:
|
|
driver: local
|
|
redis-data:
|
|
driver: local
|
|
# Auto-generated machine secrets (jwt/db/redis). Keep it — deleting it orphans
|
|
# the DB password from the Postgres volume.
|
|
picpeak-secrets:
|
|
driver: local
|
|
|
|
networks:
|
|
picpeak-network:
|
|
driver: bridge
|