fix(upload): auto-throttle on low-memory hosts + correct documented RAM minimum (#628)
The README claimed 2GB RAM as the minimum, but two background-processor worker loops × sharp.concurrency(2) means up to four libvips threads can decode full-resolution images in parallel — peak RSS lands at 1.5GB+ on a batch of 20MP+ photos. Add Postgres + Redis + Node baseline and one heavy batch on a 2GB VPS OOM-kills the backend, surfacing as 503s on thumbnails until restart:unless-stopped brings it back. Reported in #602, filed as #628. Three changes, smallest-surface-area each: 1. backgroundProcessor.js — on startup, when UPLOAD_PROCESSOR_CONCURRENCY is NOT set and os.totalmem() reports < 3GB, default to 1 instead of 2 and log a one-shot warning naming the override env var. Explicit env-var setters keep their value. os.totalmem() reports container memory under cgroup v2 so this works in Docker / k8s as well as bare metal. 2. README.md — bumped the documented minimum from 2GB to 4GB, kept 2GB only as a "Low-memory hosts" recipe pointing at UPLOAD_PROCESSOR_CONCURRENCY=1 with the throughput trade-off spelled out. Added the 503-on-OOM symptom so the next reporter finds it via search. 3. docker-compose.production.yml — commented mem_limit / memswap_limit example on the backend service. Off by default (don't surprise existing deployments) but visible to operators thinking about shared/multi-tenant hosts. restart:unless-stopped already on every service. No code path for memory-aware runtime throttling (Luca's option 4) — out of scope for a bug fix; tracked separately if #1-#3 don't close the case.
This commit is contained in:
@@ -69,6 +69,15 @@ services:
|
||||
redis:
|
||||
condition: service_healthy
|
||||
restart: unless-stopped
|
||||
# Memory cap (optional, recommended on shared / multi-tenant hosts):
|
||||
# uncomment to bound the backend's RSS. Sharp/libvips decodes the full
|
||||
# uncompressed image before resize, so a multi-photo upload batch can
|
||||
# spike memory. With a cap set, the kernel OOM-killer takes the
|
||||
# container instead of the whole host; restart:unless-stopped brings
|
||||
# it back. Match this to the RAM budget you've allocated for picpeak
|
||||
# (`docker stats` shows the live usage).
|
||||
# mem_limit: 3g
|
||||
# memswap_limit: 3g
|
||||
healthcheck:
|
||||
# Backend exposes /health on internal port 3000.
|
||||
# The backend image only ships wget (Alpine base) — using curl
|
||||
|
||||
Reference in New Issue
Block a user