Files
picpeak/backend/src/services/imageProcessor.js
T
Luca 8db8527f9e feat(api): Lightroom round-trip — read proofing marks, put finished edits back (#745) (#1165)
* feat(api): Lightroom round-trip — read marks, put edits back (#745)

Gets a client's proofing verdict into a desktop catalogue and a finished
edit back over its proof, without anyone re-matching files by hand.

Three parts:

**Keep the camera filename.** photos.original_filename is the only carrier
of `IMG_1234.JPG` — the stored filename is rewritten by
generatePhotoFilename. But replacePhoto() overwrites original_filename with
whatever name the new file arrives under, so the first re-upload of a
renamed render destroys the key the NEXT round-trip needs. Migration 185
adds photos.source_filename, written once at ingest and never touched by a
replace, backfilled from original_filename so existing galleries can still
match on their first pass. The backfill sits outside the column guard and
keys on whereNull, so a run that dies partway self-heals instead of leaving
half the rows empty forever.

**Read the marks.** GET /api/v1/events/:id/photos returns each photo with
its client colour tallies, the caller's own marks, and a merged colour +
rating. Guards copied from the sibling upload route (apiTokenAuth +
read scope + photos.view + requireEventOwnership). Filters: marked_only,
mark_source, color_labels, my_color_labels, min_rating, my_min_rating.

The route filters to a page of ids with PhotoFilterBuilder, then enriches
just those through photoExportService.getPhotosWithFeedback — the two
halves already existed and neither does both, and going id-first keeps the
per-colour tally query bounded by page size.

services/markMerge.js decides how three possible opinions (guest colours,
guest star average, the photographer's own row in photo_admin_marks)
collapse into the one colour and one rating Lightroom has room for. Colour
goes to the photographer on a tie — one deliberate choice beats an
aggregate a tie-break already had to guess at. Rating takes the max,
because a rating is a magnitude and losing the higher one quietly demotes
a photo somebody rated highly. Its roundRating matches
XmpGenerator.mapRating exactly so the API and an XMP sidecar can never
disagree about how many stars a photo has.

**Put the edit back.** POST /api/v1/events/:id/photos accepts an optional
replaces_photo_id and routes to the existing replacePhoto(), preserving
the photo's id, feedback, comments and position. The plugin stores the
picpeak id on the catalogue photo, so the id survives the editor renaming
the render — which makes it the reliable key, not the filename. Scoped to
the event in the URL: a token inherits its owner's powers across every
event they can see, so an unscoped id would let one gallery overwrite
another's photo.

For renders whose RAW never went through the plugin, findReplacementCandidate
gains an opt-in number_token mode matching on the trailing digit run.
Deliberately the LONGEST run and never a fixed last-N slice: multi-camera
shoots disambiguate by prefixing the camera index into the number
(cam11234.jpg / cam21234.jpg), and a last-4 slice reads 1234 from both
bodies and reintroduces exactly the collision the prefix removes.
Ambiguity is refused, never guessed.

Also drops the multer temp file on the two new early returns — this route
only unlinks in its catch block.

* refactor(api): one rating-rounding rule, and apply match_mode where it counts

Three things the pre-review pass turned up on the round-trip work:

- `match_mode` reached the photo-cap pre-count but not the loop that
  actually picks the replacement target, so asking for `number_token`
  would have been counted and then quietly ignored. Both call sites now
  take it.

- `number_token` matching read `select('*')` over every photo in the
  event to compare one digit run. It now reads the three columns the
  match needs and re-reads the single winner in full, so a 5000-photo
  event doesn't pull 5000 full rows through memory to answer one
  question.

- `XmpGenerator.mapRating` and `markMerge.roundRating` were the same
  five thresholds written twice — the second way to do one thing that
  drifts the moment either is touched. The thresholds now live in
  markMerge and the generator delegates, which is what keeps a sidecar
  and the v1 API from ever disagreeing about a photo's star count.

* fix(api): keep the new route in the generated OpenAPI spec

The `color_labels` description carried an inline JSON example. In an
unquoted YAML scalar `{ "green": 2 }` parses as a flow mapping, so
swagger-jsdoc threw YAMLSemanticError and dropped the WHOLE route from
the spec — visible only as a warning on boot, with the route still
serving normally, which is exactly the kind of failure that survives to
release.

Found by booting a real instance rather than by reading the diff.

* fix(api): close the four blockers from review on #1165

1. Replacing an external photo silently kept serving the old file.
   resolvePhotoStorageKey gives photo.source_origin precedence and
   returns null for 'reference'/'external', so the edit was uploaded,
   the row updated and 200 returned while every viewer kept getting the
   untouched NAS original and the upload sat orphaned. replacePhoto now
   repoints the row to managed and clears external_relpath. The file on
   the share is never touched — this moves the pointer, not the data.

2. Every replacement leaked its temp file. putFromFile COPIES on local
   and uploads on S3; neither consumes the source, and replacePhoto
   never unlinked it — while the v1 route had disabled its own cleanup
   on the belief that replacePhoto moved the file. Cleanup now lives in
   replacePhoto, which closes the admin path too (adminPhotos only
   unlinks in its new-files branch, so replaced files leaked there as
   well). The v1 route also unlinks on the FAILURE path, which returned
   before any cleanup ran.

3. The download-all ZIP is invalidated after a replacement, as
   adminPhotos.js already does. Without it guests kept downloading the
   pre-edit photo indefinitely, which defeats the point of the feature.

4. The round-trip could not see reference or watcher galleries at all.
   fileWatcher and adminExternalMedia never set original_filename — the
   camera name lives in `filename` for those rows — so the backfill and
   the GET fallback both produced NULL for exactly the galleries most
   likely to be driven from Lightroom. The backfill now COALESCEs, both
   ingest paths set source_filename, and the GET falls back to filename.

Concerns:

- number_token no longer reads every photo row in the event per file. A
  LIKE on the digit run narrows the candidate set in SQL first; the
  exact trailing-run check still decides, so semantics are unchanged.
  The token is a regex-extracted digit run, so it cannot carry a
  wildcard.
- The replacement's activity entry is scoped to event.id instead of
  null. The dashboard feed excludes NULL-event rows for scoped callers
  (GHSA-jhcf), so it was vanishing from the audit trail of the
  photographer who owns the event.

Nit: dropped the unused higherPriorityColor export from markMerge.

Three regression tests cover the external repoint, the temp cleanup and
the COALESCE backfill. 21/21 pass.

* chore(migrations): renumber 185 -> 193 after gallery-folders landed

185_add_category_is_folder.js merged to main while this was in review,
so the number the PR reserved is taken and main is now at 192. Knex keys
on filename rather than the prefix, so both would have run — but
picpeakImportService guards restores with migrationOrder(), which parses
that prefix, and two files answering 185 make the forward-only check
pass a backup onto a schema missing its columns.

Renumbered with every reference: the header comment, the test that
requires the path, and the four call-site comments that cite it. The
'migration 182' reference inside it is the colour-labels migration and
is unrelated; gallery.js:1134 cites upstream's 185 and is untouched.

* fix(api): keep external_relpath when a replacement converts the row

The external-photo blocker fix cleared external_relpath along with
flipping source_origin, which closed one hole and opened another.

adminExternalMedia dedupes a re-scan on (event_id, external_relpath)
— routes/adminExternalMedia.js:195 — and migration 186 puts a unique
index on exactly that pair. With the column nulled, the next scan of
the share would not recognise the NAS original as already imported and
would insert it again, so the gallery would end up holding both the
edit and a fresh copy of the file it replaced.

Only source_origin needs to change: it is what resolvePhotoStorageKey
keys on, and every other consumer of external_relpath reads the two
together and lets source_origin decide. The stale relpath on a managed
row is inert for resolution and still correct as a dedupe key.

Test updated to assert the value is kept rather than cleared.

* fix(uploads): say when exiftool is missing instead of blaming the RAW

A server without exiftool reported `No usable embedded preview in RAW
file X.CR3: spawn exiftool ENOENT` for every RAW upload. The headline
describes a corrupt photo; the actual cause is a package that was never
installed, demoted to a trailing detail. It sends people hunting through
their camera files.

Hit while testing the Lightroom round-trip (#745): an export of RAW
originals failed 11 times with that message, and the file was fine.

RAW upload is the only feature that needs exiftool, so an install can be
missing it indefinitely and only find out when someone uploads a CR3 —
which makes the wording the whole diagnosis.

ENOENT now produces a message naming the dependency and the install
command for Debian/Alpine/macOS, and breaks out of the tag loop instead
of spawning the same missing binary twice more to report the last
failure as if it described the photo. A genuinely preview-less RAW still
gets the original message.

Verified both paths by making exiftool unreachable via PATH rather than
mocking: missing tool and unreadable file now report differently.

* fix(external): a delivered edit must win a relpath-fold collision

Follow-up to keeping external_relpath on a replaced photo. Keeping it is
what lets adminExternalMedia still dedupe the folder re-scan, but it also
leaves the row inside externalRelpathFold's sweep — and that sweep does
not merely rewrite paths, it DELETES collision losers via
externalPhotoDedupe.

The survivor was whichever row happened to be claimed first, which is
iteration order. So a replaced photo — source_origin 'managed', holding
the edit the photographer just delivered — could be deleted in favour of
the untouched camera original sitting next to it on the share.

Managed rows now claim first and therefore survive. The external row
that loses is the recoverable one: it is still on the share and a
re-scan re-imports it. The edit is not recoverable.

Note this is deliberately NOT the "skip managed rows in the fold"
shape suggested in review. Skipping would leave those rows holding a
base-relative path while every other row moved to root-relative, so the
scanner — which computes root-relative — would stop matching them and
import the camera original again as a duplicate. That is the exact bug
keeping external_relpath exists to prevent, reintroduced through a
different door. Rebasing them and protecting them from deletion keeps
both properties.
2026-08-28 15:45:56 +02:00

1380 lines
54 KiB
JavaScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
const sharp = require('sharp');
const exifr = require('exifr');
const path = require('path');
const fsp = require('fs').promises;
const os = require('os');
const crypto = require('crypto');
const logger = require('../utils/logger');
const { db } = require('../database/db');
const { getStorage } = require('./storage');
const { execFile } = require('child_process');
const { promisify } = require('util');
const execFileAsync = promisify(execFile);
// Configure sharp for better memory management with large batches
sharp.cache(false); // Disable cache to prevent memory buildup
sharp.concurrency(2); // Limit concurrent operations
// Camera RAW / DNG formats. Sharp's bundled libvips has no raw loader, so these
// can't be fed to sharp() directly — instead we extract the full-resolution JPEG
// preview that every RAW file embeds (via exiftool) and process THAT. Gated
// strictly by extension, so nothing here runs for ordinary jpg/png/webp photos.
const RAW_EXTENSIONS = new Set([
'dng', 'cr2', 'cr3', 'nef', 'nrw', 'arw', 'sr2', 'srf',
'raf', 'rw2', 'orf', 'pef', 'srw', 'raw', '3fr', 'dcr', 'kdc'
]);
function isRawFilename(name) {
if (!name || typeof name !== 'string') return false;
const ext = path.extname(name).toLowerCase().replace(/^\./, '');
return RAW_EXTENSIONS.has(ext);
}
/**
* Extract the embedded full-resolution JPEG preview from a RAW/DNG file to a
* temp .jpg and return its path. Tries the largest previews first
* (JpgFromRaw → PreviewImage → ThumbnailImage). Throws if none can be extracted
* or the result isn't a valid image — the caller treats that as a processing
* failure (photo → 'failed'), same as any unreadable upload.
*/
async function extractRawPreview(rawPath) {
const outDir = await fsp.mkdtemp(path.join(os.tmpdir(), 'picpeak-raw-'));
const outPath = path.join(outDir, `${crypto.randomBytes(4).toString('hex')}.jpg`);
const tags = ['-JpgFromRaw', '-PreviewImage', '-ThumbnailImage'];
let lastErr;
for (const tag of tags) {
try {
// `-b` writes the raw tag bytes to stdout; -w isn't reliable across tags,
// so capture stdout as a buffer and write it ourselves.
const { stdout } = await execFileAsync('exiftool', ['-b', tag, rawPath], {
encoding: 'buffer',
maxBuffer: 256 * 1024 * 1024,
});
if (stdout && stdout.length > 0) {
await fsp.writeFile(outPath, stdout);
// Validate it's a real, decodable image before handing it to the pipeline.
const meta = await sharp(outPath).metadata();
if (meta.width && meta.height) {
return { path: outPath, cleanup: () => fsp.rm(outDir, { recursive: true, force: true }).catch(() => {}) };
}
}
} catch (err) {
lastErr = err;
// exiftool missing is a DEPLOYMENT fault, not a bad file, and it fails
// identically for every tag — so stop rather than retrying the same
// spawn twice more and reporting the last one as if it described the
// photo.
if (err && err.code === 'ENOENT') break;
}
}
await fsp.rm(outDir, { recursive: true, force: true }).catch(() => {});
// Distinguish "the tool isn't installed" from "this file has no preview".
// Both used to surface as `No usable embedded preview in RAW file X:
// spawn exiftool ENOENT`, which reads as a corrupt photo and sends people
// hunting through their RAWs instead of installing a package. RAW upload is
// the only feature that needs exiftool, so an install can be missing it and
// not find out until someone uploads a CR3.
if (lastErr && lastErr.code === 'ENOENT') {
throw new Error(
'exiftool is not installed on the server, and it is required to read '
+ `RAW files (${path.basename(rawPath)}). Install it (Debian/Ubuntu: `
+ 'apt-get install libimage-exiftool-perl, Alpine: apk add exiftool, '
+ 'macOS: brew install exiftool) and retry. JPEG and other ordinary '
+ 'images do not need it.'
);
}
throw new Error(`No usable embedded preview in RAW file ${path.basename(rawPath)}: ${lastErr ? lastErr.message : 'no preview tag returned data'}`);
}
/**
* Give a Sharp-processable local image path for `localPath`. For ordinary
* images it's a pass-through (no cost). For RAW/DNG (by `sourceName` extension)
* it extracts the embedded JPEG preview and returns that, plus the basename to
* use for generated outputs so thumbnails/previews stay named after the source
* rather than the random temp file. Always call `cleanup()` when done.
*/
async function withProcessableImage(localPath, sourceName) {
if (!isRawFilename(sourceName)) {
return { path: localPath, outputBasename: undefined, cleanup: () => {} };
}
const { path: previewPath, cleanup } = await extractRawPreview(localPath);
return { path: previewPath, outputBasename: path.basename(sourceName), cleanup };
}
// Default thumbnail settings
const DEFAULT_THUMBNAIL_WIDTH = 300;
const DEFAULT_THUMBNAIL_HEIGHT = 300;
// 'inside' preserves the source aspect ratio (output ≤ width × height).
// This is the right default for masonry / mosaic / justified layouts —
// the gallery sizes each card from photo.width/height and renders the
// thumbnail with object-cover, so a thumb that already matches the
// source aspect doesn't get re-cropped (#447). Admins who want
// uniform 1:1 grid tiles can switch to 'cover' in the thumbnail settings.
const DEFAULT_THUMBNAIL_FIT = 'inside';
const DEFAULT_THUMBNAIL_QUALITY = 85;
const DEFAULT_THUMBNAIL_FORMAT = 'jpeg';
// Hero image settings - optimized for large displays
const DEFAULT_HERO_WIDTH = 1920;
const DEFAULT_HERO_HEIGHT = 1080;
const DEFAULT_HERO_QUALITY = 85;
// Preview tier (#492). Aspect-preserved downscale for the lightbox so
// guests don't pay the full 512 MB original on every photo open.
// Same long edge as the hero (admins are already sizing for it) and
// quality 85 — JPEG artefacts at this size are imperceptible to clients
// browsing on phones / Retina laptops, and storage cost stays modest
// (~200500 KB per photo vs originals at multi-MB).
const DEFAULT_PREVIEW_LONG_EDGE = 1920;
const DEFAULT_PREVIEW_QUALITY = 85;
// Responsive tiers (#1095). A whitelist, not a free-form ?w=: an open
// parameter lets anyone fill the disk with renditions nobody asked for, and
// every distinct value is a permanent cache entry.
//
// 1920 stays the default so existing preview_path rows keep their meaning and
// nothing regenerates on upgrade. The smaller tiers exist because a phone can
// show ~1170px at most, so the 1920 tier ships roughly twice the bytes it can
// use on every lightbox swipe.
const PREVIEW_WIDTHS = [640, 1280, 1920];
const THUMBNAIL_WIDTHS = [300, 600, 900];
/** Whitelist a requested width, or null. Callers treat null as "use default". */
function normalizeTierWidth(requested, allowed) {
const n = parseInt(requested, 10);
return Number.isFinite(n) && allowed.includes(n) ? n : null;
}
// Helper to parse setting value (handles both JSON-encoded and plain values)
function parseSettingValue(value) {
if (value === null || value === undefined) {
return null;
}
// Try to parse as JSON first (in case it's a JSON-encoded string like '"cover"')
try {
return JSON.parse(value);
} catch (e) {
// If it's not valid JSON, return the raw value
return value;
}
}
// Validate that fit value is valid for Sharp
function validateFitValue(fit) {
const validFitValues = ['cover', 'contain', 'fill', 'inside', 'outside'];
if (fit && validFitValues.includes(fit)) {
return fit;
}
return DEFAULT_THUMBNAIL_FIT;
}
// Get thumbnail settings from database
async function getThumbnailSettings() {
try {
const settings = await db('app_settings')
.whereIn('setting_key', [
'thumbnail_width',
'thumbnail_height',
'thumbnail_fit',
'thumbnail_quality',
'thumbnail_format'
])
.select('setting_key', 'setting_value');
const settingsMap = {};
settings.forEach(s => {
settingsMap[s.setting_key] = parseSettingValue(s.setting_value);
});
// Parse and validate fit value
const fitValue = validateFitValue(settingsMap.thumbnail_fit);
return {
width: parseInt(settingsMap.thumbnail_width) || DEFAULT_THUMBNAIL_WIDTH,
height: parseInt(settingsMap.thumbnail_height) || DEFAULT_THUMBNAIL_HEIGHT,
fit: fitValue,
quality: parseInt(settingsMap.thumbnail_quality) || DEFAULT_THUMBNAIL_QUALITY,
format: settingsMap.thumbnail_format || DEFAULT_THUMBNAIL_FORMAT
};
} catch (error) {
// If database is not ready or settings don't exist, use defaults
logger.warn('Could not fetch thumbnail settings, using defaults:', error.message);
return {
width: DEFAULT_THUMBNAIL_WIDTH,
height: DEFAULT_THUMBNAIL_HEIGHT,
fit: DEFAULT_THUMBNAIL_FIT,
quality: DEFAULT_THUMBNAIL_QUALITY,
format: DEFAULT_THUMBNAIL_FORMAT
};
}
}
const contentTypeFor = (format) => {
if (format === 'png') return 'image/png';
if (format === 'webp') return 'image/webp';
return 'image/jpeg';
};
/**
* Generate a thumbnail from a local source image path. The output is written
* to the storage backend (local fs or S3) under `thumbnails/thumb_<filename>`
* and the relative storage key is returned for DB persistence.
*
* Callers must ensure the source is on the local filesystem. For S3 mode
* regeneration flows, fetch via `withLocalCopy(storage, sourceKey, fn)` first.
*
* options.outputBasename — override the basename portion of the thumbnail
* filename (default: basename of imagePath). Used for external/reference
* photos where the source basename can collide across events (#423) — the
* import path passes a per-photo unique basename so two events both
* referencing `IMG_0001.jpg` don't clobber each other's thumbnail.
*/
async function generateThumbnail(imagePath, options = {}) {
const sourceBasename = path.basename(imagePath);
const outputBasename = options.outputBasename || sourceBasename;
const storage = getStorage();
// Get thumbnail settings
const settings = await getThumbnailSettings();
// Tag against the CONFIGURED canonical width, not the 300 default — the tag
// has to agree with the key ensureThumbnailAtWidth probed for. On an install
// with thumbnail_width=600 a w=300 request used to write `thumb_<name>` while
// the caller looked for `thumb_w300_<name>`: the cache never hit, so every
// single request re-downloaded the original and ran Sharp, and the file it
// left behind was in no cleanup list.
const canonicalWidth = settings.width || DEFAULT_THUMBNAIL_WIDTH;
const widthTag = options.width && options.width !== canonicalWidth
? `w${options.width}_`
: '';
const thumbnailFilename = `thumb_${widthTag}${outputBasename}`;
const thumbnailRelKey = path.posix.join('thumbnails', thumbnailFilename);
// `options.regenerate` deliberately does NOT delete the existing object
// first (#1129).
//
// It used to, and the delete ran BEFORE sharp had even opened the source —
// so a source that could not be read (a NAS mount that blipped, a corrupt
// file) left the old thumbnail already gone and returned null, with the
// database still pointing at it. One bulk regeneration during a mount outage
// could therefore strip every canonical thumbnail in a reference gallery and
// leave the whole library serving 404s.
//
// Nothing is lost by dropping it: LocalFsStorage.put stages to a temp file
// and renames over the target, which replaces atomically, and an S3 put
// overwrites by key. So the write replaces the old rendition either way —
// the only thing the delete added was a window in which there was no
// thumbnail at all.
try {
// First, verify the source image is complete and valid
const metadata = await sharp(imagePath).metadata();
if (!metadata.width || !metadata.height) {
throw new Error('Invalid image metadata - file may be incomplete');
}
let sharpInstance = sharp(imagePath, {
limitInputPixels: 268402689, // ~16k x 16k max
sequentialRead: true,
failOn: 'none'
});
// Apply EXIF orientation before resizing (#1185). Without this a photo
// whose Orientation tag is not 1 — routine for portrait shots on bodies
// that tag rather than rotate the sensor data — is resized from the raw
// pixels and comes out sideways. `.withMetadata(false)` below then strips
// the tag, so the browser has no hint left to correct it either, which is
// why this cannot be left to the client.
//
// Unconditional, unlike resizeToBox and generatePreviewImage. Those open
// multi-frame sources with `animated: true` and must not rotate them,
// because `.rotate()` flattens to the first frame. This one never passes
// that option, so it already produces a still — guarding on `pages` here
// would protect an animation that was being discarded anyway, and leave
// the thumbnail in raw orientation while the stored dimensions describe
// the rotated one.
sharpInstance = sharpInstance.rotate();
// Strip EXIF/metadata from thumbnails (privacy: prevent GPS leak etc.)
sharpInstance = sharpInstance.withMetadata(false);
// options.width/height override the admin setting for responsive tiers
// (#1095). The configured `fit` is kept deliberately: the grid renders
// with object-cover, so every tier must be cropped the same way or the
// browser would swap between differently-framed images as the viewport
// changes.
sharpInstance = sharpInstance.resize(options.width || settings.width, options.height || settings.height, {
withoutEnlargement: true,
fit: settings.fit,
position: 'center'
});
if (settings.format === 'jpeg') {
sharpInstance = sharpInstance.jpeg({
quality: settings.quality,
progressive: true,
mozjpeg: true
});
} else if (settings.format === 'png') {
sharpInstance = sharpInstance.png({
quality: settings.quality,
compressionLevel: 9,
progressive: true
});
} else if (settings.format === 'webp') {
sharpInstance = sharpInstance.webp({
quality: settings.quality,
effort: 4
});
}
const buffer = await sharpInstance.toBuffer();
if (!buffer || buffer.length === 0) {
throw new Error('Generated thumbnail is empty');
}
await storage.put(thumbnailRelKey, buffer, { contentType: contentTypeFor(settings.format) });
return thumbnailRelKey;
} catch (error) {
const msg = (error && error.message) ? error.message : String(error);
logger.error(`Failed to generate thumbnail for ${sourceBasename}: ${msg}`);
// No cleanup delete here either, for the same reason as above (#1129).
// This was "clean up any partially uploaded object", but there cannot be
// one: `storage.put` is the last statement in the try, every throw above
// it happens before anything is written, and put itself stages to a temp
// file and only renames on success. So the only object this delete could
// ever have removed is the PREVIOUS, perfectly good rendition — which is
// exactly the thumbnail a failed regeneration must leave alone.
return null;
}
}
/**
* Check if a thumbnail exists and is valid. For local-fs storage we open the
* file with sharp to confirm it parses; for S3 we trust the byte-integrity
* checks built into the protocol and only verify size > 0.
*/
async function isThumbnailValid(thumbnailPath) {
const storage = getStorage();
try {
const stat = await storage.stat(thumbnailPath);
if (!stat || stat.size === 0) {
return false;
}
if (storage.kind() === 'local') {
const localPath = storage.resolveLocalPath(thumbnailPath);
await sharp(localPath).metadata();
}
return true;
} catch (error) {
return false;
}
}
/**
* The dimensions a viewer actually sees, given EXIF orientation (#1185).
*
* sharp reports `metadata.width`/`height` as the pixels are stored, not as
* they are displayed. Orientation values 5-8 carry a 90° rotation, so for
* those the two are swapped — which is why a portrait photo from a body that
* tags rather than rotates was landing in the database as landscape, and why
* masonry and justified layouts sized its tile with the wrong aspect ratio on
* top of the image itself being unrotated.
*
* Everything that renders these photos now applies `.rotate()`, so the stored
* numbers have to describe the rotated result to match.
*
* @param {Object} metadata - a sharp metadata object
* @returns {{ width: number|null, height: number|null }}
*/
/**
* Does this image's EXIF orientation mean `.rotate()` will move its pixels?
* (#1198)
*
* Distinct from orientedDimensions, and the distinction matters. Orientations
* 2, 3 and 4 are a mirror, a 180° turn and a mirrored 180° turn: every pixel
* moves, but width and height are unchanged. A square image with 5-8 is the
* same story. So "did the dimensions change?" is not the same question as "was
* this image transformed", and anything keyed to derived data — face bounding
* boxes, cached previews — has to ask the second one or it silently skips
* exactly those cases.
*
* 1 means no transform. Absent means no tag, which is also no transform.
*
* @param {Object} metadata - a sharp metadata object
* @returns {boolean}
*/
function hasOrientationTransform(metadata) {
const o = metadata && metadata.orientation;
return typeof o === 'number' && o >= 2 && o <= 8;
}
function orientedDimensions(metadata) {
if (!metadata || !metadata.width || !metadata.height) return { width: null, height: null };
const swap = metadata.orientation >= 5 && metadata.orientation <= 8;
return {
width: swap ? metadata.height : metadata.width,
height: swap ? metadata.width : metadata.height,
};
}
/**
* Wraps a callback that needs the source image as a local file. In local-fs
* mode the storage path is used directly (no copy); in S3 mode the object is
* streamed to a tmp file which is removed afterwards.
*/
async function withLocalCopy(sourceKey, fn) {
const storage = getStorage();
if (storage.kind() === 'local') {
return fn(storage.resolveLocalPath(sourceKey));
}
const tmpDir = await fsp.mkdtemp(path.join(os.tmpdir(), 'picpeak-src-'));
const tmpPath = path.join(tmpDir, `${crypto.randomBytes(4).toString('hex')}_${path.basename(sourceKey)}`);
try {
await storage.getToFile(sourceKey, tmpPath);
return await fn(tmpPath);
} finally {
await fsp.rm(tmpDir, { recursive: true, force: true }).catch(() => {});
}
}
/**
* Regenerate thumbnail if it's broken or missing.
*
* Works for both managed photos (stored via the storage backend, possibly
* S3) and external/reference photos (#423 — sourced from a local mount
* outside the managed storage tree, e.g. NAS over SMB/NFS). External
* photos historically had thumbnail_path=null, which forced the gallery
* to fall back to streaming the full original on every tile — minutes of
* load time for a 100-photo NAS-mounted gallery.
*/
async function ensureThumbnail(photo) {
const { resolvePhotoStorageKey, resolvePhotoFilePath } = require('./photoResolver');
const event = await db('events').where('id', photo.event_id).first();
if (!event) {
logger.error(`ensureThumbnail: event ${photo.event_id} not found for photo ${photo.id}`);
return null;
}
// Check if thumbnail exists and is valid (works for any source).
if (photo.thumbnail_path) {
const isValid = await isThumbnailValid(photo.thumbnail_path);
if (isValid) {
return photo.thumbnail_path;
}
logger.warn(`Invalid thumbnail detected for photo ${photo.id}, regenerating...`);
}
const isExternal = photo.source_origin === 'external' || photo.source_origin === 'reference';
let newThumbnailPath;
if (isExternal) {
// External: source is on a local mount path. No withLocalCopy needed
// (storage-backend abstraction doesn't apply — this is a direct fs
// read). Use a per-photo unique outputBasename so two events both
// referencing the same NAS basename can't clobber each other's thumb.
let localPath;
try {
localPath = resolvePhotoFilePath(event, photo);
} catch (e) {
logger.error(`Failed to resolve external file for thumbnail (photo ${photo.id}): ${e.message}`);
return null;
}
const sourceBasename = path.basename(photo.external_relpath || photo.filename || `photo-${photo.id}`);
const outputBasename = `ext${photo.id}_${sourceBasename}`;
logger.info(`Ensuring thumbnail for external photo ${photo.id} from ${localPath}`);
newThumbnailPath = await generateThumbnail(localPath, { regenerate: true, outputBasename });
} else {
let sourceKey;
try {
sourceKey = resolvePhotoStorageKey(event, photo);
} catch (e) {
logger.error(`Failed to resolve original key for thumbnail (photo ${photo.id}): ${e.message}`);
return null;
}
logger.info(`Ensuring thumbnail for photo ${photo.id} from key: ${sourceKey}`);
newThumbnailPath = await withLocalCopy(sourceKey, async (localPath) => {
const proc = await withProcessableImage(localPath, sourceKey);
try {
return await generateThumbnail(proc.path, { regenerate: true, outputBasename: proc.outputBasename });
} finally {
await proc.cleanup();
}
});
}
if (newThumbnailPath) {
await db('photos')
.where({ id: photo.id })
.update({ thumbnail_path: newThumbnailPath });
logger.info(`Regenerated thumbnail for photo ${photo.id}`);
return newThumbnailPath;
}
return null;
}
async function generateVideoPlaceholder(originalFilename, options = {}) {
const parsed = path.parse(originalFilename || '');
const baseName = parsed.name || 'video';
const thumbnailFilename = `thumb_${baseName}.jpg`;
const thumbnailRelKey = path.posix.join('thumbnails', thumbnailFilename);
const storage = getStorage();
const settings = await getThumbnailSettings();
const width = settings.width || DEFAULT_THUMBNAIL_WIDTH;
const height = settings.height || DEFAULT_THUMBNAIL_HEIGHT;
if (options.regenerate) {
await storage.delete(thumbnailRelKey).catch(() => {});
}
try {
const svg = `
<svg width="${width}" height="${height}" viewBox="0 0 ${width} ${height}" xmlns="http://www.w3.org/2000/svg">
<defs>
<linearGradient id="grad" x1="0%" y1="0%" x2="100%" y2="100%">
<stop offset="0%" stop-color="#0f172a" stop-opacity="0.9"/>
<stop offset="100%" stop-color="#1e293b" stop-opacity="0.9"/>
</linearGradient>
</defs>
<rect width="${width}" height="${height}" rx="18" fill="url(#grad)"/>
<circle cx="${width / 2}" cy="${height / 2}" r="${Math.min(width, height) / 6}" fill="rgba(255,255,255,0.85)"/>
<polygon points="${width / 2 - 10},${height / 2 - 14} ${width / 2 - 10},${height / 2 + 14} ${width / 2 + 16},${height / 2}" fill="#0f172a"/>
<text x="50%" y="${height - 18}" font-family="Arial, sans-serif" font-size="16" fill="rgba(255,255,255,0.9)" text-anchor="middle">
VIDEO
</text>
</svg>
`;
const buffer = await sharp(Buffer.from(svg))
.resize(width, height, { fit: 'cover' })
.jpeg({ quality: settings.quality || DEFAULT_THUMBNAIL_QUALITY })
.toBuffer();
await storage.put(thumbnailRelKey, buffer, { contentType: 'image/jpeg' });
return thumbnailRelKey;
} catch (error) {
logger.error('Failed to generate video placeholder thumbnail:', error.message);
return null;
}
}
/**
* Generate a hero-optimized image for gallery headers
* Outputs a 1920x1080 image suitable for full-width hero sections
*/
async function generateHeroImage(imagePath, options = {}) {
const filename = options.outputBasename || path.basename(imagePath);
const heroFilename = `hero_${filename}`;
const heroRelKey = path.posix.join('heroes', heroFilename);
const storage = getStorage();
if (options.regenerate) {
await storage.delete(heroRelKey).catch(() => {});
}
try {
const metadata = await sharp(imagePath).metadata();
if (!metadata.width || !metadata.height) {
throw new Error('Invalid image metadata - file may be incomplete');
}
const heroWidth = options.width || DEFAULT_HERO_WIDTH;
const heroHeight = options.height || DEFAULT_HERO_HEIGHT;
const quality = options.quality || DEFAULT_HERO_QUALITY;
let sharpInstance = sharp(imagePath, {
limitInputPixels: 268402689,
sequentialRead: true,
failOn: 'none'
});
// EXIF orientation, same reasoning as generateThumbnail (#1185) — and
// unconditional for the same reason: no `animated: true` on the input, so
// this output is a still whatever the source was.
sharpInstance = sharpInstance.rotate();
// Strip EXIF/metadata from hero images (privacy: prevent GPS leak etc.)
sharpInstance = sharpInstance.withMetadata(false);
sharpInstance = sharpInstance.resize(heroWidth, heroHeight, {
withoutEnlargement: false,
fit: 'cover',
position: 'center'
});
sharpInstance = sharpInstance.jpeg({
quality: quality,
progressive: true,
mozjpeg: true
});
const buffer = await sharpInstance.toBuffer();
if (!buffer || buffer.length === 0) {
throw new Error('Generated hero image is empty');
}
await storage.put(heroRelKey, buffer, { contentType: 'image/jpeg' });
logger.info(`Generated hero image for ${filename}${heroRelKey}`);
return heroRelKey;
} catch (error) {
const msg = (error && error.message) ? error.message : String(error);
logger.error(`Failed to generate hero image for ${filename}: ${msg}`);
await storage.delete(heroRelKey).catch(() => {});
return null;
}
}
/**
* Check if a hero image exists and is valid
*/
async function isHeroValid(heroPath) {
const storage = getStorage();
try {
const stat = await storage.stat(heroPath);
if (!stat || stat.size === 0) {
return false;
}
if (storage.kind() === 'local') {
const localPath = storage.resolveLocalPath(heroPath);
await sharp(localPath).metadata();
}
return true;
} catch (error) {
return false;
}
}
/**
* Ensure a hero image exists for a photo, regenerate if needed
*/
async function ensureHeroImage(photo) {
const { resolvePhotoStorageKey, resolvePhotoFilePath } = require('./photoResolver');
let event;
try {
event = await db('events').where('id', photo.event_id).first();
} catch (e) {
logger.error(`Failed to load event for hero image (photo ${photo.id}): ${e.message}`);
return null;
}
if (photo.hero_path) {
const isValid = await isHeroValid(photo.hero_path);
if (isValid) {
return photo.hero_path;
}
logger.warn(`Invalid hero image detected for photo ${photo.id}, regenerating...`);
}
// External sources never reach the managed backend, so resolvePhotoStorageKey
// returns null for them by design — and this function used to feed that null
// straight to withLocalCopy, which throws, so the hero route fell back to
// redirecting at the full original. #1078 fixed exactly this for
// ensurePreviewImage and nobody carried it across; it only became visible
// when the Story hero started asking for hero_url instead of the original
// (#1166), which on a reference-mode gallery quietly changed nothing.
const isExternal = photo.source_origin === 'external' || photo.source_origin === 'reference';
let newHeroPath;
if (isExternal) {
// Mirrors the external branch in ensurePreviewImage: a direct fs read off
// the mount, so no withLocalCopy, and a per-photo outputBasename so two
// events referencing the same NAS basename cannot clobber each other.
let localPath;
try {
localPath = resolvePhotoFilePath(event, photo);
} catch (e) {
logger.error(`Failed to resolve external file for hero image (photo ${photo.id}): ${e.message}`);
return null;
}
const sourceBasename = path.basename(photo.external_relpath || photo.filename || `photo-${photo.id}`);
newHeroPath = await generateHeroImage(localPath, {
regenerate: true,
outputBasename: `ext${photo.id}_${sourceBasename}`,
});
if (newHeroPath) {
await db('photos').where({ id: photo.id }).update({ hero_path: newHeroPath });
}
return newHeroPath;
}
let sourceKey;
try {
sourceKey = resolvePhotoStorageKey(event, photo);
logger.info(`Ensuring hero image for photo ${photo.id} from key: ${sourceKey}`);
} catch (e) {
const msg = (e && e.message) ? e.message : String(e);
logger.error(`Failed to resolve original key for hero image (photo ${photo.id}): ${msg}`);
return null;
}
if (!sourceKey) {
// Reference-mode event holding a row with no source_origin: the mode falls
// back to the event's and resolvePhotoStorageKey returns null. Honour the
// null-on-failure contract rather than feeding it to withLocalCopy.
logger.warn(`No managed storage key for hero image (photo ${photo.id}); skipping generation`);
return null;
}
newHeroPath = await withLocalCopy(sourceKey, async (localPath) => {
const proc = await withProcessableImage(localPath, sourceKey);
try {
return await generateHeroImage(proc.path, { regenerate: true, outputBasename: proc.outputBasename });
} finally {
await proc.cleanup();
}
});
if (newHeroPath) {
await db('photos')
.where({ id: photo.id })
.update({ hero_path: newHeroPath });
logger.info(`Regenerated hero image for photo ${photo.id}`);
return newHeroPath;
}
return null;
}
/**
* Generate a lightbox preview image (#492).
*
* Aspect-preserving downscale (`fit: 'inside'`) capped at
* DEFAULT_PREVIEW_LONG_EDGE. Distinct from generateHeroImage:
* - hero → 1920x1080 cover-cropped (gallery hero header banner)
* - preview → ≤1920px long edge, aspect preserved (lightbox tile)
*
* Output to `previews/preview_<filename>` so an admin who flips the
* setting back off can wipe the folder cleanly without touching
* thumbnails or heroes.
*
* ENCODING follows the source, it is not always JPEG. JPEG has no alpha
* channel and no second frame, so encoding everything as JPEG flattened a
* transparent PNG onto a solid background and reduced an animated GIF to its
* first frame — for every consumer of this tier: the lightbox, the slideshow,
* admin previews, face avatars. Sources with alpha or more than one page are
* encoded as WebP instead, which carries both and is still far smaller than
* the original.
*
* The output extension is rewritten to match what was actually written.
* Previously the source basename was kept verbatim, so a PNG source produced
* `preview_foo.png` holding JPEG bytes — harmless while the route hard-coded
* image/jpeg, and actively wrong now that the encoding varies. Old keys keep
* working: they are still JPEG and still served as such.
*/
async function generatePreviewImage(imagePath, options = {}) {
const filename = options.outputBasename || path.basename(imagePath);
// Non-default tiers get their own key so they cannot collide with the
// canonical preview the DB column points at.
const widthTag = options.longEdge && options.longEdge !== DEFAULT_PREVIEW_LONG_EDGE
? `w${options.longEdge}_`
: '';
const storage = getStorage();
// Probed BEFORE the key is built: the extension has to match the encoding,
// and the encoding depends on what the source turns out to be.
let probe;
try {
probe = await sharp(imagePath).metadata();
} catch (error) {
const msg = (error && error.message) ? error.message : String(error);
logger.error(`Failed to read metadata for preview of ${filename}: ${msg}`);
return null;
}
const isAnimated = (probe.pages || 1) > 1;
const needsWebp = isAnimated || probe.hasAlpha === true;
const base = filename.replace(/\.[^./\\]+$/, '');
const previewFilename = `preview_${widthTag}${base}.${needsWebp ? 'webp' : 'jpg'}`;
const previewRelKey = path.posix.join('previews', previewFilename);
if (options.regenerate) {
await storage.delete(previewRelKey).catch(() => {});
}
try {
const metadata = probe;
if (!metadata.width || !metadata.height) {
throw new Error('Invalid image metadata - file may be incomplete');
}
const longEdge = options.longEdge || DEFAULT_PREVIEW_LONG_EDGE;
const quality = options.quality || DEFAULT_PREVIEW_QUALITY;
let sharpInstance = sharp(imagePath, {
limitInputPixels: 268402689, // ~16k x 16k max
sequentialRead: true,
failOn: 'none',
// Without this an animated source is opened as its first frame only, and
// every later frame is discarded before the resize ever sees it.
// limitInputPixels still applies, and sharp counts an animated input as
// width x (height x pages) — so a pathological GIF is rejected rather
// than decoded, and the caller falls back to the original.
animated: isAnimated,
});
// EXIF orientation (#1185). Guarded here and not in the thumbnail/hero
// generators because this one DOES open multi-frame sources with
// `animated: true` above, and `.rotate()` would flatten them to a single
// frame — trading an animation for an orientation.
//
// Which leaves one corner unsolved: a multi-frame source that also carries
// an orientation tag keeps its raw orientation here while the thumbnail
// and the stored dimensions describe the rotated one. GIF has no EXIF at
// all and animated WebP effectively never sets it, so this is a real gap
// rather than a common one, and closing it properly means rotating frame
// by frame rather than dropping the animation.
if (!isAnimated) {
sharpInstance = sharpInstance.rotate();
}
// Strip EXIF — same privacy reasoning as thumbnails/heroes.
sharpInstance = sharpInstance.withMetadata(false);
// fit: 'inside' + withoutEnlargement keeps small originals at
// their native size (no upscaling artefacts) and shrinks larger
// ones until both dimensions fit inside longEdge×longEdge.
sharpInstance = sharpInstance.resize(longEdge, longEdge, {
withoutEnlargement: true,
fit: 'inside',
});
sharpInstance = needsWebp
? sharpInstance.webp({ quality })
: sharpInstance.jpeg({ quality, progressive: true, mozjpeg: true });
const buffer = await sharpInstance.toBuffer();
if (!buffer || buffer.length === 0) {
throw new Error('Generated preview image is empty');
}
await storage.put(previewRelKey, buffer, {
contentType: needsWebp ? 'image/webp' : 'image/jpeg',
});
logger.info(`Generated preview image for ${filename}${previewRelKey}`);
return previewRelKey;
} catch (error) {
const msg = (error && error.message) ? error.message : String(error);
logger.error(`Failed to generate preview image for ${filename}: ${msg}`);
await storage.delete(previewRelKey).catch(() => {});
return null;
}
}
/**
* Validate an existing preview file is non-empty + readable by Sharp.
* Mirrors isHeroValid / isThumbnailValid.
*/
async function isPreviewValid(previewPath) {
const storage = getStorage();
try {
const stat = await storage.stat(previewPath);
if (!stat || stat.size === 0) return false;
if (storage.kind() === 'local') {
const localPath = storage.resolveLocalPath(previewPath);
await sharp(localPath).metadata();
}
return true;
} catch {
return false;
}
}
/**
* Lazy-generate the preview image for a photo if missing or invalid.
* Returns the storage key or null on failure (callers fall back to
* the original URL so the lightbox never shows a broken image).
*
* Handles both managed photos (via the storage backend, possibly S3) and
* external/reference photos (#1078 — sourced from a local mount outside the
* managed storage tree). Externals used to have no branch here at all:
* resolvePhotoStorageKey returns null for them by design, that null reached
* withLocalCopy, and the throw put every lightbox open back on the full-size
* original — the exact cost the preview tier (#492) exists to avoid.
*/
/**
* A preview at a specific tier width (#1095).
*
* Deliberately separate from ensurePreviewImage rather than a parameter on it.
* That function owns photos.preview_path — one column, one canonical rendition
* — and threading a width through it would either overwrite that column with
* whatever size was asked for last, or need a column per tier. Extra tiers are
* pure cache instead: keyed by width, looked up in storage, generated on miss,
* never written to the row.
*
* Returns null on anything unexpected so callers fall back to the default
* tier, which is always the honest thing to serve.
*/
/**
* Storage keys for every responsive tier of a photo (#1095).
*
* Tiers live outside photos.preview_path deliberately — that column owns the
* canonical rendition — but that also means nothing else knows they exist.
* Delete, bulk-delete, archive and regenerate all operate on preview_path
* alone, so without this the tiers survive their own photo: orphaned on disk
* forever after a delete, and served stale forever after a regenerate.
*
* Derived rather than tracked: the key scheme is deterministic, so there is
* nothing to keep in sync and no migration.
*/
function previewTierKeys(photo) {
if (!photo) return [];
const isExternal = photo.source_origin === 'external' || photo.source_origin === 'reference';
const sourceBasename = path.basename(
(isExternal ? (photo.external_relpath || photo.filename) : photo.path) || `photo-${photo.id}`
);
const outputBasename = `p${photo.id}_${sourceBasename}`;
return PREVIEW_WIDTHS
.filter((w) => w !== DEFAULT_PREVIEW_LONG_EDGE)
.map((w) => path.posix.join('previews', `preview_w${w}_${outputBasename}`));
}
/** Best-effort removal of every responsive tier for a photo. */
async function deletePreviewTiers(photo) {
const storage = getStorage();
await Promise.all(previewTierKeys(photo).map((k) => storage.delete(k).catch(() => {})));
}
/**
* Thumbnail storage keys for every responsive tier of a photo (#1095).
* Mirrors previewTierKeys — see there for why they are derived rather than
* tracked.
*
* Every width is listed, the canonical one included, and deliberately: which
* width is canonical depends on the thumbnail_width setting, so on a
* 600-configured install it is w300 that exists as a tier file. Reading the
* setting here would make the whole cleanup path async for no gain — deleting
* a key that was never written is already a swallowed no-op, so the inclusive
* list is both simpler and the one that cannot strand a file.
*/
function thumbnailTierKeys(photo) {
if (!photo) return [];
const isExternal = photo.source_origin === 'external' || photo.source_origin === 'reference';
const sourceBasename = path.basename(
(isExternal ? (photo.external_relpath || photo.filename) : photo.path) || `photo-${photo.id}`
);
const outputBasename = `p${photo.id}_${sourceBasename}`;
return THUMBNAIL_WIDTHS
.map((w) => path.posix.join('thumbnails', `thumb_w${w}_${outputBasename}`));
}
async function deleteThumbnailTiers(photo) {
const storage = getStorage();
await Promise.all(thumbnailTierKeys(photo).map((k) => storage.delete(k).catch(() => {})));
}
/**
* A thumbnail at a specific tier width (#1095).
*
* Same contract as ensurePreviewImageAtWidth: pure cache, keyed by width,
* never written to photos.thumbnail_path. The key is scoped by photo id for
* every source type — basenames are not unique across events, and a tier is
* served from a cache hit without re-reading the source, so an unscoped key
* would hand one gallery's photo to another.
*/
/**
* Tier storage key -> the in-flight generation for it (#1128). Module scope so
* every concurrent request for one tile shares a single Sharp pass.
*/
const inFlightThumbnailTiers = new Map();
async function ensureThumbnailAtWidth(photo, width) {
if (!width) return ensureThumbnail(photo);
// Against the CONFIGURED canonical width, not the 300 default. An install
// that set thumbnail_width to 600 already has a 600px thumbnail; generating
// a w600 tier for it would download the original and run Sharp to produce a
// byte-equivalent duplicate, once per photo.
const settings = await getThumbnailSettings();
const canonicalWidth = settings.width || DEFAULT_THUMBNAIL_WIDTH;
if (width === canonicalWidth) return ensureThumbnail(photo);
// Videos never take the tier path. Their thumbnail is a poster frame from
// videoProcessor, not a resize of the stored file, so the code below would
// hand the video itself to Sharp — after withLocalCopy has downloaded the
// whole thing on an S3 backend. Nothing caches that failure, so a crawler
// walking ?w= over a gallery of videos repeats the download every request.
if (photo.media_type === 'video' || String(photo.mime_type || '').startsWith('video/')) {
return ensureThumbnail(photo);
}
const { resolvePhotoStorageKey, resolvePhotoFilePath } = require('./photoResolver');
const storage = getStorage();
let event;
try {
event = await db('events').where('id', photo.event_id).first();
} catch (e) {
return null;
}
if (!event) return null;
const isExternal = photo.source_origin === 'external' || photo.source_origin === 'reference';
const sourceBasename = path.basename(
(isExternal ? (photo.external_relpath || photo.filename) : photo.path) || `photo-${photo.id}`
);
const outputBasename = `p${photo.id}_${sourceBasename}`;
const key = path.posix.join('thumbnails', `thumb_w${width}_${outputBasename}`);
try {
if (await storage.stat(key)) return key;
} catch (e) {
// regenerate below
}
// Scale the height from the configured aspect ratio rather than forcing a
// square. Thumbnails are square on a default install, but the settings API
// accepts any width/height in 50..1000 — and with fit:'cover' a 300x200
// canonical next to a 600x600 tier are two different crops, so the photo
// would visibly reframe as the tile size changes.
const height = Math.round(width * (settings.height / canonicalWidth));
// One generation per tier key, however many tiles ask for it (#1128).
//
// A grid issues one request per tile simultaneously, and on a cold gallery
// every one of them misses the stat above. Without this each would run its
// own Sharp pass over the same source — and for an external photo, re-read
// the whole original off the NFS mount to do it. 79 tiles meant 79 decodes
// of the same file, which is also what made the delete race easy to hit.
//
// Per-process only. Two pods still generate independently, which is
// harmless: the write ends in an atomic rename, so they converge on
// byte-identical output.
const pending = inFlightThumbnailTiers.get(key);
if (pending) return pending;
const work = (async () => {
try {
// NOT `regenerate: true` (#1128). This path is only reached on a cache
// MISS, so there is nothing to regenerate — but that flag makes
// generateThumbnail open by DELETING the target. Request A publishes the
// tier, B stats it and heads for storage.get(), and C — still inside
// generation from its own earlier miss — unlinks the file B is about to
// open. B's lazy ReadStream then raised an ENOENT nothing was listening
// for and Node exited.
//
// Without the flag the write is a plain put: LocalFsStorage stages to a
// temp file and renames, which is atomic, so a concurrent reader sees
// either the old file or the new one and never a hole.
if (isExternal) {
const localPath = resolvePhotoFilePath(event, photo);
return await generateThumbnail(localPath, { outputBasename, width, height });
}
const sourceKey = resolvePhotoStorageKey(event, photo);
if (!sourceKey) return null;
return await withLocalCopy(sourceKey, async (localPath) => {
const proc = await withProcessableImage(localPath, sourceKey);
try {
return await generateThumbnail(proc.path, { outputBasename, width, height });
} finally {
proc.cleanup();
}
});
} catch (e) {
logger.warn(`Thumbnail tier w${width} failed for photo ${photo.id}: ${e.message}`);
return null;
}
})();
inFlightThumbnailTiers.set(key, work);
try {
return await work;
} finally {
// In a finally so a rejection cannot poison the key for the process
// lifetime — the next request re-attempts rather than adopting a failure.
inFlightThumbnailTiers.delete(key);
}
}
async function ensurePreviewImageAtWidth(photo, width) {
if (!width || width === DEFAULT_PREVIEW_LONG_EDGE) return ensurePreviewImage(photo);
const { resolvePhotoStorageKey, resolvePhotoFilePath } = require('./photoResolver');
const storage = getStorage();
let event;
try {
event = await db('events').where('id', photo.event_id).first();
} catch (e) {
return null;
}
if (!event) return null;
const isExternal = photo.source_origin === 'external' || photo.source_origin === 'reference';
const sourceBasename = path.basename(
(isExternal ? (photo.external_relpath || photo.filename) : photo.path) || `photo-${photo.id}`
);
// ALWAYS scoped by photo id, managed rows included. Basenames are not unique
// across events — two galleries can each hold an IMG_0001.jpg — and because a
// tier is served straight from a cache hit without re-reading the source, a
// collision hands one gallery's photo to another. Scoping by id is what makes
// the cache safe to trust; it is not a tidiness choice.
const outputBasename = `p${photo.id}_${sourceBasename}`;
const key = path.posix.join('previews', `preview_w${width}_${outputBasename}`);
// Cache hit: nothing to do. This is the common path once a gallery has been
// browsed at a given size.
try {
if (await storage.stat(key)) return key;
} catch (e) {
// fall through and regenerate
}
try {
if (isExternal) {
const localPath = resolvePhotoFilePath(event, photo);
return await generatePreviewImage(localPath, {
regenerate: true, outputBasename, longEdge: width,
});
}
const sourceKey = resolvePhotoStorageKey(event, photo);
if (!sourceKey) return null;
return await withLocalCopy(sourceKey, async (localPath) => {
const proc = await withProcessableImage(localPath, sourceKey);
try {
// outputBasename, not proc.outputBasename: the RAW path returns the
// source basename, which would drop the photo-id scoping above and
// reintroduce the cross-gallery collision.
return await generatePreviewImage(proc.path, {
regenerate: true,
outputBasename,
longEdge: width,
});
} finally {
proc.cleanup();
}
});
} catch (e) {
logger.warn(`Preview tier w${width} failed for photo ${photo.id}: ${e.message}`);
return null;
}
}
async function ensurePreviewImage(photo) {
const { resolvePhotoStorageKey, resolvePhotoFilePath } = require('./photoResolver');
let event;
try {
event = await db('events').where('id', photo.event_id).first();
} catch (e) {
const msg = (e && e.message) ? e.message : String(e);
logger.error(`Failed to load event for preview (photo ${photo.id}): ${msg}`);
return null;
}
if (!event) {
logger.error(`ensurePreviewImage: event ${photo.event_id} not found for photo ${photo.id}`);
return null;
}
if (photo.preview_path) {
const ok = await isPreviewValid(photo.preview_path);
if (ok) return photo.preview_path;
logger.warn(`Invalid preview detected for photo ${photo.id}, regenerating…`);
}
const isExternal = photo.source_origin === 'external' || photo.source_origin === 'reference';
let newPreviewPath;
if (isExternal) {
// Mirrors ensureThumbnail's external branch: the source is a direct fs
// read off the mount, so no withLocalCopy. The per-photo outputBasename
// keeps two events that reference the same NAS basename from clobbering
// each other's preview.
let localPath;
try {
localPath = resolvePhotoFilePath(event, photo);
} catch (e) {
logger.error(`Failed to resolve external file for preview (photo ${photo.id}): ${e.message}`);
return null;
}
const sourceBasename = path.basename(photo.external_relpath || photo.filename || `photo-${photo.id}`);
const outputBasename = `ext${photo.id}_${sourceBasename}`;
logger.info(`Ensuring preview for external photo ${photo.id} from ${localPath}`);
newPreviewPath = await generatePreviewImage(localPath, { regenerate: true, outputBasename });
} else {
let sourceKey;
try {
sourceKey = resolvePhotoStorageKey(event, photo);
} catch (e) {
const msg = (e && e.message) ? e.message : String(e);
logger.error(`Failed to resolve original key for preview (photo ${photo.id}): ${msg}`);
return null;
}
if (!sourceKey) {
// Reference-mode event holding a row with no source_origin: the mode
// falls back to the event's and resolvePhotoStorageKey returns null.
// Honour the documented null-on-failure contract instead of feeding
// null into withLocalCopy, which throws out of this function.
logger.warn(`No managed storage key for preview (photo ${photo.id}); skipping preview generation`);
return null;
}
newPreviewPath = await withLocalCopy(sourceKey, async (localPath) => {
const proc = await withProcessableImage(localPath, sourceKey);
try {
return await generatePreviewImage(proc.path, { regenerate: true, outputBasename: proc.outputBasename });
} finally {
await proc.cleanup();
}
});
}
if (newPreviewPath) {
await db('photos').where({ id: photo.id }).update({ preview_path: newPreviewPath });
return newPreviewPath;
}
return null;
}
/**
* Extract capture date from EXIF metadata
*/
async function extractCaptureDate(imagePath) {
try {
const exif = await exifr.parse(imagePath, {
pick: ['DateTimeOriginal', 'CreateDate', 'DateTimeDigitized', 'ModifyDate']
});
if (!exif) {
return null;
}
const captureDate = exif.DateTimeOriginal ||
exif.CreateDate ||
exif.DateTimeDigitized ||
exif.ModifyDate;
if (captureDate) {
if (captureDate instanceof Date) {
const now = new Date();
const minDate = new Date('1990-01-01');
if (captureDate > minDate && captureDate <= now) {
return captureDate;
}
}
if (typeof captureDate === 'string') {
const parsed = new Date(captureDate);
if (!isNaN(parsed.getTime())) {
return parsed;
}
}
}
return null;
} catch (error) {
logger.debug(`Could not extract EXIF date from ${path.basename(imagePath)}:`, error.message);
return null;
}
}
/**
* Downscale to fit inside a box, for the download-resolution feature (#858).
*
* `fit: 'inside'` + `withoutEnlargement` is exactly the "up to" semantic the
* requester asked for on #858: the box is a maximum, aspect ratio is kept
* (so a 3:2 box leaves a 4:3 photo slightly smaller than the box on one
* edge), and an image already smaller than the box is returned untouched
* rather than upscaled into mush.
*
* Takes and returns a Buffer so callers can chain resize → watermark without
* a tmp file. Returns the input unchanged when `box` is null ('original').
* Never throws: on a corrupt/undecodable source it logs and returns the input,
* because failing a download outright is worse than serving the full size.
*/
async function resizeToBox(inputBuffer, box, options = {}) {
if (!box || !box.width || !box.height) return inputBuffer;
try {
const probe = sharp(inputBuffer, { limitInputPixels: 268402689, failOn: 'none' });
const metadata = await probe.metadata();
// Already inside the box — hand back the original bytes rather than
// re-encoding, which would only cost quality and CPU.
if (metadata.width && metadata.height
&& metadata.width <= box.width && metadata.height <= box.height) {
return inputBuffer;
}
const format = (metadata.format || '').toLowerCase();
// Animated sources must be re-opened with `animated: true`, otherwise
// sharp keeps only the first frame and the download silently loses its
// animation. `.rotate()` would flatten an animated source, so it is
// applied only to stills (where EXIF orientation actually exists).
const animated = (metadata.pages || 1) > 1;
const image = animated
? sharp(inputBuffer, { limitInputPixels: 268402689, failOn: 'none', animated: true })
: probe.rotate();
let pipeline = image
.resize(box.width, box.height, { fit: 'inside', withoutEnlargement: true });
// Re-encode in the SOURCE format. The download routes keep the original
// filename and mime type, so emitting JPEG for a .gif would ship
// mislabelled bytes. GIF is an accepted upload format (multerConfig.photos).
//
// HEIC/HEIF is the exception: sharp builds generally cannot ENCODE it, and
// the browser can't display the original anyway (see originalNeedsPreview
// in gallery.js). Rather than emit JPEG bytes under a .heic name, leave
// those downloads at original size — correct-but-larger beats
// mislabelled-and-broken.
if (format === 'heif' || format === 'heic') {
return inputBuffer;
}
if (format === 'png') {
pipeline = pipeline.png({ compressionLevel: 6 });
} else if (format === 'webp') {
pipeline = pipeline.webp({ quality: options.quality || 90 });
} else if (format === 'gif') {
pipeline = pipeline.gif();
} else {
pipeline = pipeline.jpeg({ quality: options.quality || 90, mozjpeg: true });
}
return await pipeline.toBuffer();
} catch (e) {
logger.warn(`resizeToBox failed (${box.width}x${box.height}), serving original: ${e.message}`);
return inputBuffer;
}
}
module.exports = {
orientedDimensions,
hasOrientationTransform,
ensurePreviewImageAtWidth,
ensureThumbnailAtWidth,
thumbnailTierKeys,
deleteThumbnailTiers,
previewTierKeys,
deletePreviewTiers,
PREVIEW_WIDTHS,
THUMBNAIL_WIDTHS,
normalizeTierWidth,
resizeToBox,
generateThumbnail,
isThumbnailValid,
ensureThumbnail,
generateVideoPlaceholder,
generateHeroImage,
isHeroValid,
ensureHeroImage,
generatePreviewImage,
isPreviewValid,
ensurePreviewImage,
extractCaptureDate,
withLocalCopy,
isRawFilename,
extractRawPreview,
withProcessableImage,
RAW_EXTENSIONS,
};