5db0a76cce94de03f86295ba2bd6ba526661d16d
54 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Paul Nothaft
|
70f5a8c54e |
fix(gallery): bound and reclaim storage reads in the remaining zip builders (#1410)
* fix(gallery): bound and reclaim storage reads in the remaining zip builders PR 1402 fixes the cached-zip builder. The same unguarded pattern — one storage read appended per photo, archiver draining one at a time, nothing destroying the rest — is still present in three other places, two of which a gallery guest reaches with no admin credentials: - routes/gallery/downloads.js, download-all and download-selected - services/downloadJobService.js, the custom-resolution job builder, which is worse in one respect: its per-photo catch skips a bad source without ever destroying the stream it had already opened, so every skipped photo leaked a socket for the life of the process An unread S3 response body holds its socket open indefinitely — the SDK arms its socket timeout on a 3s delay and clears it the moment response headers land, so a fast response never gets one — and archiver's abort() does not touch its source streams. That is the mechanism behind the incident described in PR 1402: 43 of 50 pooled sockets held with unread bytes, uploads and gallery reads starved behind them, a process restart the only way out. utils/archiveStreamGuard.js caps reads in flight at 2 and destroys whatever is still open on every exit: an error, a failed append, and — for the two guest routes — the client closing the tab mid-download, which previously left every appended-but-undrained read parked forever. Deliberately does not touch downloadZipService.js, so there is no conflict with 1402. Once that lands, its inline equivalent can move onto this helper. Local-filesystem installs are unaffected either way: they take archiver's archive.file(path) branch and open no sockets. Relates to issue 1399 * fix(gallery): survive a cancelled download and a read that dies while queued Two failures found reviewing the previous commit, both reachable on an ordinary download. A client hanging up mid-download aborted the archive, but the append loop's `break` still fell through to archive.finalize(), which rejects with ABORTED. The catch then called errorResponse over a response whose ZIP headers had already gone out, throwing ERR_HTTP_HEADERS_SENT from an async Express 4 handler with nothing to catch it — an unhandled rejection, on a cancelled download, which can take the process down. Both bulk routes now return without finalizing, and the catch stays quiet once headers are sent. A read that errored while still QUEUED behind another was absorbed by the guard's own error listener. archiver had not attached its source listener yet, so it never learned the stream had died, and the dead stream stayed in the queue: the archive hung when it reached it, and in downloadJobService the build held its concurrency slot with it. The guard now reports such a failure through onFatalError so the caller aborts the archive and reclaims the rest. Streams the guard destroyed itself are deliberately not reported — that is the caller's own teardown, already running. Relates to issue 1399 --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local> |
||
|
Paul Nothaft
|
662516a5ad | fix: retain revocations for tokens without expiry | ||
|
Paul Nothaft
|
f0e6d2dfb1 |
fix: enforce gallery access and consolidate gallery workflows (#1357)
Harden gallery authentication and authorization, consolidate gallery workflows, and prevent token-bearing URLs from leaking through nginx request error logs. |
||
|
Paul Nothaft
|
69754f8a2c |
fix(email): scrub gallery passwords from the sent-mail archive (#1340)
* fix(email): scrub gallery passwords from the sent-mail archive
The email queue kept every gallery password and client PIN in clear
text in email_data and rendered_html after the mail was sent, and the
Messages reading pane handed them back to any admin with the messaging
flag. A password hash in the events table bought nothing while the
plaintext sat next to it.
Once a mail is out, or its retries are exhausted, the processor now
masks secret-looking variables (password, passcode, pin) in email_data
and replaces their values in the rendered body, plain and HTML-escaped.
The reading pane applies the same masking to rows archived before this
change. Pending rows keep the real values so a retry still sends them.
Relates to issue 1271
* fix(email): keep a quoted ">" from cutting an attribute value out of redaction
The tag splitter stopped at the first ">", so a template attribute such as
title="{{gallery_password}} > details" left the password unmasked in the
archived HTML while email_data was already masked. The tokenizer is now
quote-aware; a tag with an unbalanced quote falls through as text and is
scrubbed there.
* fix(email): scrub secrets inside HTML comments in the archived body
A comment such as <!-- PIN: {{client_password}} --> was split off as a tag
and its body, which has no attribute, was never scrubbed. Comments are now
one segment and their content is masked whole.
---------
Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
|
||
|
Paul Nothaft
|
1151e96144 |
fix(security): validate CSS urls last, after every pass that moves text
Fifth bypass, and the same root cause as the first: sanitizeCSS validated, then kept rewriting. `<[^>]*>` deletes the span it matches, and `<">` takes a quote with it. So `--x:x<">;background:url(https://evil.example/p.gif);--y:x<">` was scanned with the url() safely inside a string, and the tag strip below then removed the quotes that made it so — shipping a live remote background with no warning. The file already carried the rule: "any pass that can join tokens has to happen before validation, not after." It has now been broken three separate times — by the HTML-comment strip (#1290), the control- character strip, and the tag strip. Rather than fix a third instance in place, the URL scan is now the LAST step, so what is validated is always the bytes that get served. All eight known bypass classes are pinned, together with the legitimate data: URI, quoted font stack and escaped selector that must survive untouched. Refs #1264 |
||
|
Paul Nothaft
|
4196e83a5f |
fix(security): use CSS whitespace, not JavaScript's, in the url() reader
Third bypass of this scanner found in one review pass, and the same
shape as the others: the lexer and a browser disagreeing about where a
token begins.
JavaScript's `\s` matches U+00A0; CSS whitespace is exactly space, tab,
LF, CR and FF. Skipping an NBSP as whitespace let the scanner read the
quote after it as a legitimate quoted data: URI and swallow a remote
url() inside that "string" —
.a{background:url(<NBSP>"data:image/png);background:url(https://evil…);--x:");}
came through untouched, with no warning, and survived re-sanitising. A
browser treats NBSP as an ordinary character, so that is an UNQUOTED
url-token ending at the first `)`, leaving the remote background live.
All three token readers now use an explicit CSS whitespace class.
Ordinary spacing around a data: URI still works, and is pinned.
Refs #1264
|
||
|
Paul Nothaft
|
b6dc0991ce |
fix(security): check for an escaped identifier before consuming the escape
My previous commit introduced this. Handling `\` outside strings before
readIdentifier meant a LEADING escape was eaten before the url check
saw it: `\75` is the CSS escape for `u`, so `.a{background:\75rl(...)}`
is url() to a browser and passed through untouched, with no warning —
a bypass the base version did not have. An escape mid-identifier
(`u\72l`) was unaffected, which is why the first tests missed it.
The escape branch now runs AFTER readIdentifier, which already decodes
leading escapes itself. What is left for it is the case it was added
for: `\'`, which must not be read as opening a string.
Both spellings are pinned, along with the legitimate escaped selector
and data: URI that must survive untouched.
Refs #1264
|
||
|
Paul Nothaft
|
1cf82746b7 |
fix(security): close two CSS url() bypasses the sanitizer dedup exposed
Both found by review against the correct base, and both are cases the
second stripRemoteCssUrls pass had been catching before this PR removed
it. Verified against the real functions before and after.
An escaped quote outside a string. `\'` is an escaped identifier
character, not a string opener, but the scanner stepped onto the
apostrophe, entered string mode and copied the rest of the stylesheet
unexamined — so `.hero{--marker:\';background:url(https://evil/p.gif)}`
kept a live remote URL. Escapes are now consumed as a unit outside
strings.
An unterminated quote. Trusting one meant a single stray apostrophe
disabled scanning for everything after it. An unclosed quote is a parse
error, so the safe reading is to emit it as an ordinary character and
keep scanning; a newline also ends a string, as it does in CSS.
The entity mismatch behind the second case. sanitize-html writes `"`
inside an attribute as `"`, so the scanner and the recipient's
browser disagreed about where strings begin: in
`style="font-family:"don't";background:url(...)"` the browser
decodes first, reads the apostrophe as ordinary text inside a real
string, and fetches the background — a tracking pixel by another name.
Style attributes are now decoded before scanning and re-encoded after,
which also stops the old code silently deleting quotes from the value.
Also detaches the image handlers before releasing the canvas source.
That one did NOT reproduce: measured in both Chromium and WebKit,
neither fires `error` when the attribute is removed after a successful
load. Applied anyway because the ordering is free and the failure it
would cause is silent — canvasFailed set, the canvas swapped for an
<img>, and the image decoded a second time, the exact opposite of what
the release is for.
Refs #1264, #1287
|
||
|
Paul Nothaft
|
99f54a3954 |
fix(security): strip control characters before scanning CSS for url()
Review follow-up on the sanitizer dedup. sanitizeCSS already carried the rule — "any pass that can join tokens has to happen before validation, not after" — written above the URL scan to explain why it runs after the HTML-comment strip. The control-character strip is exactly such a pass and sat eleven lines below it. So `u<CTRL>rl(https://tracker.example/p.gif)` was scanned as clean, and the strip below then joined it into a live remote request with no warning. Newlines are control characters here too, so `u\nrl(...)` did it without an exotic byte. Verified against the real function before and after: all five variants returned a live remote url() and now return `none` plus the blocked-URL warning. This PR is what exposed it. Dropping newsletterService's second stripRemoteCssUrls pass was right — the duplicate hid a defect in the shared sanitizer rather than fixing it — but it removed the belt that was catching this for the newsletter path. Fixing the ordering fixes it for every caller instead of restoring the second pass. Refs #1264 |
||
|
Paul Nothaft
|
a7d0972b13 |
fix(security): make the CSS sanitizer's remote-URL block actually block
sanitizeCSS "blocked" a remote url() by prefixing it with a /* BLOCKED URL */ COMMENT and leaving the URL in place. CSS comments are discarded during tokenization, so the declaration a browser parsed still carried the live URL — while adminCssTemplates returned sanitization_warnings claiming it had been stopped. Protection that reports success is worse than none, which is why it survived review. Scope is narrow: sanitizeCss (lowercase, the public-site path) never included the pattern and permits remote URLs by design — a test now pins that. Only sanitizeCSS (uppercase) was affected; outside this repo's newsletter branch its sole caller is adminCssTemplates.js. Migration 200 is required, not cosmetic: gallery.js serves css_templates.css_content VERBATIM as text/css and does not re-sanitize on read, so fixing the write path alone would leave every existing template serving its URL forever. Review follow-ups replaced the regex with a small three-state lexer (comment / string / identifier) over the RAW text, after five further bypasses: a ")" inside a quoted url(), CSS escapes (u\72l), the HTML comment strip JOINING tokens into a live url() after the scan, an escaped quote desynchronising the scan, and a quote inside a comment. Escapes are decoded only to decide, never to rewrite — a clean input now round-trips byte-identical, which also keeps unaffected rows out of the migration's write path. Severity is low (writing a template needs branding.edit) but the harm is a gallery visitor's IP reaching a third party from a page the operator believes carries no remote requests. |
||
|
Paul Nothaft
|
063977d97d |
fix(security): never serve a photo under its stored MIME, and stop trusting the chunked-upload type
chunked-upload/init stored the client-declared mimeType on the photo row and the gallery, secure-image and protected-image routes echoed it as Content-Type, so a JPEG/HTML polyglot declared as text/html rendered inline on the app origin for every guest. The admin photo route already resolved the type safely (#908 review); that logic now lives in utils/photoContentType and every serving route uses it. The chunked path derives the MIME from the filename extension and requires that extension to be on the admin allow-list, matching what the multipart path enforces through its multer fileFilter. |
||
|
Paul Nothaft
|
3e46530072 |
fix(security): contain logo, favicon and PDF-logo unlinks to their upload directories
Settings > Branding persisted logo_url / favicon_url verbatim and on clear
unlinked path.join(storage, url) behind a startsWith('/uploads/logos/')
check, which '..' segments pass. The business-profile PDF logo did the same
behind a /pdf-logo-\d+\./ marker test, and used absolute values as given.
Either let a settings.edit or settings.banking holder delete any file the
process can reach.
Both now resolve through helpers in utils/safePath that only ever name a
flat leaf inside the fixed directory. The /favicon.ico streamer is narrowed
the same way: it contained to the whole uploads/ root, which also holds
signed contracts and transfer files.
|
||
|
Paul Nothaft
|
0ca0e4a922 |
fix(security): verify the signature before writing a token to the revocation list
revokeToken() base64-decoded the payload without checking the signature and inserted a row keyed on id-iat-type, the same key isTokenRevoked() matches for real sessions. The logout endpoints are unauthenticated, so anyone could forge a payload naming another user's id, type and login second and log them out remotely; a far-future exp also left rows that cleanup never swept. Expiry is still ignored so logging out an expired session stays idempotent. |
||
|
Paul Nothaft
|
98aa06aeff |
fix(email): read naive SQLite timestamps as UTC, and page the candidates
Codex review round 2 on #1273. Both findings restore the false all-clear that round 1 set out to remove, by different routes. Both timestamp columns default to CURRENT_TIMESTAMP, which SQLite renders as a zone-less 'YYYY-MM-DD HH:MM:SS' in UTC -- and Date.parse reads that shape as LOCAL time. On a TZ=America/New_York deployment a row due now looked four hours away and never reached the waiting list; nine hours the other way, fresh mail read as long overdue. The parser now stamps the zone the value actually carries. That parser moved to utils/queueTimestamps so it can be tested honestly. This suite runs in UTC, where reading a zone-less value as local and as UTC give the same answer, and process.env.TZ does not reliably re-bind mid-process -- my first attempt at these tests passed against the broken code for exactly that reason. They now force TZ in a child process, so they fail on any host. The candidate rows are paged rather than cut off with one LIMIT. The time filter runs in JS, so a queue holding more than a page of future-scheduled rows -- split-payment invoices are exactly that shape -- filled the window with rows that all got filtered out and hid the due row behind them, reporting nothing waiting. Paging also drops the dependency on ORDER BY created_at meaning anything, which it does not on SQLite once numeric and text timestamps mix. Bounded at 10k scanned; past that the response is a sample, which the 200-row cap already made it. 12 more tests. The paging one fails before this commit, and all four naive-timestamp ones fail against the old parsing on any host. |
||
|
peipeimo
|
4f352dec39 |
fix(auth): treat zxcvbn suggestions as advice, not blocking errors (#1050)
validatePassword() appended zxcvbn's feedback.suggestions to the errors array unconditionally, and validity is errors.length === 0 — so any password that merely earned a suggestion was rejected even when it satisfied every configured rule. The effective policy was stricter than the configured complexity level and invisible to the admin. Suggestions now surface only alongside a real strength failure. They stay available to callers in result.feedback.suggestions, so a UI can still show them as guidance while typing. The weak-password fixture is assembled from parts rather than inlined: an 8-char alphanumeric literal next to validatePassword( reads as a hardcoded credential to the required GitGuardian check. Both fixtures pin their zxcvbn score — the compliant one is load-bearing at exactly the moderate minimum (2), and a future zxcvbn bump promoting it to 3 would leave the test green while no longer covering the bug. Co-authored-by: Peifu Mo <peipeimo@users.noreply.github.com> |
||
|
Paul Nothaft
|
4f684eb482 |
fix(feedback): name the camera original in the exports, not just the stored file (#1224) (#1228)
Both feedback exports carried only `photos.filename` — the sanitized stored name (`wedding-smith_individual_1755892345.jpg`). Acting on client picks means finding the master on disk, and that name matches nothing in a Lightroom catalog, so the export could not be joined to anything. Adds the camera-original name to the long and pivot shapes, and by extension to the archive's feedback_data.csv/.json, which reuse the same query. COALESCE(source_filename, original_filename), not original_filename alone: the latter is overwritten the first time an edited render is uploaded over a proof (#745), so an export taken after a round-trip would name the render rather than the master and silently stop matching. source_filename is written once at ingest and survives a replace by design (migration 193). That case is the load-bearing test. Aliased to `original_filename` — the name the sibling photo export already uses for this column, and the question the reader is asking. Left empty when neither is known rather than echoing the stored name: blank reads as "no match possible", where repeating the sanitized name invites a match attempt against a file that does not exist under it. The column is added, not swapped: `filename` is untouched, so anything reading the old column keeps working. Reported by the 8digit/picpeak fork, which has carried a narrower version of this patch (original_filename only) across rebases. Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local> |
||
|
Luca
|
e2844d1909 |
feat(gallery): colour labels for client proofing, and one global default per feedback type (#1044) (#1137)
Colour labels for client proofing, plus the photographer's own stars and colours in the admin grid. - Guest colour labels alongside likes/reactions, opt-in per event (defaults off so live galleries do not change mid-proofing), with 'colors' and 'lightroom' keybind schemes. - One global default per feedback type, replacing the per-type scatter. - Admin marks live in their own table (photo_admin_marks) so they can never reach a guest-facing surface. - XMP export prefers a real label, keeping the rating-derived mapping as a fallback. Review: concurrent-write loss on the mark update path, migration index idempotency and error classification all fixed in 7139bcae; migrations renumbered to 182/183 in 8fecdfae after 180/181 were taken on main. Merged with admin privileges: bypass-size-gate is a required check that fails on size alone for review-bypass authors and never re-evaluates on review, which is its designed behaviour once a maintainer has approved. |
||
|
Paul Nothaft
|
f735d26422 |
fix(gallery): a missing thumbnail tier must not take the backend down (#1128)
The first load of a gallery whose ?w= tiers do not exist yet could exit the Node process — not 500 one tile, kill the backend. Two defects stacked. The reader: LocalFsStorage.get() returns a lazy fs.createReadStream, so an ENOENT arrives after the await returned and outside the route's try/catch. An unhandled 'error' event is a process-level throw. pipeStreamToResponse attaches the handler the routes were missing — 404 for a vanished source, connection destroyed if bytes are already on the wire, file headers cleared so the JSON error is not served as image/jpeg or cached as a broken tile for an hour. Applied to all nine streaming responses in gallery.js. The writer: ensureThumbnailAtWidth passed regenerate:true, whose first act is to DELETE the target — on a path only reached when the tier is absent. A grid fires one request per tile, so one request unlinked the file another had just published and handed to a reader. Without the flag the write is an atomic rename. Generation is now also deduped per tier key: 8 concurrent requests ran 5 Sharp passes before, 1 after. Reported with a full diagnosis by @BraynArts. |
||
|
Paul Nothaft
|
52db982661 |
fix(gallery): make per-event banner overrides actually work, both banners (#440, #932) (#1064)
* fix(gallery): make per-event banner overrides actually work, both banners (#440, #932) The promo banner shipped with a per-event inherit/custom/off override that never reached a guest. GalleryView reads promo_mode from the /photos payload, and /photos never sent it — so every gallery resolved to 'inherit'. Setting a gallery's promo banner to "Off" did nothing; the global banner kept rendering. The info banner (#932) mirrored that shape and inherited the same gaps. Four places dropped the fields; all four now carry both banners: 1. GET /gallery/:slug/photos — send promo_mode/promo_markdown alongside the info fields. This is the fix that makes "Off" mean off. 2. POST /admin/events — the validators accepted both banners and the insert discarded them, so an API client could POST info_mode:'off', get 201, and find the row on 'inherit'. Markdown is stored only for 'custom', matching the PUT rule. 3. POST /admin/events/:id/duplicate — copy both from the source row. The dialog promises the copy "inherits the branding, behaviour, feedback, and category configuration"; a muted gallery un-muting on duplication is the opposite of that. 4. PUT /admin/events/:id — resolve the effective mode from the STORED row when a partial update sends only the markdown. Previously updates.promo_mode was undefined on such a request and the text was parked on an inherit/off gallery, then resurfaced when someone later switched it to 'custom'. The lookup is lazy: one extra query, only on that path. The two normalisation blocks are now one loop over both banners, so the pair can't drift apart again. Verified in a browser, both directions against the same global banner: promo_mode='off' -> not rendered; 'inherit' -> rendered. The /photos payload went from promo_mode ABSENT to carrying the value. * fix(gallery): thread promo into the reveal view, drop stale markdown on duplicate External review, round 1 on this PR. Two gaps in the plumbing it introduced: - The reveal-hidden branch copied only the info fields from /photos. Now that /photos carries promo too, a reveal-hidden gallery with promo_mode 'off' still fell back to 'inherit' and showed the global banner on the first load after login. Thread both banners there. - The duplicate copied markdown verbatim. A row written before the PUT normalisation landed can hold text while its mode is 'inherit'/'off', so the copy inherited hidden text that would resurface the moment someone switched it to 'custom' — violating the very invariant this PR establishes. Copy markdown only when the source mode is 'custom'. Test covers the stale-markdown source explicitly. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local> |
||
|
Paul Nothaft
|
b48fa62eea |
feat(gallery): info banner above the photo grid (#932) (#1063)
* feat(gallery): info banner above the photo grid (#932) A short informational note rendered at the TOP of a gallery, above the photos. Distinct from the promotional banner (#440), which stays by the footer for marketing copy — the reporter's case is an onboarding hint ("use the menu button to filter"), which is useless below a gallery the guest has to scroll past first. Mirrors the promo feature's shape rather than inventing a second one: a global default in Settings → Branding (branding_info_markdown) plus a per-event inherit/custom/off override. Markdown via the existing MarkdownContent sanitiser — no raw HTML, no CSS injection. Empty global default means nothing renders, so upgrading changes nothing visible. Deliberately NOT included: an alignment knob (this is short helper copy, not marketing layout) and guest dismissal — the issue lists dismissal as a nice-to-have, and it needs per-guest persistence that is its own decision. Migration 176 is idempotent (hasColumn / existing-key guarded). Note on the payload plumbing: the per-event fields travel in the /photos response, not just /info. GalleryAuthContext seeds its cached event from the gallery LOGIN response — a small identity subset — so anything absent there is undefined right after a guest signs in. /photos is the payload that refreshes on every gallery load, which is why the fields were added there and why GalleryView reads them from `data.event`. Verified in a browser across all three modes; reading them from the context event instead silently collapsed every override back to 'inherit'. * fix(branding): map branding_info_markdown on read so saving can't wipe it (#932) External review caught this. BrandingSettings declared no info_markdown and formatBrandingSettings never mapped branding_info_markdown, so BrandingPage's hydration — setBrandingSettings(prev => ({ ...prev, ...formatted })) — kept the empty-string initializer instead of the persisted value. The form loaded blank and the next Save posted '' back, wiping a configured banner. Silently: the gallery keeps rendering the old copy until that save lands. This is the same bug the footer/promo fields hit in #441 + #440 / #460, which the read mapper still carries a comment about. Add the field to the interface and the mapper, and pin the round-trip for the whole editable branding set so the next field added is caught by a test rather than by a user losing copy. Verified: the new test fails 3/4 with the mapper line removed. * fix(gallery): honour the info-banner override in the reveal-hidden view (#932) External review, round 2. The hidden-until-reveal branch renders GalleryLayout with the context `event`, which is seeded from the gallery login response and carries no banner fields — so while a gallery was hidden, a per-event 'off' silently resolved to 'inherit' and the global banner appeared on a gallery the admin had muted. Resolve the fields there the same way the main render path does. The two full-page layouts (gallery-premium, gallery-story) are deliberately left alone: they return before GalleryLayout and render no header, footer or promo banner either — injecting a wrapper into layouts documented as having 'their own integrated UI' would be a design change, not a fix. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local> |
||
|
Paul Nothaft
|
3a11e6ebb5 |
fix(pdf): RFC 6266-encode Content-Disposition on quote/invoice PDFs (#1024) (#1055)
* fix(pdf): RFC 6266-encode Content-Disposition on quote/invoice PDFs (#1024) The six quote/invoice PDF endpoints interpolated buildPdfFilename()'s result straight into `inline; filename="${filename}"`. That result deliberately preserves non-ASCII (it doubles as the PDF's internal Title metadata), and HTTP header values are latin1 — so a customer label reaching the header directly failed in one of two ways: - U+0080-U+00FF (ä ö ü ß — every German umlaut): no throw. The raw byte goes out and the client reads back a mangled name. Silent corruption. - above U+00FF (Polish ł, Czech ř, Turkish ş, €, Cyrillic, CJK, emoji): Node's setHeader rejects it with ERR_INVALID_CHAR. The throw lands after the PDF buffer is already rendered, so the request 500s. Note this corrects the issue's diagnosis: it reported umlauts as the 500 case, but umlauts are inside latin1 and mangle rather than throw. Both symptoms share this root cause and both are fixed here. Route through buildContentDisposition() (utils/filenameSanitizer, already used by secureImages.js), which emits an ASCII fallback plus the RFC 5987 `filename*=UTF-8''…` form, so the unicode name survives in browsers and the header stays legal. Applied to all six sites: adminQuotes (persisted + preview), adminInvoices (persisted + preview), customer (quote + invoice). Also correct buildPdfFilename's docstring, which advertised the preserved non-ASCII as suitable for Content-Disposition — the exact misreading that produced these call sites. * test(pdf): pin the ASCII fallback for fully non-Latin customer names (#1024) A name written entirely in another script leaves the legacy filename= token with just the document number (Q-2026-0042_.pdf) — filename* carries the real name. That's the intended trade, but it's the token a client without RFC 5987 support actually saves, so assert it stays legal, non-empty and carries the document number rather than leaving it unpinned. * fix(pdf): don't split surrogate pairs when truncating the filename (#1024) Codex review caught this. sanitiseSegment caps each segment at 80 UTF-16 code units, so a cap landing inside an astral character (emoji, rarer CJK) left a dangling high surrogate. encodeURIComponent throws URIError: URI malformed on a lone surrogate, so buildContentDisposition — the helper this PR routes the six PDF endpoints through — 500'd for e.g. company_name = 'a'.repeat(79)+'🎉', well inside the 120-char validator limit. Same 500 the PR set out to remove, reached a different way. Drop the orphaned surrogate instead of widening the cap, so the byte budget the limit exists to protect is unchanged. Tests cover both boundary cases and assert the cap semantics; they fail against the previous slice(). --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local> |
||
|
Paul Nothaft
|
6de30e5bf1 |
fix(docker): default NODE_ENV=production so non-compose deploys don't fall back to SQLite (#1038) (#1039)
* fix(docker): default NODE_ENV=production so non-compose deploys don't fall back to SQLite (#1038) knexfile.js selects its config block by NODE_ENV and the `development` block defaults to sqlite3. The image never set NODE_ENV, so every deployment that doesn't go through our compose files — Kubernetes, Helm, plain `docker run` — silently ran on SQLite and ignored DB_HOST/DB_USER/DB_PASSWORD. It stayed invisible because wait-for-db.sh is shell: it reads DB_HOST directly, connects to Postgres, creates the database and logs "PostgreSQL is up" in the same container where the Node process then writes to a SQLite file. Migrations go through src/database/db.js → the same knexfile, so they also ran against SQLite, leaving the provisioned Postgres database empty. Setting the default alone would be unsafe: an affected install would flip to Postgres on its next image pull and come up against an EMPTY database, which reads as total data loss. So this adds a guard that runs before migrations touch anything: - logs the resolved engine + target at boot (nothing did before, which is why this went unnoticed for so long) - refuses to start when pointed at a virgin Postgres while a populated SQLite file exists, naming the file and the .picpeak export path for moving the data, with PICPEAK_ALLOW_EMPTY_PG=true as the escape hatch - warns but boots when Postgres settings are present yet SQLite is in use Compose files already set NODE_ENV explicitly, so compose users are unaffected. The engine-selection tests resolve knexfile in a child process with a clean cwd — dotenv.config() would otherwise let a developer's backend/.env decide the answer instead of the knexfile defaults under test. Fake credentials in the describeEngine tests are built at runtime rather than written inline, so secret scanners don't flag a literal after `password:`. Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM * fix(db): stay on SQLite instead of blocking, and add a migration path (#1038) Reworks the guard from the previous commit after walking through what an existing install actually experiences on its next image pull. Blocking was the wrong trade. An operator who had unknowingly been running on SQLite (because the image left NODE_ENV unset) would have pulled the fix and got a CrashLoopBackOff: data safe, galleries offline, for something they did not do. Now the boot RESOLVES the engine before migrations run and stays on whichever one holds the data: - Postgres configured but holding no galleries, while a populated SQLite file exists → keep serving from SQLite, print what happened and how to migrate. Nothing moves until the operator decides. - once Postgres holds the data, the next restart switches over on its own. - an explicit DATABASE_CLIENT is always honoured. The check is keyed on Postgres holding DATA, not on it having tables: a stray `run-migrations` against the empty database creates every table, which would otherwise blind the check and strand the operator on an empty install. wait-for-db.sh resolves the engine and exports DATABASE_CLIENT before the migration step, so the runner and the server always agree. Manual migration runs (no entrypoint, no exported client) now refuse rather than build a schema in the wrong database. Adds scripts/migrate-sqlite-to-postgres.js for moving the data across. It reuses the .picpeak export/import services rather than hand-rolling a cross-engine copy — they already handle FK suspension, JSON columns and Postgres sequence resync. Two things had to be added for the SQLite → Postgres direction, both opt-in and CLI-only so the upload/restore UI is untouched: - `allowEngineSwitch` relaxes the importer's same-engine guard - cross-engine row coercion: SQLite has no real date or boolean types, so its rows carry epoch numbers where Postgres wants a timestamp and 0/1 where it wants a boolean. Postgres rejects both outright ("date/time field value out of range: 1786548038763"). Coercion is driven by the TARGET schema, never guessed from the value. Verified end to end against a real PostgreSQL 15: a seeded SQLite install migrated across with booleans, timestamps and foreign keys intact, and the serial sequences correctly advanced (the next INSERT got id 2, not a primary-key collision). Photo files on disk are never touched and the SQLite file is left in place as a rollback. Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM * fix(db): close four review findings on the SQLite fallback + migration (#1038) External review (codex) found four issues, all confirmed against the code and fixed here. Two of them could have cost data. 1. The engine resolver was reachable only through wait-for-db.sh. A Kubernetes manifest that sets `command`/`args`, or a plain `docker run … node server.js`, bypasses the entrypoint — exactly the deployment styles this fix targets. With NODE_ENV now baked into the image, such an install would have resolved to Postgres and come up against an empty database while its SQLite data sat there unseen. server.js now resolves the engine itself, before anything requires knexfile, via the same script the entrypoint uses. Verified by running `node server.js` directly against an install with stranded SQLite data: it logs the banner and serves SQLite. 2. Cross-engine loads double-encoded JSON. SQLite has no json type, so its json columns are TEXT holding JSON; the export dumps that as a string and serialiseJsonColumns stringified it again, storing `true` as the scalar string "true". app_settings.setting_value is json on every install, so this reshaped every migrated setting. The text is decoded before serialisation now — verified against a real Postgres: json_typeof(setting_value) is `boolean`, matching a native install exactly. 3. The migration could silently miss concurrent writes. If the backend keeps serving, rows written after the export never reach Postgres and vanish from view once the engine switches. The script now fingerprints the SQLite tables whose loss would be noticed, checks for drift BEFORE loading Postgres (so a detected race leaves the target untouched) and again after, and refuses with the exact rows that moved. It also says plainly to stop the backend first. 4. The child phases shared stdout with winston. Outside production, and whenever LOG_TO_CONSOLE=true, createPicpeak's own log line was concatenated with the archive path and the migration failed on a bogus filename. Payloads travel through a result file now; verified with LOG_TO_CONSOLE=true. Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM * fix(db): close review round 2 — six more data-safety findings (#1038) 1. The engine choice is now PINNED once the data is in Postgres. Previously the boot decided from "does Postgres hold galleries", so an operator who later deleted every gallery would be sent back to the stale pre-migration SQLite file while their settings, admins and CRM data stayed in Postgres. The migration writes a marker next to the database file (and retires the file itself by renaming it); the marker wins over any probe. 2. The migration refused to overwrite Postgres only when it held GALLERIES. A target with admins, customers, invoices or projects but no galleries was wiped without --force. Both the source and target checks now look for user data across the tables that are empty on a fresh install. 3. Same bug in the other direction: an install with no galleries but real admins/settings/customers was refused a migration it was entitled to. 4. Drift detection covered four tables and only count/max(id), so an in-place UPDATE (event edit, password change) or a write to any other table passed unnoticed. It now fingerprints every table the export carries, including max(updated_at). It still is not a substitute for stopping the backend, and the script says so rather than implying a guarantee. 5. probeSqliteData() treated an unreadable or corrupt file as "no data", which would have switched the install to an empty Postgres — the very failure this module exists to prevent. It fails closed now and stays on SQLite so the real error surfaces. 6. The "you are leaving SQLite data behind" warning was unreachable: setting DATABASE_CLIENT skipped the probes, so the branch that produces it never had the inputs. Postgres and SQLite are both probed whenever Postgres is the engine in play. Also: the final verification compares row counts for EVERY table rather than just galleries, and flags only a shortfall — the import legitimately adds an app_settings row (setSessionsValidAfter) that made the strict equality fail on a first real run. Verified against a real PostgreSQL 15 end to end, including: the marker keeps an install on Postgres after every gallery is deleted; removing the marker and restoring the file rolls back to SQLite as documented. Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM * fix(db): close review round 3 — occupancy, bootstrap admin, secrets in /tmp (#1038) 1. Both engine probes judged occupancy by GALLERIES alone. An install whose galleries were all deleted, but which still has admins, customers or accounting records, was treated as empty: on the SQLite side that meant booting the empty Postgres and appearing to lose everything; on the Postgres side it meant diverting a live install to a stale SQLite file. Both now look across the tables that are empty on a fresh install, matching the migration script. 2. The migration ran migrate-schema BEFORE checking the target, and migration 001 seeds a bootstrap admin when ADMIN_PASSWORD is set (common on legacy installs). The occupancy check then saw that admin and refused, pushing the operator towards --force against a genuinely empty database. The target is read first now. 3. probeSqliteData()'s warning went through the app logger, which writes to STDOUT when LOG_TO_CONSOLE=true — and the resolver's stdout is the protocol channel wait-for-db.sh captures, so DATABASE_CLIENT could have been set to a JSON log line. Diagnostics take an injected sink (stderr in the resolver), and the shell now validates the value it captured instead of trusting it. 4. The .picpeak archive holds password hashes, SMTP credentials and API keys in plaintext, and was only removed on the fully-successful path — any drift or import failure left it in /tmp. Every exit path removes it now. 5. A database-only migration still hauled every business-doc and upload through /tmp and back into the same volume. createPicpeak takes includeFiles:false for this path; rows move, files stay where they already are. Verified against a real PostgreSQL 15: a gallery-less install with only an admin account now stays on SQLite and migrates successfully with ADMIN_PASSWORD set; the resolver emits exactly one token on stdout with LOG_TO_CONSOLE=true and a corrupt database; a drift failure leaves Postgres untouched and no archive behind. Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM * fix(db): pin the boot to SQLite while a migration is unfinished (#1038) Review round 4. A migration that dies after touching Postgres leaves rows behind — schema creation alone seeds a bootstrap admin when ADMIN_PASSWORD is set, and a drift or row-count failure can leave a partial load. Since the occupancy probes were widened in round 3, those rows read as "Postgres is occupied", so the next restart would switch engines and hide the SQLite data that is still the database of record. The script now writes a pin file next to the database BEFORE its first Postgres write and clears it only on success (after the success marker exists, so no restart in between can pick the wrong engine). While the pin is present the resolver stays on SQLite and explains why. Verified against a real PostgreSQL 15 by reproducing the exact scenario: a migration failed mid-run with ADMIN_PASSWORD set, leaving one bootstrap admin in Postgres. With the pin the next boot resolves to sqlite3; with the pin removed it resolves to pg — the failure this closes. The subsequent successful re-run clears the pin and the boot moves to Postgres. Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM * fix(db): close review round 5 — occupancy, path drift, retry, host default (#1038) 1. A seeded bootstrap admin counted as "Postgres is occupied". core/001_init.js inserts one whenever ADMIN_PASSWORD is set, so a Postgres that was initialised once and never used would have beaten a SQLite file full of real galleries — the exact failure the guard exists to prevent, reintroduced by widening the probe in round 3. The two sides are deliberately asymmetric now: the SQLite probe counts any user data (err towards keeping data visible), the Postgres probe ignores rows that schema creation seeds (err towards requiring proof of real use). 2. The guard resolved DATABASE_PATH with its own logic while knexfile trimmed whitespace and collapsed the legacy duplicated-backend form. A path either engine normalised differently meant probing a file nobody uses, concluding there was no SQLite data, and booting an empty Postgres. The resolution now lives in one module both require. 3. Re-running after a partial migration — the documented recovery — was refused unless the operator passed the destructive-sounding --force, because the half-written rows read as target data. An unfinished run of this same script is now recognised as a safe retry. 4. wait-for-db.sh verified readiness against its own default host (`postgres`) while knexfile's production block defaults to `db`. With NODE_ENV now baked in, a bare `docker run` without DB_HOST would have passed the readiness check against one host and then dialled another. The entrypoint exports the exact connection it verified. Compose sets DB_HOST explicitly and is unaffected. Verified: a Postgres holding only a seeded admin now loses to real SQLite data; a DATABASE_PATH with surrounding whitespace resolves to the identical file in both knexfile and the guard. Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM * fix(db): close review round 6 — explicit-client bypasses, retry scope, cleanup (#1038) 1. An explicit DATABASE_CLIENT bypassed the unfinished-migration pin, because decideBootEngine honoured it first. docker-compose sets DATABASE_CLIENT=pg, so a failed migration would have restarted on a half-written Postgres on exactly the deployments that pin it. Worse in the other direction: with DATABASE_CLIENT=sqlite3, a SUCCESSFUL migration renames the source file, so the next start created a NEW, empty SQLite database and served that. The pin now outranks explicit pg (clearing the marker is the override), explicit sqlite3 is left alone since it already points at the data, and the migration refuses up front when the deployment pins anything other than pg. 2. The retry allowance was bound to the SQLite file, not to the target. An operator who repointed DB_HOST/DB_NAME between attempts could have replaced an unrelated populated database without --force. The pin records the target and the allowance only applies when it matches. 3. The printed rollback did not roll back: with data on both sides and no marker, the resolver still selects Postgres. It now spells out all three steps, including DATABASE_CLIENT=sqlite3. 4. A failure inside createPicpeak left a partial archive — plaintext hashes and credentials — in the caller-supplied temp dir, which that service deliberately does not clean. The export phase removes it on error. Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM * fix(db): close review round 7 — pin bypass on direct start, real admins (#1038) 1. server.js only ran the engine resolver when DATABASE_CLIENT was unset, so a deployment that both bypasses the entrypoint (Kubernetes `command:`) AND pins DATABASE_CLIENT=pg never consulted the migration pin — the round-6 fix was unreachable on exactly that path, and a failed migration would have served a half-populated Postgres. The resolver now also runs whenever a pin file exists. 2. Round 5 excluded admin_users from Postgres occupancy to stop a seeded bootstrap admin counting as real data. That over-corrected: an install that has completed first-run setup but has no galleries yet has exactly one user-created row — an admin — so Postgres looked empty and, with a stale SQLite file present, the boot would switch away and the admin's credentials and configuration would disappear. core/001_init.js seeds must_change_password=true; setupService writes false once a human completes setup. The FLAG, not the table, distinguishes them, and a legacy NULL counts as a real admin. Verified against a real PostgreSQL 15: a Postgres holding only the seeded row loses to real SQLite data, the same Postgres wins once setup is completed, and a server started directly with DATABASE_CLIENT=pg and a pin present comes up on SQLite with the warning. Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM * fix(db): close review round 8 — reset admins, CLI config, JSON nulls (#1038) 1. must_change_password is mutable: resetAdminPassword() re-raises it on REAL accounts (userManagementService.js:474). Round 7's discriminator therefore read a gallery-less Postgres whose only admin had been reset as an untouched bootstrap seed — and with a stale SQLite file present, the boot would have switched away and hidden those live credentials. The rule is layered now: more than one admin, any admin that has logged in, or must_change_password false all count as use. Only core/001_init.js's exact leftovers — one admin, never logged in, still flagged — read as a seed. 2. The CLI read process.env directly but never loaded the configuration the child phases get through knexfile, so running it directly (or via `docker exec`, which does not inherit wait-for-db.sh's exports) failed the pre-flight checks even with valid settings in backend/.env or /run/secrets/db_password. Both sources are loaded up front now. 3. The migration's target check counted a seeded bootstrap admin as user data while probePgData classified the identical row as empty, so migrating into a previously-initialised-but-unused Postgres demanded --force. Same rule on both sides. 4. Cross-engine JSON handling is simpler and no longer lossy. SQLite keeps json columns as TEXT holding valid JSON and Postgres accepts JSON text directly, so the correct action is to pass them through untouched. Round 1 parsed then re-serialised them to undo a double-stringify; that round-tripped the JSON literal `null` into SQL NULL, changing data and breaking NOT NULL json columns. Not serialising at all fixes both. Verified against a real PostgreSQL 15: a migrated install now carries json_typeof = null for a JSON null, object for a nested object, and boolean for a boolean — matching a native install exactly. Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM * fix(db): close review round 9 — probe error classes, marker ordering (#1038) 1. probePgData() answered every failure with "Postgres has data". That is right for an unreachable server — the app cannot run on it either way, and diverting a healthy pg install to a stale SQLite file over a transient blip would be worse — but wrong for a server that answers and then fails the query, which is what a half-built or damaged schema looks like. That is not evidence of data, and reporting it as such booted the empty Postgres and hid a populated SQLite file: the exact failure this guard exists to prevent. Reachability is now established with SELECT 1 first, so the two cases get opposite answers: unreachable → leave the configured engine alone; reachable-but-uninspectable → unproven, and the SQLite side wins if it actually holds data. 2. The success marker was written after the SQLite file was renamed away. A failure in between — a full disk — left the source retired with no marker: the next attempt reported "No SQLite database", the in-progress pin stayed, and the operator never saw the rollback path. The marker is written first and updated with the retired filename once the rename succeeds, so a failure at any point leaves everything recoverable. Verified against a real PostgreSQL 15: a reachable database whose admin_users table lacks the probed column now resolves to sqlite3 rather than hiding the data, while an unreachable host still resolves to pg. Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM * fix(db): don't fail the migration on empty SQLite-only tables (#1038) Review round 10. The final verification flagged every source table missing from Postgres, regardless of whether it held rows — and SQLite-only tables do exist: initializeDatabase() creates an `events_new` scratch table and, when its legacy column copy throws, the catch swallows the error and leaves the empty table behind (db.js:236). The importer correctly skips tables Postgres does not have, so verification then reported a mismatch AFTER the data had already landed, exited 1, and left the install pinned to SQLite with no way to finish. An absent target table only matters if the source actually had rows. Empty ones are now listed and skipped. Reproduced both ways against a real PostgreSQL 15 with an events_new table present: without the fix the run ends in "ROW COUNTS DO NOT MATCH" and leaves the in-progress pin; with it, the table is reported as skipped, the migration completes and the pin is released. Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM * fix(db): a completed migration overrides an implicit SQLite config (#1038) Review round 11. The migration allowed the one configuration it should have worried about most: DATABASE_CLIENT unset AND NODE_ENV not "production", which resolves to the development block — i.e. sqlite3. That is precisely the state the affected installs are in, since it is why they ended up on SQLite at all, so an operator can easily run the migration before fixing it. The script then renames the source database away, and the next start resolved to the implicit sqlite3, created a NEW empty database and served it — after reporting success. The success marker now overrides an IMPLICITLY resolved sqlite3 when Postgres settings are present, because the marker is durable proof of where the data actually went. An explicit DATABASE_CLIENT=sqlite3 still wins: that is the documented rollback. The script says something rather than refusing — refusing would block exactly the population this exists for. Reproduced with NODE_ENV and DATABASE_CLIENT both empty, against a real PostgreSQL 15: the migration completes, the source is renamed away, and the next boot resolves to pg with the data intact. Before this it resolved to sqlite3 and would have served an empty database. Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM * refactor(db): drop the dead reachability flag in probePgData (#1038) github-code-quality flagged `if (reachable)` as always true, and it is right: the unreachable branch returns, so everything below it runs only when the probe connected. The variable and the conditional were leftovers from a first draft that used a single catch for both failure classes. No behaviour change — the two error paths still return opposite answers. Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM * fix(db): refuse to choose when both databases hold data (#1038) Review round 12. 1. An install that ran on PostgreSQL, lost NODE_ENV/DATABASE_CLIENT, and kept working on SQLite has REAL data on both sides: old rows in Postgres, newer ones in SQLite. The stranded-data rule only protected SQLite when Postgres was empty, so pulling this fix would have booted Postgres and hidden every gallery created since the switch — the exact failure this PR exists to prevent, in a variant I had not considered. A completed migration leaves a marker saying which side is current. Without one, two populated databases are a conflict: the boot stops and prints both targets, the two DATABASE_CLIENT values that resolve it, and the migration command that merges them. This is the only deliberate refusal in the change — guessing here would hide data AND split subsequent writes across two databases. 2. probePgData was handed knexConfig.connection even when knexfile had resolved to SQLite (a completed migration whose environment still says sqlite3), so node-postgres dialled its own localhost defaults instead of DB_HOST/DB_NAME — false "unreachable" diagnostics and a needless delay on every boot. The probe target is now built from the environment when the config is not pg. The conflict is honoured by all three entry points: the resolver exits 3 with an empty stdout, wait-for-db.sh stops the container, and server.js refuses to start. Two existing tests asserted that Postgres wins when both sides hold data. They encoded the pre-conflict assumption and described a state that cannot occur after a real migration (which always leaves a marker); both now pass the marker. Found while testing: the resolver's logger shim had no .error, so the conflict path threw, was swallowed by the fallback, and silently chose Postgres — the precise outcome this refuses to make. The shim is complete now. Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM * fix(db): symmetric bootstrap rule, one resolved Postgres target (#1038) Review round 13. Both findings are consequences of earlier rounds. 1. The conflict rule added in round 12 counted an untouched SQLite bootstrap admin as data. core/001_init.js seeds one whenever ADMIN_PASSWORD is set — including into the accidental SQLite database — so a healthy Postgres install that had ever started once without NODE_ENV would have had a seeded-only SQLite file beside it, been declared a both-populated conflict, and REFUSED TO BOOT. The bootstrap discrimination is applied on both sides now; a setup-completed or logged-in admin still counts as real use on either. 2. The CLI's child phases inherited whichever knexfile block NODE_ENV selected. The development block defaults Postgres to localhost/postgres/photo_sharing, production to db/picpeak/picpeak — and this script is explicitly meant to run with NODE_ENV unset. With DB_USER/DB_NAME left to defaults it would therefore have migrated into `photo_sharing`, after which following the script's own advice to set NODE_ENV=production pointed the app at an empty `picpeak`. The target is resolved once, with production defaults, and passed explicitly to every phase — so the block knexfile happens to pick can no longer decide which database the data lands in. The pin and success marker record that same resolved identity. Verified against a real PostgreSQL 15: a live Postgres beside a seeded-only SQLite file now boots pg rather than refusing, flipping that admin to setup-completed restores the conflict, and a migration records localhost:7102/picpeak_r13b as its target rather than a defaulted guess. Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM * fix(db): one Postgres identity everywhere; protect the credentials file (#1038) Review round 14. Three of the six findings were the same defect as round 13's, surfacing through paths that fix did not cover: the connection used to PROBE or MIGRATE could differ from the one the application then OPENS, because knexfile's development block points Postgres at localhost/postgres/photo_sharing while production uses db/picpeak/picpeak. 1. server.js exported only DATABASE_CLIENT=pg after the resolver decided, so knexfile filled in host/user/database from whichever block NODE_ENV selected. With SQLite already retired by a migration, that meant opening an empty database. The whole connection is pinned now. 2. Two defaults existed for DB_HOST: wait-for-db.sh resolves and exports `postgres`, knexfile's production block says `db`. Since the entrypoint exports its value, `postgres` is what a running container actually uses — so a `docker exec` migration, which inherits neither, has to agree with that, not with the default that is only reached when the entrypoint did not run. 3. The migration's Postgres phases inherited an unset NODE_ENV and therefore the development block, which ignores DB_SSL entirely — a managed Postgres requiring TLS could never be migrated into. The phases run with production semantics now. 4. core/001_init.js writes data/ADMIN_CREDENTIALS.txt, and that data directory belongs to the SOURCE install. Bootstrapping the Postgres schema replaced the operator's real credentials file with ones for a temporary admin the import immediately discards. The file is preserved across the phase, including when it fails. 5. The boot line described knexConfig, so an install redirected to Postgres by a migration marker still logged "Database engine: sqlite (...)", contradicting the warning printed one line earlier. 6. On a both-populated conflict resolveBootEngine returns client:null, and both migration runners told the operator their data was in "null" and to set DATABASE_CLIENT=null. They now present the two real choices. Verified against a real PostgreSQL 15: a migrated install started directly with NODE_ENV unset now logs `postgres (localhost:7102/picpeak_r14)` and opens it, where before it would have gone to the development block's photo_sharing. Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM * refactor(db): resolve the PostgreSQL target in exactly one place (#1038) Rounds 13 and 14 both traced back to the same thing, each time through a caller the previous fix had not covered: three different defaults existed for the same connection. knexfile development : localhost / postgres / photo_sharing knexfile production : db / picpeak / picpeak wait-for-db.sh : postgres / picpeak / picpeak (and it EXPORTS them) So a process that probed or migrated against one could hand over to a process that opened another. Patching each caller was not converging — the guard, then the CLI's child phases, then server.js — so this deletes the divergence instead. `src/utils/pgConnection.js` now owns the resolution and knexfile's development and production blocks both derive from it, as does the engine guard. Same shape as the earlier sqlitePath.js extraction, for the same reason. The database NAME is what made this dangerous: a wrong host or user fails loudly at connect time, while a wrong name connects fine and presents an empty installation. BEHAVIOUR CHANGE: with DATABASE_CLIENT=pg and no DB_* variables, a non-production environment now resolves to postgres/picpeak/picpeak instead of localhost/postgres/photo_sharing. Deployments are unaffected — compose sets these explicitly and wait-for-db.sh exports them — but a local machine running Postgres bare now needs DB_HOST=localhost DB_USER=postgres DB_NAME=photo_sharing (or DATABASE_CLIENT=sqlite3, which is what backend/.env already uses). The failure mode of getting this wrong is a refused connection, not a silently empty database. Side effect worth having: DB_SSL is now honoured whatever NODE_ENV says, so the managed-Postgres case is fixed at the root rather than by forcing production semantics onto the migration's child phases. The test block keeps its own photo_sharing_test default — isolation is the point there. Verified: every block plus the guard resolve identically from the same environment; explicit DB_* still wins; production's pool tuning is preserved; and a full SQLite → PostgreSQL migration with NODE_ENV unset lands in the right database with JSON shapes intact. Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM * fix(db): two more components that guessed the database instead of asking (#1038) Both found while sweeping for copies of the connection defaults. Checked in detail first — one of my suspicions about them was wrong. scripts/set-admin-password.js hand-rolled its own knex config while all four sibling scripts (reset-admin-password, create-admin, show-admin-credentials, reset-admin-mfa) use the application's connection. Two consequences: - it read DB_CLIENT, a variable nothing else in this codebase sets, so it defaulted to Postgres and could not work on a SQLite install at all; - it defaulted to database `picpeak_dev`, a name no other component uses. It now uses `require('../src/database/db')` like its siblings, so it follows whatever engine the install actually runs on. Timestamps are written as ISO strings because it reaches SQLite now, where raw Date objects are the documented landmine. NOT changed: the script's "all existing sessions have been invalidated" notice is accurate — auth.js compares token iat against password_changed_at — and it deliberately leaves must_change_password alone, which is right for an operator choosing a password rather than being issued one. routes/adminSystem.js re-derived three things the live connection already knows, and each could disagree with it: - the engine, from DATABASE_CLIENT || 'sqlite3' — so a Postgres install without an explicit DATABASE_CLIENT took the SQLite branch; - the Postgres database, from DB_NAME || 'picpeak'; - the SQLite file, from a hardcoded ../../data/photo_sharing.db that ignored DATABASE_PATH entirely. All three now come from db.client.config, with pg_database_size(current_database()). Verified: set-admin-password works on SQLite (new hash verifies, old rejected) and still on PostgreSQL; and on a SQLite install with a custom DATABASE_PATH the size logic reports the real database (1,748,992 bytes) where the old code reported a different file entirely (1,851,392) — or 0 where that path does not exist. Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM * fix(db): bind the migration marker to its target; fix a phantom table (#1038) Review round 15. 1. The marker records `host:port/database`, but only its EXISTENCE was checked. Repoint DB_NAME or DB_HOST at a different, empty PostgreSQL after migrating and the marker would vouch for that one too — booting it, presenting an empty installation, and suppressing the SQLite fallback while the real data sits in the recorded target and the renamed rollback copy. The marker is compared against the current connection now, and a mismatch stops the boot with both targets named and the two ways out. 2. `incoming_invoices` is not a table — supplier documents live in `inbound_documents` (core migration 124). Both occupancy lists skip tables that do not exist, so those records were silently not protecting anything: an install whose only remaining data was inbound documents could be switched away from, or overwritten without --force. Verified every other name in the lists against the live schema at the same time. Verified: a marker naming picpeak_original with picpeak_mk configured refuses with exit 3 and prints both; making them agree boots pg. Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local> |
||
|
Paul Nothaft
|
89dc9623c1 |
fix(feedback): persist guest feedback settings, unshadow the guest route (#1030) (#1031)
Enabling Guest Feedback on an event could silently do nothing.
1. `updateEventFeedbackSettings` spread the request body straight into the
knex UPDATE. The admin event form posts its whole client-side state,
including three keys that were never columns on event_feedback_settings
(`enable_rate_limiting`, `rate_limit_window_minutes`,
`rate_limit_max_requests`), so the write threw and the route answered 500.
Writable columns are now whitelisted; identity columns and timestamps stay
server-managed.
2. EventDetailsPage swallowed that 500 in a bare `catch {}` ("Error already
handled by mutation" — it is a different request), so the admin was left
looking at "Event updated successfully" while the toggle never persisted.
The error is surfaced now and the settings query is invalidated on success.
3. gallery.js declared a duplicate `GET /:slug/feedback-settings`. server.js
mounts galleryRoutes before galleryFeedback, so it shadowed the real
handler and dropped the per-guest caps (#655) from the guest payload — the
gallery could never render the favorite/like limits or their counters.
Timestamps are written as ISO strings so they round-trip on both engines.
Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM
Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
|
||
|
Paul Nothaft
|
671c4dbd56 |
fix(events): make event_date/expires_at nullable on SQLite (#1029) (#1035)
Clearing a gallery's expiration failed on every SQLite install with
SQLITE_CONSTRAINT: NOT NULL constraint failed: events.expires_at
surfacing in the admin UI as "Failed to update event".
Migration 061 added the event_require_event_date / event_require_expiration
settings and dropped the NOT NULL on both columns — but only for Postgres. It
skipped SQLite on the premise that "SQLite doesn't enforce NOT NULL as
strictly", which is untrue, so "never expires" was never reachable there. The
#426 work that allows clearing the expiration on edit therefore never worked
on SQLite either.
Migration 174 finishes 061 for SQLite. Knex implements .alter() on SQLite by
recreating the table; migration 073 already does that on `events`, so the path
is well-trodden. Postgres is skipped — it was handled in 061 and .alter() there
would needlessly rewrite a column that is already correct.
The test asserts against the real engine (the Jest harness runs SQLite) and
reproduces the reporter's exact error without the migration.
Claude-Session: https://claude.ai/code/session_0168gubtwYYacJv8weAjy8DM
Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
|
||
|
Paul Nothaft
|
1b4e5fee3e |
fix(security): bound inbound-mail resources, redact secrets from logs (GHSA-2qf9, pgmp, r794) (#959)
* fix(security): bound inbound-mail resources, redact secrets from logs (GHSA-2qf9, pgmp, r794) GHSA-2qf9 — emailIntakeService downloaded, parsed and persisted every message with no size, attachment-count or attachment-byte limit, reachable unauthenticated by anyone who can email the operator's mailbox: - fetch the envelope with `size` (same cheap pass) and refuse an oversized message BEFORE downloading its source; - cap attachment count and cumulative attachment bytes; - limits env-overridable, defaults generous for real supplier invoices. The teeth were in the dedup key. received_emails.message_id is varchar(512) UNIQUE, and the failure path wrote `err-<uid>-<Date.now()>`, which can never match the envelope-derived messageId the dedup pass compares against — so an oversized (or overlong-Message-ID) mail was re-downloaded every poll forever, and an OOM-kill/restart just resumed the loop. Size-skips are now recorded under the REAL message id, and overlong ids collapse to a stable sha256 key that always fits the column. GHSA-pgmp / r794 — new sanitizeForLog() util (key-name deny-set, recursive, cycle-safe) applied to the three request-body log sites in adminEvents/crud.js, plus sanitizeValidationErrors() because express-validator's errors.array() embeds the SUBMITTED value per field — a rejected plaintext password was still logged. Scope is wider than filed: the update path also logged client_password_hash and a LIVE client_share_token bearer credential. Also: the one-time setup token was logged at warn AND printed to stdout on every first boot, putting a live first-admin credential in combined.log, security.log and `docker logs`. It is now written to the 0600 token file and only surfaced when that write fails — the last-resort path it existed for. * fix(security): codex round 3 — repair the first-run token recovery flow (GHSA-r794) Two regressions from keeping the setup token out of the logs. 1. server.js decided whether to print the token by calling existsSync() on the candidate path. That answers a different question than "did the write succeed": a stale, read-only or directory-shaped SETUP_TOKEN reports as present, so the banner suppressed the live token and pointed the operator at content that is not it — leaving the current token only in combined.log under default production logging. setupService now records the path the write actually produced and exposes it via writtenSetupTokenFile(). 2. The setup screen, its EN/DE strings, README, SIMPLE_SETUP and .env.example all still told first-time users to run `docker compose logs backend | grep -i "setup token"`. On the normal path that command now returns a path banner and no credential, so the documented browser-first onboarding could not be completed. They now point at `docker compose exec backend cat /app/data/SETUP_TOKEN`, with the log fallback described as what it is — the failure path. Claude-Session: https://claude.ai/code/session_01F211U4dDbEj4zXiyKbi9me --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local> |
||
|
Paul Nothaft
|
9050affd8d |
fix(security): unauth share_token leak (HIGH) + restore path-traversal, logo file-read, branding path keys (#946)
* fix(security): stop unauth share_token leak + block restore path-traversal, logo-path file read, branding path keys * test: update resolveLogoFile for the c7x5 containment (reject outside-storage absolute paths, keep inside) * fix(security): codex round-1 — escape LIKE wildcards in share-link resolve, keep in-storage absolute logos, guard restore verification - shareLinkService: escape %/_ in the link_partial LIKE fallback so an anonymous /resolve/____… wildcard can't match an arbitrary share_link and leak its bearer token (reopened GHSA-rh8r). Explicit ESCAPE for SQLite. - resolveLogoFile: re-add the raw absolute candidate but keep it subject to the storage-root containment filter (GHSA-c7x5) so legit in-storage absolute logos resolve while /etc/passwd stays rejected. - restoreService: apply the same pathEscapes guard in post-restore verification so a skipped traversal entry isn't fs.access'd/hashed. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local> |
||
|
Paul Nothaft
|
b7005692b3 |
fix(security): resolve DNS before vetting external hostnames (SSRF cluster) (#941)
* fix(security): resolve DNS before vetting external hostnames (SSRF cluster) * fix(security): harden SSRF fix per review (rsync backup path, S3 config-save, webhook transient-DNS retry) * fix(security): S3 endpoint validation on any endpoint update + no-connect on unresolved webhook host (codex r2) --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local> |
||
|
Paul Nothaft
|
8a87c9274b |
fix(security): block guest access to hidden/client-only photos across bulk + secure routes (#939)
* fix(security): block guest access to hidden/client-only photos across bulk + secure routes * fix(security): harden hidden-photo fix per review (stale ZIP cache, legacy token mint, SQLite bool, client rebuild) * fix(security): invalidate ZIP cache on photo visibility/category change (codex r2) * fix(security): recheck photo visibility at signed/secure serve time (TOCTOU) + invalidate ZIP on client visibility change (codex r3) --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local> |
||
|
Paul Nothaft
|
defeae9634 |
fix(uploads): prevent cross-photo contamination from filename collisions and non-atomic writes (#931) (#933)
* fix(uploads): prevent cross-photo contamination from filename collisions and non-atomic writes (#931) * test: pin the suffixed photo filename format in the NFD pipeline suite (#931) * test: make the suffix-uniqueness check deterministic-in-practice (#931) * fix(uploads): widen the anti-collision suffix to 48 bits (#931) * fix(uploads): hide staging files from list() + share one watermark limiter process-wide (#931) * fix(uploads): reclaim orphaned staging files + revalidate watermark settings in queued jobs (#931) --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local> |
||
|
Paul Nothaft
|
d9ad982373 |
fix(tests): raise migration-boot hook timeout pins to the 120s default (#900)
The 3.97.0-beta.0 release PR (#899) failed its backend Tests job on slideshowPublic.test.js: bootCrmDb's full migration chain crossed the suite's explicit 30s beforeAll timeout argument on a slow runner. #860 raised the config default and the jest.setTimeout pins to 120s, but hook-ARGUMENT pins override the config default and were left behind — same time-bomb, different syntax. Every beforeAll that boots the migration chain and pinned 30s/60s is raised to 120000 (16 suites). Untouched on purpose: the three suites whose pinned hooks don't run migrations (webhookDelivery, imageProcessor.storage, storageBackend) and publicQuotes' 30s pin on the rate-limit lockout test — neither grows with the migration chain. No test logic changed. Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local> |
||
|
Paul Nothaft
|
6a048d08bd |
feat(feedback): let guests remove their star rating (#884) (#893)
* feat(feedback): let guests remove their star rating (#884) Clicking your current rating again clears it. rating: 0 is the wire contract: the validator now accepts 0, and the service deletes the guest's rating row (instead of storing a 0 that would drag the photo average down) and recalculates photo stats. The lightbox stars send 0 on a same-star click; PhotoRating already did, but the backend rejected it with a 400 until now. * fix(feedback): harden the rating-clear path (#884 review round) External review follow-ups: numerically normalize the clear sentinel so a numeric-string "0" can't slip into the update/insert paths (validator now also toInt()s), delete the full guest-scoped rating set on clear so racy duplicate rows can't survive in the average (same defense as the reaction path), and refresh the visible average/count after the identity-modal submit path like the direct paths do. * fix(feedback): round-2 review fixes for rating clear (#884) - Clear sentinel matches only an explicit 0 / "0" — malformed input (undefined, NaN, garbage strings) can no longer delete a rating. - Lightbox survives the photo list shrinking while open (clearing your rating under the Rated filter drops the photo on refetch): index is re-anchored and the lightbox closes when the list empties, instead of crashing on an out-of-range index. - Story layout gets the same same-star-to-clear behavior, keyed off the session-local my-rating map, and an explicit 0 no longer falls back to displaying the photo average. * fix(feedback): refresh guest-scoped caches after rating changes (#884 review round 3) - GalleryView's onFeedbackChange now also invalidates ['my-feedback', slug]: in guest identity mode the Rated/Liked filter membership and chip counts come from that query (#538), so a cleared rating never left the Rated filter until the 30s staleTime lapsed. - PhotoRating invalidates gallery-photos + my-feedback on success: the parent refetch fires optimistically in onMutate and could capture pre-mutation state, with nothing refreshing after the server accepted. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local> |
||
|
Paul Nothaft
|
3d6c9848dc |
feat(feedback): emoji reactions on photos (#839) (#855)
* feat(feedback): emoji reactions on photos (#839) Per-photo emoji reactions from a fixed curated set (❤️ 😂 😍 👏 🎉), one reaction per guest per photo — same emoji toggles off, another switches in place. Stored as feedback_type='reaction' rows with per-guest scoping identical to likes (guest_id when present, device hash otherwise). - migration 164: allow_reactions toggle (default on, still gated by the opt-in feedback_enabled master switch), photo_feedback.reaction value column, denormalized photos.reaction_count - emoji whitelist enforced in the route validator AND the service (shared constants/reactions.js, mirrored in the frontend) - per-emoji tallies + my_feedback.reaction in the photo feedback endpoint; hidden-by-moderator reactions leave all counts - reactions ride the existing rate limiting (like-tier), guest identity modes, and moderation actions; long + pivot exports carry the emoji - gallery: reaction bar in the photo feedback panel (grid lightbox); admin: allow_reactions toggle next to likes, analytics tile, create/duplicate event paths - i18n for all 8 locales; 9 service-level tests * fix(feedback): reach reactions without comments; numeric analytics totals (#839) - the lightbox feedback-panel toggle was gated on allow_comments only — with comments off the new reaction bar was unreachable; the gate now opens for comments OR reactions - the analytics summary now coerces Postgres string counts to numbers: total_feedback concatenated instead of adding ("00006") * fix(feedback): harden reactions from review round 1 (#839) - per-emoji tallies are gated on show_feedback_to_guests — with sharing off a guest sees only their own selection, no aggregate counts - reaction toggle/switch operate on the guest-scoped row SET, so rows duplicated by the (like-parity) check-then-insert race collapse on the next interaction instead of counting twice - rate-limit defaults merge UNDER the persisted settings object — stored rows predating the reaction key otherwise dropped it to the generic 100/h fallback - optimistic revert uses the pre-mutation value via mutation context; the onError closure sees the post-optimistic render, so the old revert froze the wrong state on failed toggles * fix(feedback): review round 2 — hide reaction_count with sharing off, admin list shows emoji (#839) - summary.reaction_count is gated on show_feedback_to_guests like the per-emoji map, keeping the "no aggregates while sharing is off" promise consistent - the admin feedback list renders the reaction emoji on reaction rows and the type filter gains a Reactions option (7 locales; es has no types block and falls back to EN defaults) * fix(feedback): register reaction activity types with translated labels (#839) photo_reaction / guest_feedback_reaction are logged by the submission paths but were absent from the frontend activity-type union and the admin.activities label maps — the recent-activity feed would have shown the raw identifiers. All 8 locales. * feat(feedback): reactions in guest CRM and the premium gallery layout (#839) - guest CRM: per-guest reaction counts in the list aggregation and a Reacted tab (photo grid with emoji badges) + stats card in the guest detail modal; picks/aggregate/exports stay selection-only by design - premium layout: its own yet-another-react-lightbox now gets a fixed reaction-bar overlay (per-photo fetch, optimistic switch) — reactions were otherwise unreachable in this layout since it bypasses the shared PhotoLightbox - allowReactions threaded through the layout feedbackOptions; guest i18n keys for the 7 locales that carry the guests block * fix(feedback): portal the premium reaction bar to document.body (#839) Inside the layout tree an ancestor stacking context (framer-motion transforms) painted the bar under yarl's body-level portal — visible but unclickable, every tap landed on the slide image. As a direct body child the z-index 10000 genuinely wins over yarl's 9999. Verified by clicking through in the running app. --------- Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local> |
||
|
Paul Nothaft
|
8060fedf6a |
fix(security): read the password-complexity key the settings UI writes (#843)
* fix(security): read the password-complexity key the settings UI writes The settings UI saves the admin's complexity choice as security_password_complexity (useSettingsState.ts prefixes security_ to password_complexity), but getPasswordComplexitySettings() queried security_password_complexity_level — written by nothing — so the setting was silently ignored and password validation always used the 'moderate' default. Spotted in the filpgame fork (their main, 2026-07-14). * fix(security): accept the Postgres json-column shape of the complexity value (codex review of #843) On SQLite the TEXT column returns the JSON-stringified value ('"very_strong"'), but on Postgres (production default) setting_value is a json column and arrives already decoded ('very_strong') — the bare JSON.parse threw and the outer catch silently fell back to 'moderate' again. Parse with fallback, mirroring getAppSetting's documented pattern; test now covers both driver shapes + the empty-value default. |
||
|
Paul Nothaft
|
340d91bdd5 |
feat(security): harden .picpeak restore robustness — sessions, roles, sequences
Implements the three restore-hardening items deferred from the #811 Codex review (all validated against a real Postgres, see __tests__/integration/ picpeakRestorePg.test.js). Backend-only; targets main (feature, not a backport). 1. Global session cutoff (utils/sessionCutoff.js). A restore reassigns admin/ customer/event ids, so ANY pre-restore JWT can rebind to a different restored principal. Revoking just the importing token wasn't enough. importFromPicpeak now stamps a unix-second cutoff in app_settings after the restore commits, and adminAuth / galleryAuth / verifyGalleryAccess / customerAuth reject any token whose iat predates it (cached 30s → one in-memory compare on the hot path). The operator's forced re-login mints a token past the cutoff, so it passes. 2. Role preservation across an RBAC replace (captureOperatorRole / preserveOperatorRole). The operator's role + granted permission NAMES are captured before the wipe; after roles/role_permissions are replaced the role is resolved by NAME against the restored data, and re-created with its grants if the backup omits it — so a crafted or cross-instance backup can't silently downgrade or lock out the operator. reinjectCurrentAdmin now returns the operator's id so the row can be re-pointed at the resolved role. 3. Postgres identity-sequence resync (resyncSequences). batchInsert writes explicit ids without advancing the sequences, so the next natural insert into any restored table collided on the PK. Runs AFTER commit (setval isn't transactional) and guards every table with a column-existence check — pg_get_serial_sequence RAISES on id-less tables like role_permissions. No-op on SQLite. Tests: SQLite unit tests for the cutoff and role preservation; a gated Postgres integration suite (npm run test:pg with PICPEAK_PG_TEST_URL) covering sequence resync, the id-less-table guard, explicit-id reinject, role re-creation, and a full cross-instance replaceAllTables run asserting operator preservation, role re-establishment, FK integrity, and collision-free post-restore inserts. Stacks on #811 (shares the reinject hardening); merge after it. |
||
|
Paul Nothaft
|
9cd6b08441 |
fix(security): reject ZIP-slip entries in archive/backup restore (GHSA-jfhw-fj23-fx6x)
node-stream-zip's extract(null, root) writes each entry to path.join(root, entry.name) without neutralising '../', so a crafted archive entry named '../../uploads/logos/evil.svg' escaped the target dir and overwrote arbitrary files (logos, .env, route files → RCE on source deploys). Requires admin with archives.restore. Adds assertZipEntriesWithin() to utils/safePath.js — a lexical containment check run on the entry list BEFORE extract() — and guards both extract sinks: adminArchives.js (the reported route) and picpeakImportService.js (the sibling .picpeak import, same sink). Adds unit tests for traversal, absolute-path, and sibling-prefix entries. |
||
|
Paul Nothaft
|
8c86518aad |
fix(events): NaN from slideshow seed breaks event creation on PostgreSQL
The create route seeds show_interval_ms/show_transition_ms from app_settings through an inline guard that pre-checked Number.isFinite(+v) but then used parseInt(v). The two disagree for null/''/true — +null is 0 (finite) while parseInt(null) is NaN — so when the slideshow settings rows are absent (getAppSetting returns its null default), NaN flowed through Math.min/Math.max into the INSERT. PostgreSQL rejects NaN for integer columns; SQLite silently stores NULL, which is why every SQLite-based test passed while POST /api/admin/events 500'd on the PG dev stack and broke the e2e smoke suite. Fix: parse first, then check — clampIntOrUndefined in utils/numericHelpers (unit-tested against every failure-mode input). Verified end-to-end: the previously-failing minimal create now succeeds against the PG dev stack. |
||
|
Paul Nothaft
|
0b74b51f40 |
Merge pull request #703 from Luca-Timo/ci/whatsnew-highlights-workflow
ci(whatsnew): generate release highlights via GitHub Models |
||
|
Luca
|
5582644dc4 |
fix(whatsnew): decode HTML entities and trim em-dash detail in fallback bullets
The Features-fallback showed raw changelog text, so a commit subject like 'branded URL shortener — /s/<slug> with OG injection' surfaced two problems in the admin banner: - release-please escapes <slug> to <slug>; React renders the literal entity, so the banner read '/s/<slug>'. Decode the entities (< > & " '), & last to avoid double-decoding. - the technical tail leaked into a user-facing highlight. Drop a trailing '— detail' clause (em dash only, so 'mark-paid' is untouched) so the bullet reads as the headline 'branded URL shortener'. Only affects the deterministic fallback; curated <!-- whatsnew --> blocks are unchanged. |
||
|
Paul Nothaft
|
56c2386c90 |
feat(gallery): branded URL shortener — /s/<slug> with OG injection (#699)
Issue 3 from #699 (@alexvaltchev's report): expose a custom-named short URL per event that bots scrape for OG previews and browsers redirect to the underlying gallery. WhatsApp / iMessage / Facebook cache the OG metadata by the URL they crawl, so the SHORT URL becomes the cache key — admins can rotate or split-test underlying gallery URLs without re-pushing a fresh link to clients. Additive feature; no existing route, table, or column is modified. ## Backend - `gallery_short_urls` table (migration 150): id, short_slug UNIQUE, event_id FK CASCADE, target_path TEXT, created_by/at, hit_count, last_hit_at, deleted_at/by. hasTable-guarded so the migration is idempotent on re-run. - `src/services/galleryShortUrlService.js` — validator + CRUD + resolver. Slug rules: `/^[a-z0-9](?:[a-z0-9-]{0,62}[a-z0-9])?$/`, reserved blocklist (admin, api, auth, gallery, og, s, login, ...). target_path snapshots at create-time from the event + global short-URL toggle, so a later flip of the toggle does NOT silently change where existing short URLs resolve. - `src/routes/adminShortUrls.js` — `GET/POST /api/admin/events/:eventId/short-urls`, `DELETE /api/admin/short-urls/:id`. Structured errors: 400 INVALID_SLUG, 409 SLUG_TAKEN (with `suggested`), 404 EVENT_NOT_FOUND. Gated by events.view / events.edit + requireEventOwnership. - `server.js` /s/:shortSlug public route. Bot UA → server-render the same OG metadata the existing /og/gallery/<slug> handler produces, then override og:url to point at /s/<shortSlug> itself (cache-key invariant — social platforms key by the URL they scrape). Browser UA → 302 to target_path. Soft-deleted slug → 410 Gone (intentional-delete signal, distinct from 404 unknown slug). Hit accounting is fire-and-forget. ## Frontend - `services/shortUrls.service.ts` — list/create/remove. - `components/admin/ShortUrlsCard.tsx` — per-event card on the EventDetailsPage. Form for custom or auto-generated slug, list with copy-to-clipboard + soft-delete. SLUG_TAKEN error surfaces the service's `suggested` slug with a "use suggested" button. - i18n: events.shortUrls.* added to EN + DE. ## Tests 78 new tests, all passing: - `__tests__/utils/galleryShortUrlValidation.test.js` (48) — pure- function tests for validateSlug: accepts/rejects, reserved-slug blocklist, path-traversal + URL-injection vectors. - `__tests__/integration/galleryShortUrls.test.js` (19) — service layer against a real SQLite DB. Covers custom + auto-generated slugs, collision + SLUG_TAKEN + suggested, target_path snapshotting (backward-compat invariant), soft-delete + slug rotation, hit counting. - `__tests__/integration/galleryShortUrlRoute.test.js` (11) — HTTP-level: 302 redirect for browser UA, 200 + OG HTML for bot UA, og:url canonical points at /s/<slug>, 410 for soft-deleted + orphaned events, 404 unknown + malformed. Regression sweep: 47 existing migration-chain integration tests still pass; migration 150 is additive only. ## Backward compatibility - Existing `/gallery/<slug>`, `/gallery/<32-hex-share-token>`, `/gallery/<slug>/show/<token>`, `/og/gallery/<slug>`, `/og/gallery/<slug>/cover` routes are untouched. - The `/s/` namespace is new; no existing route lives there. - Migration 150 only ADDs the new table — no ALTERs on existing schema, no destructive changes. - target_path is snapshotted at create-time so flipping the global "Use short gallery URLs" setting after a short URL exists does NOT change where that short URL resolves. |
||
|
Luca
|
500cf8522e |
feat(updates): "What's New" highlights after update + pre-update teaser
Surfaces release highlights to admins, sourced from the GitHub release notes (no AI at runtime). Bullets are written once per release in CI via GitHub Models (see docs/ci/whatsnew-highlights.yml) into a <!-- whatsnew --> block; the app reads that block and falls back to the changelog's "### Features" for releases without it — so it works against today's releases immediately. - backend utils/whatsNew.parseWhatsNew(body): curated block else Features section, strips scope/PR-links, de-dups, caps at 8 (tested). - GET /admin/system/updates/whatsnew: highlights for every version moved through since the per-instance marker (whatsnew_last_seen_version); fresh installs self-anchor silently. Best-effort, never errors. - POST /admin/system/updates/whatsnew/seen: advance the marker (per-instance). - /admin/system/updates also returns latestHighlights for the teaser. - Frontend: WhatsNewBanner (green bar -> modal with "Full changelog" link) on the dashboard via adminService; UpdateNotification shows a "New features include:" teaser. i18n de/en. No migration (uses app_settings). |
||
|
Luca
|
4670292139 |
feat(invoices): optional sub-cent rounding reconciliation ("Rundung" row)
Per-line totals are each rounded to the cent before the net is summed, so
a long time-based invoice can drift a few Rappen from qty × rate — e.g.
68 h × 32.25 = 2193.00, but the 21 rounded line totals sum to 2193.02. This
is the standard "sum of rounded lines" convention (Stripe/QuickBooks/Xero
do the same) and it foots, but some issuers want the total to match the
customer's arithmetic.
New per-issuer setting `crm_invoice_round_total` (default OFF, no migration —
read via getAppSetting with a false default). When on, the create paths store
the full-precision net rounded ONCE (cleanNetMinor), and the drift is shown to
the reader as an explicit "Rundung" row:
Betrag Netto 2'193.02 (= Σ visible line totals, still foots)
Rundung -0.02
Gesamtbetrag 2'193.00
- New util src/utils/invoiceRounding.js (cleanNetMinor) mirrors the
migration-119 hierarchy (priced sub-items override their parent) but sums
at full precision; rate-agnostic, so mixed hourly rates reconcile to one
clean net. Single document-level VAT rate ⇒ one Rundung row.
- computeTotals (quotes) + createInvoice + payload-preview gain the toggle.
- Render contexts derive the row as storedNet − Σ(line totals); legacy/off
documents have equal values ⇒ adjustment 0 ⇒ byte-identical output.
Suppressed on Storno/Mahnung (negated net + sign-flipped lines).
- Storno/tax-report stay correct: both use the stored net scalar, which is
the clean value (createStorno negates net_amount_minor; it never re-sums).
- pdf-i18n: totals_rounding in all 6 locales (de/en/fr confident; nl/pt/ru
machine-translated — flag for native review).
- Frontend: toggle on Settings → CRM (Invoices), default off.
Tests: backend/__tests__/utils/invoiceRounding.test.js (real 68h invoice,
mixed rates, discounts, sub-item hierarchy, no-op case).
|
||
|
Paul Nothaft
|
ab501459a4 |
feat(analytics): pluggable trackers — Umami + Rybbit + Custom (#663 Phase 1)
Implements the hybrid scope agreed on in #663: two native adapters (Umami + Rybbit) for trackers we'd keep maintained, plus a Custom script-paste mode for everyone else (Plausible, Matomo, Pirsch, GA4, GoatCounter, Fathom, Cloudflare Web Analytics). Phase 2 (Plausible native, deeper metrics) explicitly deferred until someone asks. ## Architecture **Backend `services/trackers/`**: - `TrackerAdapter` shape (single method): `fetchDeviceBreakdown` → `{ desktop, mobile, tablet } | null`. Null = route falls back to access_logs heuristic. - `umamiAdapter.js` — extracted from the `services/umamiClient.js` that landed in #662. Same 10 test contract preserved. - `rybbitAdapter.js` — new. Hits `/api/site/{id}/breakdown?dimension= device` with Bearer auth, accepts both bare-array and `{data:[...]}` envelope variants, tolerates `sessions`/`visitors`/`value`/`count` metric keys. - `customScriptSanitiser.js` — sanitize-html with a tracker-tight allowlist (`<script>` / `<noscript>` / `<link rel=preconnect| dns-prefetch>` / `<meta>`). Strips event-handler attributes, `javascript:` and `data:` URLs. - `index.js` factory: `resolveAdapter()` reads `analytics_tracker_provider` setting → dispatches. Back-compat: when provider is unset, infers `umami` from the legacy `analytics_umami_enabled` flag so #662 installs keep working without an admin touching settings. **Backend routes**: - `adminDashboard.js /analytics`: now goes through `resolveAdapter()`. Old `fetchUmamiDeviceBreakdown` direct import removed; both `umamiClient.js` and its test file deleted (replaced by the adapter shape). - `adminSettings.js PUT /analytics`: validates the new `analytics_tracker_provider` enum, sanitises any incoming `analytics_custom_head_html` on save via the sanitiser. Masks the new `analytics_rybbit_api_key` on every GET — same pattern as Umami's API key and recaptcha secret. - `publicSettings.js`: emits `analytics_tracker_provider`, `rybbit_url`/`rybbit_website_id` (only when provider=rybbit), and the pre-sanitised `analytics_custom_head_html` (only when provider=custom). Legacy `umami_*` fields stay for back-compat. **Frontend**: - `analytics.service.ts` reworked into a provider-aware shape. `initialize({provider, ...config})` dispatches to Umami / Rybbit / Custom / None. `track()` calls dispatch to `window.umami.track` / `window.rybbit.event` / no-op based on the loaded provider. - `App.tsx` `AnalyticsBootstrap` reads `analytics_tracker_provider` from public-settings and routes to the right `initialize` call. Legacy `umami_enabled`-based path preserved as fallback when the new field is missing. - `AnalyticsTab.tsx` (Settings → Analytics) reworked with a "Provider" dropdown switching between None / Umami / Rybbit / Custom panels. Each panel renders its own config fields; Custom panel surfaces an explicit CSP-reminder banner. - `useSettingsState.ts` shape extended with `tracker_provider`, `rybbit_url`/`rybbit_website_id`/`rybbit_api_key`, `custom_head_html`. Save mutation keeps `umami_enabled` in sync with `tracker_provider==='umami'` for back-compat with downstream consumers (publicSettings shape, embedded iframe). - `publicSettings.service.ts` type extended. **i18n**: EN + DE for the provider heading + description + dropdown options + Rybbit fields + Custom HTML field + CSP warning. ## Custom mode — script execution caveat When the gallery `<head>` receives the custom HTML, simply assigning innerHTML to a container element wouldn't execute the embedded `<script>` tags (per the HTML spec, dynamically-inserted scripts via innerHTML are non-running). `analytics.service.ts:120-130` re-creates each `<script>` element manually so the browser actually evaluates it. Non-script nodes (link, meta, noscript) move in directly. ## Tests **Backend** (42 cases, all pass locally): - `umamiAdapter.test.js` (10) — pinned from the original `umamiClient.test.js`: missing-config / URL shape / encoding / payload normalisation / `laptop`→`desktop` / unknown buckets / empty / non-2xx / invalid JSON / network error. - `rybbitAdapter.test.js` (9) — same shape adapted for Rybbit: bare-array + envelope payload, `sessions`/`visitors`/`dimension` key tolerance, encoding, failure modes. - `trackerFactory.test.js` (6) — resolves null for `none`/`custom`, correct adapter for `umami`/`rybbit`, back-compat path via legacy `analytics_umami_enabled`, garbage-provider defensive null. - `customScriptSanitiser.test.js` (12) — Plausible-style passthrough, Umami-style passthrough, inline body passthrough, `<noscript>` allowed, `<link rel="preconnect|dns-prefetch">` allowed, `<link rel="stylesheet">` stripped, disallowed tags stripped, `javascript:`/`data:` URLs stripped, `on*` event handlers stripped, defensive on malformed input. - `analyticsDateMerge.test.js` (5) — preserved from #662. **Frontend**: full 84-case vitest suite green; tsc + eslint clean on changed files. Adapter changes are narrow refactors of code covered by backend tests; no new analytics-page unit test added. ## End-to-end smoke (dockerised backend + my changes mounted) ``` test 1 (back-compat: no provider, umami_enabled=true) → factory returns umami adapter, /analytics returns devicesSource:access_logs (umami fetch to fake host fails gracefully). ✓ test 2 (invalid provider value) → 400 "analytics_tracker_provider must be one of: none, umami, rybbit, custom" ✓ test 3 (save custom HTML with XSS payload) → stored sanitised: `<script>alert(1)</script>evil<script async defer data-domain="x.com" src="https://plausible.io/js/script.js"></script>` (<div> stripped; script tags survive but CSP `script-src 'self'` still blocks inline + non-allowlisted external at runtime) ✓ test 4 (public-settings exposes the provider switch) → `analytics_tracker_provider: 'custom'`, `analytics_custom_head_html: '<sanitised>'` ✓ ``` ## Out of scope (next discussions) - **Plausible native** — covered via Custom mode for now; native is Phase 2 if someone explicitly asks. - **CSP "trusted domains" admin input** — Phase 1.5. For now operators add their tracker domain to nginx/proxy CSP manually; the new CSP-reminder banner in the Custom panel makes that clear. - **Refactor `(window as any).umami.track(...)` direct calls** in PhotoLightbox/PhotoGrid to go through `analyticsService.track()` so events fire on the right tracker. Currently a no-op when Umami isn't loaded; functional but not optimal. Closes #663 Phase 1. |
||
|
Paul Nothaft
|
7534447b6c |
fix(analytics): admin dashboard reads correct fields + Umami device API (#661)
Reporter @alexvaltchev hit three independent bugs on the Analytics
Dashboard. All three fixed in one PR; pluggable-tracker support
(Rybbit, Plausible, etc.) left for a separate discussion.
## Bug A — Summary cards showed 0
Two layers, both fixed.
**Frontend** (`AnalyticsPage.tsx:142-149`): the cards summed
`chartData[].views/uniqueVisitors/downloads`. The backend now (and
already) emits a dedicated `totals` object computed via separate
COUNT queries, which is what the cards should read. Postgres returns
counts as strings, so coerce via `Number()`.
**Backend** (`adminDashboard.js:268-282`): the chartData merge used
`dateObj.date === row.date`. On Postgres, pg's driver auto-converts
`DATE(timestamp)` to a JS Date object — the string-equality match
failed silently and `chartData` stayed all-zero on every Postgres
install with traffic. Added a `normaliseDateKey()` helper that
returns YYYY-MM-DD regardless of driver shape, plus `Number()`
coercion on the counts. SQLite path unchanged.
## Bug B — "Umami Not Configured" banner despite valid config
`AnalyticsPage.tsx:90` did `settings.reduce(...)` on the
`/admin/settings` response. That endpoint returns a
key/value **object** (verified at `adminSettings.js:108-149`), not
an array, so `.reduce` threw `data.reduce is not a function` and
the catch silently rendered the "Not Configured" banner even on
perfectly-configured installs. Read the umami keys directly off the
response object.
## Bug C — Device breakdown 0/0/0
Two-pronged fix.
**Primary path — Umami device API** (`services/umamiClient.js`,
wired into `adminDashboard.js`). When the admin provides an Umami
v2 API key (new setting `analytics_umami_api_key`), the backend
fetches the per-period device breakdown from Umami's
`/api/websites/:id/metrics?type=device` endpoint. Umami tracks
devices natively — far more accurate than our coarse user-agent
heuristic. The new `devicesSource` field in the response lets the
UI hint at where the numbers came from.
**Fallback hardening — local heuristic** (`adminDashboard.js:296-320`).
The existing access_logs `LIKE '%Mobile%' / '%Tablet%'` query stays
in place as a fallback for installs without Umami. Hardened with:
`whereNotNull('user_agent')` skips rows we never captured a UA on,
`Number()` coercion on COUNT results (pg returns strings), and a
guard against divide-by-zero when access_logs is empty.
## API key handling
Mirrors the existing recaptcha-secret pattern: stored plaintext in
`app_settings`, masked as `••••••••` on every GET via the existing
`adminSettings.js` GET handlers, and the frontend save mutation
silently drops the masked sentinel so re-saving without typing a
new key preserves the stored value.
## End-to-end smoke (dockerised backend with my fixes applied)
```
chartData total views: 27 ← previously 0 (date merge broken on PG)
totals: {'views': '27', 'downloads': '3', 'uniqueVisitors': '1'}
devices: {'desktop': 100, 'mobile': 0, 'tablet': 0} ← was 0/0/0
devicesSource: access_logs ← falls back correctly
analytics_umami_api_key (GET /settings/analytics): ••••••••
```
## Tests
**Backend** (15 new cases):
- `umamiClient.test.js` (10): missing-config → null, URL shape +
`x-umami-api-key` header, websiteId URL-encoding, `{x,y}` →
percentages, `laptop` → `desktop` mapping, unknown buckets
dropped, empty payload → null, non-2xx → null, invalid JSON →
null, network error → null.
- `analyticsDateMerge.test.js` (5): YYYY-MM-DD string pass-through,
ISO timestamp slice, JS Date (pg shape) → YYYY-MM-DD, null/empty
→ null, coercion for unexpected types.
**Frontend**: full 84-case vitest suite still green (no analytics
unit tests existed before; not adding any here — the changes are
narrow and the unit-level confidence comes from the type system +
the backend smoke above).
Closes #661 (bugs A + B + C). Rybbit / pluggable tracker support is
the next conversation per the issue author's follow-up.
|
||
|
Paul Nothaft
|
f2814e4a4c |
feat(feedback): per-guest favorite + like caps with mobile-friendly limit modal (#655)
Reporter @Duecki1 wants to stop telling guests "pick only 5 photos" by
hand. Per-event cap, enforced server-side, with a clear popup when the
11th click would exceed the limit. Per-guest scope matches the "every
couple picks their top 10" mental model; per-gallery aggregate is
explicitly NOT in scope (creates weird "first 10 visitors use up all
slots" race conditions).
## Schema (migration 141)
Two nullable columns on `event_feedback_settings`:
- `max_favorites_per_guest`
- `max_likes_per_guest`
null / 0 = unlimited (preserves current behaviour for every existing
install — operator must opt in). Both shipped together because the
code path is identical; photographers can cap either, both, or neither.
## Backend
- `feedbackService.submitFeedback` cap check on the INSERT branch only.
Toggle-off (un-favoriting) is always allowed, so a guest at 10/10
can free a slot by un-clicking an existing favorite.
- New `countGuestFeedback(eventId, type, guestId, guestIdentifier)` —
matches the exact same guest-key shape the existing duplicate-check
uses (guest_id when present, fallback to guest_identifier in simple
identity mode).
- Limit reduction grandfathers: admin lowering 20→10 keeps existing
rows in place; new adds blocked until the guest removes some.
- Route layer (`galleryFeedback.js` POST) translates a `limit_reached`
service-return into a structured 403 with `code:
'FAVORITE_LIMIT_REACHED'` / `'LIKE_LIMIT_REACHED'`, `limit`, and
`current_count`. Stable UI contract.
- `feedback-settings` GET exposes the caps so the gallery UI can
optionally render a counter near the heart icon (UI extension TBD;
the modal alone is the contract this PR commits to).
- `feedbackValidation`: range guard `0..10000`, null allowed,
per-field error messages.
## Frontend — the popup
New `FeedbackLimitReachedModal` component renders via a `createPortal`
to `document.body` so it escapes any lightbox / sticky parent stacking
context and reliably sits above everything else.
Mobile-first responsive:
- `items-end sm:items-center` — slides up from the bottom on phones
(native action-sheet feel), centers on desktop (familiar modal).
- `w-full sm:max-w-md` — full-width on phones, clamps to 420px on
desktop.
- `rounded-2xl sm:rounded-xl` — more rounded on phones for the
sheet feel.
- `pb-[env(safe-area-inset-bottom)]` — respects the iOS home indicator
and Android gesture bar.
- `z-[60]` — above the lightbox's z-50.
Title + body + "8 of 10 used" pill + "Got it" button. Backdrop click +
Escape both dismiss. Focus management lands on the OK button so
keyboard / screen-reader users can dismiss immediately.
New `useFeedbackLimitModal()` hook is the shared API: components
on every submit-feedback site wire `onError: (err) => handleError(err)`
and render `{limitModal}` in their JSX. Returns `true` from
`handleError` when the error is a structured cap-reached 403 (so the
caller can skip its generic error toast). PhotoFavorites + PhotoLikes
+ PhotoLightbox all wire through the hook — every favorite/like submit
path is covered, including the lightbox's three different submit
sites (guest mode, simple mode, post-identity-modal-confirm).
## Admin UI
`FeedbackSettings` card gets a new "Per-guest limits" section that
only renders when at least one of `allow_favorites` / `allow_likes` is
on. Two numeric inputs (0 / empty = unlimited) side-by-side on
desktop, stacked on mobile. Hint text covers the limit-reduction
grandfathering semantics so admins aren't surprised.
## i18n
EN + DE for:
- Modal title + body (parameterized with `{{limit}}`)
- Counter pill (parameterized with `{{current}}` / `{{limit}}`)
- OK button label
- Admin field labels + hints + section header + grandfathering note
## Tests
**Backend** (`__tests__/utils/feedbackPerGuestLimit.test.js`, 8 cases):
- null cap → unlimited (back-compat)
- 0 cap → unlimited (UI convenience)
- cap=10: rows 1-10 succeed, 11 returns limit_reached
- toggle-off frees a slot at the cap
- limit reduction grandfathers existing rows
- per-guest scope: guest A's cap doesn't affect guest B
- favorite cap doesn't block likes (per-type)
- like cap returns LIKE_LIMIT_REACHED-shaped payload
**Frontend** (`__tests__/useFeedbackLimitModal.test.ts`, 7 cases):
- Non-axios errors → null
- Non-403 axios errors → null
- 403 with wrong code → null
- FAVORITE_LIMIT_REACHED parsed
- LIKE_LIMIT_REACHED parsed
- Falls back to code-implied type when feedback_type missing
- Missing numeric fields → 0 (not NaN)
All 15 pass. tsc --noEmit clean. eslint clean on changed files.
Closes #655.
|
||
|
Paul Nothaft
|
16055cdc41 |
feat(whatsapp): admin-selectable template parameters + reorder (#647 follow-up)
Reporter @Rekoo-PS confirmed the language fix unblocked sending, then
hit a second gap: their template uses only `{{1}} = event_name` +
`{{2}} = gallery_link`, but the legacy `buildComponents` hardcoded all
5 positional values from the `gallery_ready` shape (customer_name,
event_name, gallery_link, password_line, expiry_date). Meta rejected
with a parameter-count mismatch even after the language matched.
This adds a per-config slot list — which built-in values to send, and
in what positional order — so admins can match templates of any shape
without code changes.
## Schema (migration 138)
Additive `template_params` TEXT column on `whatsapp_configs` (default
empty string = legacy 5-slot behaviour for existing installs). Stored
as a JSON-serialized array of slot keys: `customer_name`, `event_name`,
`gallery_link`, `password_line`, `expiry_date`. Unknown / duplicate /
non-string entries are sanitized out at read time.
## Processor
- `parseTemplateParams(raw)` — defensive parser; falls back to the
5-slot default on empty / malformed / all-invalid input.
- `buildComponents(data, metaLang, params)` — emits ONLY the listed
slots in the listed order, computed via a small switch on slot key.
The password line still receives the locale-specific 🔒 label and
the empty-when-no-real-password sentinel handling.
- Processor reads `config.template_params` once per cycle and passes
the parsed array to `buildComponents` per message.
## Admin route
- GET surfaces `template_params` as the parsed array (default 5-slot
when null/empty).
- PUT round-trips the incoming array through `parseTemplateParams`
before persisting, so the stored value is always the canonical
sanitized JSON.
- Test send rebuilt to use the same `buildComponents` path so the
admin's test message matches their configured slot shape — a
reporter who configures 2 slots gets a 2-parameter test send, not
the legacy 5-parameter payload.
## UI
- `WhatsAppTab` gets a checkbox + up/down list under the Template
language field. Each slot shows its current `{{N}}` position when
checked, an em-dash when unchecked. Live preview below the list:
"Your template will receive: {{1}} = event_name, {{2}} = gallery_link".
- EN + DE i18n for the field labels, hint, preview, and per-slot
human-readable names.
## Tests
- 17 unit tests in `__tests__/utils/whatsappBuildComponents.test.js`
covering: parseTemplateParams sanitization (unknown keys, duplicates,
non-strings, malformed JSON, all-invalid fallback, pre-parsed array
acceptance) and buildComponents shape (reporter's 2-slot case,
reorder, empty list, locale-specific password label, password
sentinel handling, expiry omission).
- All 17 + the 34 existing networkValidation tests pass.
## Migration numbering
Sits at 138 on top of PR #649's migration 137. If #646 (Live Slideshow)
merges before this, #646's own 137 + 138 take precedence and this
needs renumbering to 139. Coordinated via PR #646's review thread.
## Honest caveat
Still no Meta Business API account on my side. Spec-built, sanitizer +
shape unit-tested, lint + tsc clean. End-to-end against Meta needs the
reporter (or a maintainer with an account) to verify. If a real
round-trip surfaces a mismatch, drop it in #647 and I'll iterate.
|
||
|
Paul Nothaft
|
b8211e9944 |
fix(security): close BOLA on photo-export + NAT64 SSRF in URL guard
Two security advisories landed against the open #641 branch — bundling both because they touch independent surfaces and PR #641 is the next beta ship vehicle. **GHSA-9v4w-jrhx-g5wr (BOLA on /admin/photo-export/:eventId/*)** — the three /:eventId-scoped routes in `adminPhotoExport.js` (filtered, filter-summary, export) ran `adminAuth + requirePermission(...)` but not `requireEventOwnership`, so any non-super-admin admin/editor with photos.view (or photos.download) could enumerate + export the photos of events created by other admins — leaking `original_filename`, which routinely encodes client identity. Sibling `adminPhotos.js` applies the middleware on every :eventId route; this file was the single drift. Reporter: Wernerina. **GHSA-wmjx-pc37-272r (NAT64 SSRF in `isPrivateIPv6`)** — the old implementation did naive string-prefix checks (`startsWith('fc')`, `startsWith('fe80')`) and had zero coverage for NAT64 (`64:ff9b::/96` per RFC 6052, `64:ff9b:1::/48` per RFC 8215). On instances with NAT64/DNS64 egress, a webhook URL like `http://[64:ff9b:1::a9fe:a9fe]/` translated through the gateway and reached 169.254.169.254 — exfiltrating cloud metadata (IAM creds) into `webhook_deliveries.response_body`. Rewrote `isPrivateIPv6` to expand the address to its canonical 8-group form, block both NAT64 prefixes, decode embedded IPv4 from IPv4-mapped (`::ffff:0:0/96`) and deprecated IPv4-compatible (`::/96`) forms and re-check via `isPrivateIPv4`, and fail closed on any parse failure. Reporter: tonghuaroot. Added 34 unit tests covering: both NAT64 prefixes in hex + mixed dotted-quad notation, IPv4-mapped IPv6 hex + mixed, deprecated ::IPv4 form, legacy fc00::/fd00::/fe80::/::1/:: cases stay blocked, and public IPv6 (Google/Cloudflare/Google IPv6) negative controls stay allowed. Refs: GHSA-9v4w-jrhx-g5wr, GHSA-wmjx-pc37-272r |
||
|
Luca
|
86dff75898 |
test(accounting): cover export scope, unique-violation detector, PDF page cap
Closes the test gaps from the PR #622 work + the export-scope feature: - export scope: scopeLedger/normalizeScope (exported via _internal) unit tests + renderTaxReportCsv income/cost/all output assertions (income drops supplier rows, cost drops invoice rows, filename gets the scope tag). - isUniqueViolation: Postgres 23505 / SQLITE_CONSTRAINT / "UNIQUE constraint failed" message, false for FK + nullish (the IMAP claim-first race detector). - getRenderedPagePath: out-of-range pages reject with PAGE_OUT_OF_RANGE before touching pdftoppm/disk (the per-file resource bound). |
||
|
Luca
|
cd6d57839b |
fix(accounting): PR #622 blockers — CSV formula injection + IMAP double-ingest race
Blocker 1 — CSV/Banana formula injection. Neither csvEscape (ledgerService) nor the tax-report CSV escape nor the unquoted tab-separated Banana cell formatter prefixed risky leading chars, so an admin-/sender-controlled cell beginning with = + - @ TAB CR executes as a formula when the Treuhänder opens the export. New shared util neutralizeSpreadsheetFormula() prepends a single quote; wired into all three sinks (quoted CSV + unquoted Banana). Unit test pins one of each char. Blocker 2 — IMAP intake double-ingest race. received_emails.message_id was INDEX, not UNIQUE, and the poller ingested attachments BEFORE writing the audit row, so a second replica / rolling-deploy overlap double-ingested the same mail. Migration 128 makes message_id UNIQUE (nulls stay distinct); the intake now CLAIMS the message row (status='processing') BEFORE ingesting — a concurrent claim hits the unique constraint and skips cleanly (shared isUniqueViolation helper). Stale 'processing' rows (worker crashed mid-ingest) are reclaimed after 10 min so no attachment is orphaned. NOT done (deliberate): the suggested UNIQUE on inbound_documents.file_sha256 — that column is a SOFT dedup key by design (manual re-uploads are kept as flagged 'duplicate' rows + duplicate_of_id for the Duplikat disposition); a unique index would break that feature. The file race only yields an extra 'unsorted' row (a data-quality nit, caught by the existing manual Duplikat backstop), not a double-count. Rationale to be added to the PR reply. |
||
|
Luca
|
621ce942b5 |
feat(email): per-weekday business hours + manual queue flush
Move the scheduled-email business-hours floor onto the business profile
as Google-style per-weekday opening blocks (multiple blocks/day for lunch
breaks). Migration 114 adds business_profile.business_hours (JSON) +
scheduled_email_floor_enabled; emailProcessor snaps a queued email to the
next open block, read in the profile timezone. Editor lives under
Settings → Business profile.
Add an admin "Send queued emails now" flush (POST /admin/email/flush-queue)
that drains the queue immediately, ignoring the business-hours floor — the
escape hatch before maintenance/updates. processEmailQueue now takes
{ignoreSchedule, limit} and returns send counts; the scheduled interval
run is unchanged.
|
||
|
Paul Nothaft
|
975a815f99 |
Merge branch 'beta' into fix/email-normalization-574
Resolves a conflict with the CRM merge (#555) that landed on beta between when this branch was cut and now. Two conflict regions in backend/src/routes/adminCustomers.js: 1. **Require block** — both branches added new requires after customerAccountsService. Kept both: this branch's emailNormalization import AND beta's customerHoursService + invoiceService imports (the CRM merge added the hours-billing + invoice-creation paths to this router). 2. **Edit-customer validators** — both branches changed the same set of body() validators in the PUT /:id handler. This branch added the IDENTITY_PRESERVING_NORMALIZE_EMAIL options arg to normalizeEmail; beta changed every body() to optional({ nullable: true }) so passive-customer records that store nulls for missing profile fields don't reject on save. Kept both: the nullable pattern from beta + the email-normalization options from this branch. Preserved beta's explanatory comment about the nullable choice. Also patched one NEW normalizeEmail site the CRM merge introduced: - backend/src/routes/adminCustomers.js:231 — POST /admin/customers now exists (CRM-era customer-create endpoint). Same options arg applied. backend/src/routes/adminBusinessProfile.js has an isEmail() WITHOUT normalizeEmail() on the issuer email — intentional (no normalization means no risk of the Gmail dot-strip bug for that field), no change needed. All 18 normalizeEmail sites now pass IDENTITY_PRESERVING_NORMALIZE_EMAIL. 7/7 regression tests still pass. Lint clean on the merged file. |
