Compare commits

..

6 Commits

Author SHA1 Message Date
Paul Nothaft 1ad8ad5b68 chore(stable): release 3.45.8 (#903)
Build and Push Docker Images / build-backend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-backend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-backend (push) Blocked by required conditions
Build and Push Docker Images / build-frontend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-frontend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-frontend (push) Blocked by required conditions
Build and Push Docker Images / summary (push) Blocked by required conditions
2026-07-29 10:52:37 +00:00
Paul Nothaft 962f1d9586 fix(tests): raise jest timeouts to the 120s convention (stable) (#902)
Stable backport combining #860 (never reached stable) and #900:

- jest.config.js gains testTimeout: 120000 — stable still ran on Jest's
  5s default for anything unpinned, while its migration chain (134 core
  migrations via backports) is nearly as long as beta's.
- All 19 suite-level jest.setTimeout(30000/60000) pins raised to 120s;
  local pins override the config default (#860's rationale).
- All 15 hook-ARGUMENT timeout pins on migration-booting beforeAll
  hooks raised to 120s (#900's rationale — the 3.97.0-beta.0 release PR
  failed on exactly this class on the beta side).

Untouched: the three suites whose pinned hooks don't run migrations
(webhookDelivery, imageProcessor.storage, storageBackend) and
publicQuotes' 30s pin on the rate-limit lockout test.

No test logic changed.

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
2026-07-29 12:48:50 +02:00
Paul Nothaft a7885846ac chore(stable): release 3.45.7 (#881)
Build and Push Docker Images / build-backend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-backend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-backend (push) Blocked by required conditions
Build and Push Docker Images / build-frontend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-frontend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-frontend (push) Blocked by required conditions
Build and Push Docker Images / summary (push) Blocked by required conditions
2026-07-27 07:58:07 +00:00
Paul Nothaft d868aac703 fix(security): close 5 Trivy alerts — postcss/tar bumps + drop npm from the runtime image (stable) (#879)
* fix(security): close the 5 open Trivy alerts — dep bumps + drop npm from the runtime image

Backend deps:
- postcss 8.5.10 -> 8.5.18 (CVE-2026-45623, GHSA-r28c-9q8g-f849; the pin
  exists to force sanitize-html's transitive copy onto a fixed version)
- tar pin/override >=7.5.16 -> >=7.5.21, resolves 7.5.22
  (GHSA-r292-9mhp-454m)

Runtime image:
- Remove the npm CLI from the final stage instead of upgrading it: npm's
  bundled node_modules ship tar 7.5.19 and brace-expansion 5.0.7 (no npm
  release bundles the fixed versions — checked 11.18.0 and 12.0.1), and
  npm never runs in production. wait-for-db.sh now invokes the migration
  runners via node directly. This ends the recurring npm-bundled-CVE
  alert class; the previous 'npm install -g npm@11' line was itself a
  patch for the last batch. (stable)

* fix(restore): run post-restore migrations via node — the image ships no npm

restoreService still shelled out to 'npm run migrate:safe' after a
restore; with npm removed from the runtime image that would ENOENT into
the non-fatal catch, silently leaving a restored older backup on a
schema behind the running code until the next container restart. Invoke
migrations/run-migrations-safe.js through node directly, matching
wait-for-db.sh. The PR #596 source-contract test now pins the new
invocation. (stable)
2026-07-27 09:54:43 +02:00
Paul Nothaft 577b7fa6ae chore(stable): release 3.45.6 (#877)
Build and Push Docker Images / summary (push) Blocked by required conditions
Build and Push Docker Images / build-backend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-backend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-backend (push) Blocked by required conditions
Build and Push Docker Images / build-frontend (linux/amd64, ubuntu-latest) (push) Waiting to run
Build and Push Docker Images / build-frontend (linux/arm64, ubuntu-24.04-arm) (push) Waiting to run
Build and Push Docker Images / merge-frontend (push) Blocked by required conditions
2026-07-27 07:10:18 +00:00
Paul Nothaft a27c705e39 fix(backup): make backup settings actually apply (#871) (stable) (#875)
* fix(backup): make backup settings actually apply (#871) (stable)

- Wire the What-to-Backup toggles into the walker: honor
  backup_include_thumbnails / backup_include_photos (opt-out,
  default ON) and accept the UI's backup_include_archives spelling
  for the archived gate (the engine expected _archived, so the
  Archives checkbox silently never worked).
- Fix the 167.6 TB dashboard size: file_size_bytes is a bigint that
  node-postgres returns as a string, and the S3 path concatenated it
  onto the byte counter; coerce to Number at the source.
- Compute the real next scheduled run (cron-parser) and return it as
  nextBackup; the UI read a field the API never sent and rendered a
  hardcoded 'Not scheduled'. A named schedule label now beats the
  stray default cron the UI always sent, which silently turned
  weekly schedules into daily 03:00 runs.
- Never back up filesystem noise (.nfs* silly-renames, .DS_Store,
  Thumbs.db) and honor backup_exclude_patterns in the walker
  (previously rsync-only).
- Remove the compression/encryption toggles from the configuration
  UI: no backend implementation exists, and collecting an encryption
  passphrase while uploading plaintext is a false promise.

* fix(backup): close the review gaps in the settings wiring (stable)

- The UI's backup_include_archives now beats the migration-seeded
  backup_include_archived: every install has the singular key seeded
  true, so the alias-only-when-absent lookup made unchecking Archives
  a no-op.
- rsync destinations now receive the de-selected What-to-Backup paths
  and the noise filters as anchored --exclude args; previously rsync
  synced the whole storage root and the walker's selection only shaped
  the manifest, which then misreported what was actually transferred.
- Escape regex metacharacters in the walker's glob matcher: '.nfs*'
  compiled to /^.nfs.*$/ whose leading dot matched any character, so
  files like anfs-photo.jpg were silently dropped from backups.
- The Backup Coverage report now uses the same gate as the walker
  (new 'skipped-by-setting' status) instead of re-implementing it
  without the opt-out toggles and the archives alias.

* fix(backup): make the coverage diagnostics agree with the walker

- The coverage table shows the alias-aware flag value the gate actually
  used, instead of the seeded backup_include_archived shadowed by the
  UI's plural key (true next to a 'Gated off' badge).
- skipped-by-setting paths are now counted in the coverage summary
  (backend, TS contract, summary card, EN/DE locales) so the totals
  reconcile again when Photos or Thumbnails is unchecked.
- The form's thumbnail default now matches the backend's never-saved
  fallback (include): the checkbox no longer shows 'off' while
  thumbnails are being backed up, and saving an unrelated setting no
  longer flips the backup scope. (stable)

* fix(backup): keep custom crons, exclude disabled rows from rsync, normalize flag display

- Saving a named schedule no longer wipes the stored custom cron: the
  backend already prefers the label, so the cron field stays inert for
  named schedules and is preserved for switching back to Custom. A
  custom schedule now validates the 5-field expression before saving
  (the backend silently fell back to daily 02:00 on a blank value).
- resolveExcludedBackupPaths now also returns rows disabled via
  include_in_default, so rsync excludes them; the enabled-only loader
  hid them and rsync transferred their contents anyway.
- The coverage table normalizes flag values like the walker does —
  Boolean('false') displayed true beside a gated-off badge. (stable)
2026-07-27 09:06:50 +02:00
49 changed files with 562 additions and 217 deletions
+1 -1
View File
@@ -1 +1 @@
{".":"3.45.5"}
{".":"3.45.8"}
+21
View File
@@ -5,6 +5,27 @@ All notable changes to PicPeak will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [3.45.8](https://github.com/PicPeak/picpeak/compare/v3.45.7...v3.45.8) (2026-07-29)
### Bug Fixes
* **tests:** raise jest timeouts to the 120s convention (stable) ([#902](https://github.com/PicPeak/picpeak/issues/902)) ([962f1d9](https://github.com/PicPeak/picpeak/commit/962f1d95868251ddeb01a5234c8d9bd1d57429e8))
## [3.45.7](https://github.com/PicPeak/picpeak/compare/v3.45.6...v3.45.7) (2026-07-27)
### Bug Fixes
* **security:** close 5 Trivy alerts — postcss/tar bumps + drop npm from the runtime image (stable) ([#879](https://github.com/PicPeak/picpeak/issues/879)) ([d868aac](https://github.com/PicPeak/picpeak/commit/d868aac70300149e77fb9568735b6481f88a644f))
## [3.45.6](https://github.com/PicPeak/picpeak/compare/v3.45.5...v3.45.6) (2026-07-27)
### Bug Fixes
* **backup:** make backup settings actually apply ([#871](https://github.com/PicPeak/picpeak/issues/871)) (stable) ([#875](https://github.com/PicPeak/picpeak/issues/875)) ([a27c705](https://github.com/PicPeak/picpeak/commit/a27c705e392ec1e6b8d8be945127de9ff11d5db0))
## [3.45.5](https://github.com/PicPeak/picpeak/compare/v3.45.4...v3.45.5) (2026-07-26)
+10 -8
View File
@@ -37,14 +37,16 @@ ARG CACHEBUST=1
# Upgrade all packages to fix security vulnerabilities (OpenSSL, libexpat, BusyBox CVEs)
RUN echo "cachebust=${CACHEBUST}" && apk upgrade --no-cache
# Upgrade the npm CLI in the final image so its bundled deps are patched
# (sigstore 4.x, tar) — closes CVE-2026-48815 and the older @sigstore/core / tar
# Trivy alerts. Safe here: only the CLI present in the image changes. Runtime
# dependencies come from the builder stage (COPY --from=builder node_modules
# below) and the entrypoint runs node, not npm — so npm 11's install behaviour
# (the reason 10.x was pinned) never executes in this stage. npm 11 needs
# Node >=22.9, satisfied by node:22-alpine.
RUN npm install -g npm@11
# Remove the npm CLI from the final image. Nothing runs npm here: the
# entrypoint is node, runtime deps are COPY'd from the builder stage, and
# wait-for-db.sh invokes the migration runners via node directly. npm's
# bundled node_modules kept tripping Trivy (sigstore, tar 7.5.19,
# brace-expansion 5.0.7 — even npm 12.0.1 still ships the vulnerable
# copies), so shipping no npm ends that alert class instead of chasing
# per-release patches. Note: `docker exec … npm run <script>` no longer
# works in the container — use `node migrations/run-migrations-safe.js`
# and friends instead.
RUN rm -rf /usr/local/lib/node_modules/npm /usr/local/bin/npm /usr/local/bin/npx
# Install dumb-init for proper signal handling, postgresql-client for database
# checks, ffmpeg for video upload support, and su-exec for the root → nodejs
@@ -40,7 +40,7 @@ jest.mock('../../src/middleware/permissions', () => ({
requirePermission: () => (_req, _res, next) => next(),
}));
jest.setTimeout(30000);
jest.setTimeout(120000);
describe('GET /api/admin/system-health/backup-coverage', () => {
let db;
@@ -29,7 +29,7 @@ jest.mock('../../src/middleware/permissions', () => ({
requirePermission: () => (_req, _res, next) => next(),
}));
jest.setTimeout(30000);
jest.setTimeout(120000);
describe('GET /api/admin/system-health/backup-integrity', () => {
let cleanup;
@@ -23,7 +23,7 @@ const path = require('path');
const { bootCrmDb } = require('./helpers/crmDb');
jest.setTimeout(30000);
jest.setTimeout(120000);
describe('backupService — configurable walker (backup_paths)', () => {
let db;
@@ -177,4 +177,203 @@ describe('backupService — configurable walker (backup_paths)', () => {
const filesOn = await backupService.getFilesToBackup(true);
expect(filesOn.map((f) => f.relativePath)).toContain('events/archived/E3/legacy.jpg');
});
// Issue #871 — the "What to Backup" checkboxes were stored but never read.
describe('UI opt-out toggles (issue #871)', () => {
it('unchecking Thumbnails excludes thumbnails/', async () => {
seedFile('thumbnails/E1/thumb.jpg');
seedFile('events/active/E1/photo.jpg');
const files = await backupService.getFilesToBackup({
backup_include_thumbnails: false,
});
const rels = files.map((f) => f.relativePath);
expect(rels).toContain('events/active/E1/photo.jpg');
expect(rels).not.toContain('thumbnails/E1/thumb.jpg');
});
it('unchecking Photos excludes events/active', async () => {
seedFile('thumbnails/E1/thumb.jpg');
seedFile('events/active/E1/photo.jpg');
const files = await backupService.getFilesToBackup({
backup_include_photos: false,
});
const rels = files.map((f) => f.relativePath);
expect(rels).toContain('thumbnails/E1/thumb.jpg');
expect(rels).not.toContain('events/active/E1/photo.jpg');
});
it('defaults to including everything when the keys were never saved', async () => {
seedFile('thumbnails/E1/thumb.jpg');
seedFile('events/active/E1/photo.jpg');
const files = await backupService.getFilesToBackup({});
const rels = files.map((f) => f.relativePath);
expect(rels).toContain('thumbnails/E1/thumb.jpg');
expect(rels).toContain('events/active/E1/photo.jpg');
});
it("accepts the UI's plural backup_include_archives for the archived gate", async () => {
seedFile('events/archived/E4/archived.jpg');
const files = await backupService.getFilesToBackup({
backup_include_archives: true,
});
expect(files.map((f) => f.relativePath)).toContain('events/archived/E4/archived.jpg');
});
it('the UI plural key beats the migration-seeded singular key', async () => {
// Migration seeds backup_include_archived=true on every install; the
// form only ever writes the plural key, so unchecking Archives must
// win over the stale seeded value.
seedFile('events/archived/E5/archived.jpg');
const files = await backupService.getFilesToBackup({
backup_include_archived: true, // seeded default
backup_include_archives: false, // what the admin actually chose
});
expect(files.map((f) => f.relativePath)).not.toContain('events/archived/E5/archived.jpg');
});
it('rsync gets the de-selected paths and noise filters as --exclude args', async () => {
const excluded = await backupService.resolveExcludedBackupPaths({
backup_include_thumbnails: false,
backup_include_archives: false,
});
expect(excluded.map((r) => r.path)).toEqual(
expect.arrayContaining(['thumbnails', 'events/archived'])
);
const args = backupService.buildRsyncArgs(
{ backup_rsync_host: 'backup.example.com', backup_rsync_path: '/srv/backups' },
excluded.map((r) => `/${r.path}/`)
);
const excludes = args
.map((a, i) => (a === '--exclude' ? args[i + 1] : null))
.filter(Boolean);
expect(excludes).toEqual(expect.arrayContaining([
'.nfs*',
'/thumbnails/',
'/events/archived/',
]));
});
it('rows toggled off via include_in_default also become rsync excludes', async () => {
// The enabled-only loader hides these rows from the walker, but rsync
// syncs the whole storage root, so they must still appear as excludes.
await db('backup_paths').where('path', 'previews').update({
include_in_default: false,
});
const excluded = await backupService.resolveExcludedBackupPaths({});
expect(excluded.map((r) => r.path)).toContain('previews');
});
});
// Issue #871 — .nfs* silly-rename artifacts were uploaded to S3.
it('never backs up filesystem noise (.nfs*, .DS_Store)', async () => {
seedFile('thumbnails/E1/.nfs000000000000006600000008');
seedFile('events/active/E1/.DS_Store');
seedFile('events/active/E1/photo.jpg');
const files = await backupService.getFilesToBackup({});
const rels = files.map((f) => f.relativePath);
expect(rels).toContain('events/active/E1/photo.jpg');
expect(rels.some((r) => r.includes('.nfs'))).toBe(false);
expect(rels.some((r) => r.includes('.DS_Store'))).toBe(false);
});
it('the walker honors backup_exclude_patterns (previously rsync-only)', async () => {
seedFile('events/active/E1/photo.jpg');
seedFile('events/active/E1/scratch.tmp');
const files = await backupService.getFilesToBackup({
backup_exclude_patterns: ['*.tmp'],
});
const rels = files.map((f) => f.relativePath);
expect(rels).toContain('events/active/E1/photo.jpg');
expect(rels).not.toContain('events/active/E1/scratch.tmp');
});
it('glob patterns are literal outside the star (.nfs* must not eat anfs-…)', async () => {
seedFile('events/active/E1/anfs-photo.jpg');
seedFile('events/active/E1/notes-tmp');
const files = await backupService.getFilesToBackup({
backup_exclude_patterns: ['*.tmp'],
});
const rels = files.map((f) => f.relativePath);
// '.nfs*' used to compile to /^.nfs.*$/ whose dot matched any char;
// '*.tmp' used to compile to /^.*.tmp$/ which also matched 'notes-tmp'.
expect(rels).toContain('events/active/E1/anfs-photo.jpg');
expect(rels).toContain('events/active/E1/notes-tmp');
});
// Issue #871 — weekly schedules silently ran daily, and the dashboard's
// "next backup" was a hardcoded "tomorrow 02:00".
describe('schedule resolution + next run (issue #871)', () => {
it('a named label beats the stray default cron the UI used to send', () => {
expect(backupService.resolveScheduleCron({
backup_schedule: 'weekly',
backup_schedule_cron: '0 3 * * *', // old UI default, sent unconditionally
})).toBe('0 3 * * 0');
});
it('custom schedules use the cron field', () => {
expect(backupService.resolveScheduleCron({
backup_schedule: 'custom',
backup_schedule_cron: '15 5 * * 2',
})).toBe('15 5 * * 2');
});
it('falls back to the default daily cron', () => {
expect(backupService.resolveScheduleCron({})).toBe('0 2 * * *');
});
it('getNextScheduledRun is null when backups are disabled', () => {
expect(backupService.getNextScheduledRun(null)).toBeNull();
expect(backupService.getNextScheduledRun({ backup_enabled: false })).toBeNull();
});
it('getNextScheduledRun returns the real next weekly fire time', () => {
const iso = backupService.getNextScheduledRun({
backup_enabled: true,
backup_schedule: 'weekly',
backup_schedule_cron: '0 3 * * *',
});
const next = new Date(iso);
expect(Number.isNaN(next.getTime())).toBe(false);
expect(next.getTime()).toBeGreaterThan(Date.now());
expect(next.getDay()).toBe(0); // Sunday
expect(next.getHours()).toBe(3); // 03:00
});
});
// Issue #871 — "Backup Size: 167.6 TB": file_size_bytes is a bigInteger
// column, node-postgres returns int8 as a string, and the S3 path did
// `backedUpSize += size` — string concatenation.
it('getDatabaseBackupInfo coerces file_size_bytes to a number', async () => {
await db('database_backup_runs').del();
await db('database_backup_runs').insert({
backup_type: 'full',
status: 'completed',
file_path: '/backups/db/dump.sql.gz',
// Simulate the PG int8-as-string driver behaviour (sqlite stores
// whatever it is handed, so the string round-trips).
file_size_bytes: '421988',
started_at: new Date().toISOString(),
completed_at: new Date().toISOString(),
});
const info = await backupService.getDatabaseBackupInfo();
expect(typeof info.size).toBe('number');
expect(info.size).toBe(421988);
});
});
@@ -34,7 +34,7 @@ jest.mock('../../src/services/databaseBackup', () => ({
DatabaseBackupService: class {},
}));
jest.setTimeout(30000);
jest.setTimeout(120000);
describe('backupService — inline DB dump + fail-loud guard', () => {
let db;
@@ -23,7 +23,7 @@ const path = require('path');
const { bootCrmDb } = require('./helpers/crmDb');
jest.setTimeout(30000);
jest.setTimeout(120000);
describe('backupService — per-Stage-B-path statistics', () => {
let db;
@@ -14,7 +14,7 @@ const path = require('path');
const { bootCrmDb } = require('./helpers/crmDb');
jest.setTimeout(30000);
jest.setTimeout(120000);
describe('backupService — config + file collection + manifest (smoke)', () => {
let db;
@@ -7,7 +7,7 @@
const crypto = require('crypto');
const { bootCrmDb, seedMinimal } = require('./helpers/crmDb');
jest.setTimeout(30000);
jest.setTimeout(120000);
describe('booking cutover — draft invoices on hold', () => {
let db; let cleanup; let adminId; let customerId; let quoteService;
@@ -14,7 +14,7 @@ const { bootCrmDb, seedMinimal } = require('./helpers/crmDb');
// Service-level CRM calls cold-require heavy modules (pdfService,
// nodemailer, etc.) on first use; the global 5 s per-test budget is
// too tight for that. Bump it for this file only.
jest.setTimeout(30000);
jest.setTimeout(120000);
describe('discount line items (negative unit_price_minor)', () => {
let db;
@@ -6,7 +6,7 @@
const { bootCrmDb, seedMinimal } = require('./helpers/crmDb');
// bootCrmDb runs the full core-migration set in beforeAll.
jest.setTimeout(30000);
jest.setTimeout(120000);
describe('event type slug rename cascade', () => {
let db;
@@ -17,7 +17,7 @@ const request = require('supertest');
const { bootCrmDb } = require('./helpers/crmDb');
jest.setTimeout(60000);
jest.setTimeout(120000);
let db; let cleanup; let service; let app;
@@ -19,7 +19,7 @@
*/
const { bootCrmDb } = require('./helpers/crmDb');
jest.setTimeout(60000);
jest.setTimeout(120000);
let db; let cleanup; let service; let adminId;
@@ -13,7 +13,7 @@ const { bootCrmDb, seedMinimal } = require('./helpers/crmDb');
// Service-level CRM calls cold-require heavy modules (pdfService, nodemailer)
// on first use; bump the budget for this file.
jest.setTimeout(60000);
jest.setTimeout(120000);
describe('incoming-invoice categorise / re-bill chain', () => {
let db;
@@ -32,7 +32,7 @@ jest.mock('../../src/services/restoreService', () => ({
},
}));
jest.setTimeout(30000);
jest.setTimeout(120000);
describe('installFromBackupBoot', () => {
let db;
@@ -13,7 +13,7 @@ const { bootCrmDb, seedMinimal } = require('./helpers/crmDb');
// bootCrmDb runs the full core-migration set in beforeAll; under full-suite
// parallel load on a small CI runner that can exceed the 5s default. Match the
// other migration-heavy CRM suites (discountLineItems, incomingInvoiceRebill).
jest.setTimeout(30000);
jest.setTimeout(120000);
let db;
let cleanup;
@@ -21,7 +21,7 @@ beforeAll(async () => {
({ db, cleanup, tmpDir } = await bootCrmDb());
process.env.STORAGE_PATH = tmpDir; // isolate file collection to the temp dir
({ createPicpeak } = require('../../src/services/picpeakExportService'));
}, 60000);
}, 120000);
afterAll(async () => {
await cleanup();
@@ -28,7 +28,7 @@ beforeAll(async () => {
({ importFromPicpeak, validateManifest } = require('../../src/services/picpeakImportService'));
const role = await db('roles').where({ name: 'super_admin' }).first();
superAdminRoleId = role.id;
}, 60000);
}, 120000);
afterAll(async () => {
await cleanup();
@@ -13,14 +13,14 @@ const { execFileSync } = require('child_process');
const { bootCrmDb } = require('./helpers/crmDb');
jest.setTimeout(60000);
jest.setTimeout(120000);
let db;
let cleanup;
beforeAll(async () => {
({ db, cleanup } = await bootCrmDb());
}, 60000);
}, 120000);
afterAll(async () => {
if (cleanup) await cleanup();
@@ -183,22 +183,24 @@ describe('restoreService — PG branch scope contract (PR #596 review)', () => {
expect(window).toMatch(/was_successful:\s*true/);
});
it('npm run migrate:safe is invoked after the replay in restore()', () => {
it('the safe migration runner is invoked after the replay in restore()', () => {
// Contract from PR #596 round 4: backups taken on older picpeak
// versions must restore COMPLETELY on a newer image — even if new
// migrations have been added since the backup was taken. The
// restore() flow shells out to `npm run migrate:safe` AFTER the
// restore() flow shells out to the safe migration runner AFTER the
// operator-meta replay so the schema catches up to the running
// code WITHIN the restore boundary (not on the next container
// restart).
// restart). Invoked as `node migrations/run-migrations-safe.js` —
// the runtime image ships no npm, so the former `npm run
// migrate:safe` would ENOENT into the non-fatal catch.
//
// Contract:
// 1. A `migrate:safe` shell-out exists somewhere in restoreService
// 1. A run-migrations-safe shell-out exists somewhere in restoreService
// 2. It sits AFTER the replay drain — verification → replay →
// migrations is the documented order
// 3. It does NOT sit inside performDatabaseRestore (must run
// against the reinit'd pool from the parent restore())
const migrateLine = findFirst(/['"]migrate:safe['"]/);
const migrateLine = findFirst(/run-migrations-safe\.js/);
expect(migrateLine).toBeGreaterThan(0);
const replayLine = findLast(/this\.preservedMetaSnapshot\.length\s*>\s*0/);
@@ -27,7 +27,7 @@ beforeAll(async () => {
setupService = require('../../src/services/setupService');
({ getAppSetting, upsertAppSetting } = require('../../src/utils/appSettings'));
app = buildRouteApp('/api/setup', require('../../src/routes/setup'));
}, 60000);
}, 120000);
afterAll(async () => {
await cleanup();
@@ -10,7 +10,7 @@ const { bootCrmDb } = require('./helpers/crmDb');
// bootCrmDb runs the full core-migration set in beforeAll; under full-suite
// parallel load on a small CI runner that can exceed the 5s default. Match the
// other migration-heavy CRM suites (discountLineItems, incomingInvoiceRebill).
jest.setTimeout(30000);
jest.setTimeout(120000);
let db;
let cleanup;
@@ -9,7 +9,7 @@ const {
// bootCrmDb runs the full core-migration set in beforeAll; under full-suite
// parallel load on a small CI runner that can exceed the 5s default. Match the
// other migration-heavy CRM suites (discountLineItems, incomingInvoiceRebill).
jest.setTimeout(30000);
jest.setTimeout(120000);
let db;
let cleanup;
@@ -83,7 +83,7 @@ describe('admin CRM routes — auth + permission gate', () => {
// Invalid: signed with a different secret. adminAuth must reject.
const jwt = require('jsonwebtoken');
invalidToken = jwt.sign({ id: adminId, type: 'admin' }, 'WRONG-SECRET', { issuer: 'picpeak-auth' });
}, 60000);
}, 120000);
afterAll(async () => {
if (cleanup) await cleanup();
+2 -2
View File
@@ -39,7 +39,7 @@ const {
bootCrmDb, mintAdminToken, buildRouteApp,
} = require('../integration/helpers/crmDb');
jest.setTimeout(60000);
jest.setTimeout(120000);
let db;
let cleanup;
@@ -95,7 +95,7 @@ beforeAll(async () => {
({ db, cleanup } = await bootCrmDb());
adminApp = buildRouteApp('/api/admin/auth', require('../../src/routes/adminAuth'));
authApp = buildRouteApp('/api/auth', require('../../src/routes/auth'));
}, 60000);
}, 120000);
afterAll(async () => {
if (cleanup) await cleanup();
@@ -51,7 +51,7 @@ describe('publicContracts routes', () => {
contractId = inserted[0]?.id ?? inserted[0];
app = buildRouteApp('/api/public/contracts', require('../../src/routes/publicContracts'));
}, 60000);
}, 120000);
afterAll(async () => {
if (cleanup) await cleanup();
@@ -33,7 +33,7 @@ describe('publicPaymentCheck routes', () => {
({ db, cleanup } = await bootCrmDb());
await seedMinimal(db);
app = buildRouteApp('/api/public/payment-check', require('../../src/routes/publicPaymentCheck'));
}, 60000);
}, 120000);
afterAll(async () => {
if (cleanup) await cleanup();
@@ -60,7 +60,7 @@ describe('publicQuotes routes', () => {
quoteId = inserted[0]?.id ?? inserted[0];
app = buildRouteApp('/api/public/quotes', require('../../src/routes/publicQuotes'));
}, 60000);
}, 120000);
afterAll(async () => {
if (cleanup) await cleanup();
@@ -75,7 +75,7 @@ describe('admin Live Slideshow endpoints', () => {
app.use((err, req, res, next) => {
res.status(err.statusCode || err.status || 500).json({ error: err.message, code: err.code });
});
}, 30000);
}, 120000);
afterAll(async () => { await cleanup(); });
@@ -67,11 +67,10 @@ async function insertEvent(db, over = {}) {
describe('public Live Slideshow routes', () => {
let db; let cleanup; let app;
// bootCrmDb runs the full migration set against a fresh SQLite file, which
// takes <2s locally but has been observed to exceed Jest's default 5s
// `beforeAll` timeout on slower GitHub Actions runners (~5.4s — runner-to-
// runner I/O variance). Raise the hook timeout so this doesn't intermittently
// block PRs on CI; doesn't affect happy-path local runs.
// bootCrmDb runs the full migration set against a fresh SQLite file and the
// chain keeps growing via backports. Hook-argument timeouts OVERRIDE the
// 120s jest.config default (same trap as the jest.setTimeout pins) — keep
// this at 120000, matching the config.
beforeAll(async () => {
({ db, cleanup } = await bootCrmDb());
await seedMinimal(db);
@@ -86,7 +85,7 @@ describe('public Live Slideshow routes', () => {
app.use((err, req, res, next) => {
res.status(err.statusCode || err.status || 500).json({ error: err.message, code: err.code });
});
}, 30000);
}, 120000);
afterAll(async () => { await cleanup(); });
@@ -22,7 +22,7 @@ const crypto = require('crypto');
const { bootCrmDb, seedMinimal } = require('../integration/helpers/crmDb');
jest.setTimeout(30000);
jest.setTimeout(120000);
describe('backupIntegrityService.verifyDocumentArtefacts', () => {
let db;
@@ -27,7 +27,7 @@ let db; let cleanup;
beforeAll(async () => {
({ db, cleanup } = await bootCrmDb());
}, 30000);
}, 120000);
afterAll(async () => { if (cleanup) await cleanup(); });
@@ -48,7 +48,7 @@ describe('userManagementService — activate + delete (#574 follow-up)', () => {
is_active: 1, created_at: new Date(),
}).returning('id');
targetId = targetInsert[0]?.id ?? targetInsert[0];
}, 60000);
}, 120000);
afterAll(async () => {
if (cleanup) await cleanup();
@@ -115,7 +115,7 @@ beforeAll(async () => {
}).returning('id');
photoIds.push(r[0]?.id ?? r[0]);
}
}, 30000);
}, 120000);
afterAll(async () => { if (cleanup) await cleanup(); });
+4
View File
@@ -1,5 +1,9 @@
module.exports = {
testEnvironment: 'node',
// bootCrmDb() runs EVERY core migration in beforeAll and the chain keeps
// growing (134 migrations and counting via backports). 120s matches the
// beta-branch convention from #860.
testTimeout: 120000,
coverageDirectory: 'coverage',
collectCoverageFrom: [
'src/**/*.js',
+36 -14
View File
@@ -1,12 +1,12 @@
{
"name": "picpeak-backend",
"version": "3.45.4",
"version": "3.45.6",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "picpeak-backend",
"version": "3.45.4",
"version": "3.45.6",
"dependencies": {
"@aws-sdk/client-s3": "^3.850.0",
"@aws-sdk/lib-storage": "^3.850.0",
@@ -17,6 +17,7 @@
"chokidar": "4.0.3",
"cookie-parser": "^1.4.7",
"cors": "^2.8.5",
"cron-parser": "^4.9.0",
"dotenv": "^16.0.3",
"exifr": "^7.1.3",
"express": "^4.18.2",
@@ -44,7 +45,7 @@
"pdf-lib": "^1.17.1",
"pdfkit": "^0.17.2",
"pg": "^8.16.3",
"postcss": "8.5.10",
"postcss": "8.5.18",
"qrcode": "^1.5.4",
"react-i18next": "^15.6.0",
"sanitize-html": "^2.17.0",
@@ -53,7 +54,7 @@
"swagger-jsdoc": "^6.2.8",
"swagger-ui-express": "^5.0.1",
"swissqrbill": "^4.3.0",
"tar": ">=7.5.16",
"tar": ">=7.5.21",
"uuid": "^11.1.1",
"winston": "^3.8.2",
"zxcvbn": "^4.4.2"
@@ -5184,6 +5185,18 @@
"node": "^14.15.0 || ^16.10.0 || >=18.0.0"
}
},
"node_modules/cron-parser": {
"version": "4.9.0",
"resolved": "https://registry.npmjs.org/cron-parser/-/cron-parser-4.9.0.tgz",
"integrity": "sha512-p0SaNjrHOnQeR8/VnfGbmg9te2kfyYSQ7Sc/j/6DtPL3JQvKxmjO9TSjNFpujqV3vEYYBvNNvXSxzyksBWAx1Q==",
"license": "MIT",
"dependencies": {
"luxon": "^3.2.1"
},
"engines": {
"node": ">=12.0.0"
}
},
"node_modules/cross-fetch": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/cross-fetch/-/cross-fetch-4.1.0.tgz",
@@ -8505,6 +8518,15 @@
"yallist": "^3.0.2"
}
},
"node_modules/luxon": {
"version": "3.7.2",
"resolved": "https://registry.npmjs.org/luxon/-/luxon-3.7.2.tgz",
"integrity": "sha512-vtEhXh/gNjI9Yg1u4jX/0YVPMvxzHuGgCm6tC5kZyb08yjGWGnqAjGJvcXbqQR2P3MyMEFnRbpcdFS6PBcLqew==",
"license": "MIT",
"engines": {
"node": ">=12"
}
},
"node_modules/mailparser": {
"version": "3.9.14",
"resolved": "https://registry.npmjs.org/mailparser/-/mailparser-3.9.14.tgz",
@@ -9057,9 +9079,9 @@
}
},
"node_modules/nanoid": {
"version": "3.3.11",
"resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.11.tgz",
"integrity": "sha512-N8SpfPUnUp1bK+PMYW8qSWdl9U+wwNWI4QKxOYDy9JAro3WMX7p2OeVRF9v+347pnakNevPmiHhNmZ2HbFA76w==",
"version": "3.3.16",
"resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.16.tgz",
"integrity": "sha512-bzlKTyNJ7+LdGIIwy8ijFpIqEQIvafahV7eYykJ8Cvh42EdJeODoJ6gUJXpQJvej1BddH8OqTXZNE/KfbWAu8Q==",
"funding": [
{
"type": "github",
@@ -10014,9 +10036,9 @@
}
},
"node_modules/postcss": {
"version": "8.5.10",
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.10.tgz",
"integrity": "sha512-pMMHxBOZKFU6HgAZ4eyGnwXF/EvPGGqUr0MnZ5+99485wwW41kW91A4LOGxSHhgugZmSChL5AlElNdwlNgcnLQ==",
"version": "8.5.18",
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.18.tgz",
"integrity": "sha512-xdB1oSLHbz1vRWgCDalrCqEFTWzFlhqFC5tIHLMOSUIjhm3XXQ1qrFy8S/ESr1JYRRXqM3c1QFiMZUJdUTqyMQ==",
"funding": [
{
"type": "opencollective",
@@ -10033,7 +10055,7 @@
],
"license": "MIT",
"dependencies": {
"nanoid": "^3.3.11",
"nanoid": "^3.3.12",
"picocolors": "^1.1.1",
"source-map-js": "^1.2.1"
},
@@ -11732,9 +11754,9 @@
}
},
"node_modules/tar": {
"version": "7.5.19",
"resolved": "https://registry.npmjs.org/tar/-/tar-7.5.19.tgz",
"integrity": "sha512-4LeEWl96twnS2Q7Bz4MGqgazLqO+hJN63GZxXoIqh1T3VweYD997gbU1ItNsQafqqXTXd5WFyFdReLtwvRBNiw==",
"version": "7.5.22",
"resolved": "https://registry.npmjs.org/tar/-/tar-7.5.22.tgz",
"integrity": "sha512-MFO/QzvtAOmJbkhOaCTvbGcFN9L9b+JunIsDwaKljSOdcLMea3NJ1k9Usz/rjdfSXTq4dfzfeS7W4p4YOAAHeA==",
"license": "BlueOak-1.0.0",
"dependencies": {
"@isaacs/fs-minipass": "^4.0.0",
+5 -4
View File
@@ -1,6 +1,6 @@
{
"name": "picpeak-backend",
"version": "3.45.5",
"version": "3.45.8",
"description": "Backend for PicPeak event photo sharing platform",
"main": "server.js",
"engines": {
@@ -26,6 +26,7 @@
"chokidar": "4.0.3",
"cookie-parser": "^1.4.7",
"cors": "^2.8.5",
"cron-parser": "^4.9.0",
"dotenv": "^16.0.3",
"exifr": "^7.1.3",
"express": "^4.18.2",
@@ -53,7 +54,7 @@
"pdf-lib": "^1.17.1",
"pdfkit": "^0.17.2",
"pg": "^8.16.3",
"postcss": "8.5.10",
"postcss": "8.5.18",
"qrcode": "^1.5.4",
"react-i18next": "^15.6.0",
"sanitize-html": "^2.17.0",
@@ -62,7 +63,7 @@
"swagger-jsdoc": "^6.2.8",
"swagger-ui-express": "^5.0.1",
"swissqrbill": "^4.3.0",
"tar": ">=7.5.16",
"tar": ">=7.5.21",
"uuid": "^11.1.1",
"winston": "^3.8.2",
"zxcvbn": "^4.4.2"
@@ -82,7 +83,7 @@
"js-yaml": "^4.2.0",
"fast-xml-parser": ">=5.7.0",
"qs": ">=6.15.2",
"tar": ">=7.5.16",
"tar": ">=7.5.21",
"brace-expansion": ">=5.0.7",
"minimatch": ">=9.0.7",
"path-to-regexp": "0.1.13",
+10 -6
View File
@@ -212,17 +212,20 @@ async function buildConfiguredPathReport(configuredRows, config) {
const includedInDefault = Boolean(row.include_in_default);
let featureFlagValue = null;
if (row.feature_flag) {
const v = config[row.feature_flag];
featureFlagValue = v === undefined ? null : Boolean(v);
// Alias-aware: show the value the gate actually used, not a seeded
// canonical key shadowed by the UI's spelling. Normalize like the
// walker does — Boolean('false') is true.
const v = backupService.effectiveFlagValue(row, config);
featureFlagValue = v === undefined || v === null ? null : backupService.normalizeBoolean(v);
}
let coverage;
if (!includedInDefault) {
coverage = 'skipped-by-toggle';
} else if (row.feature_flag && featureFlagValue !== true) {
// null (unset) and explicit false both gate the path off — matches
// the walker's normalizeBoolean semantics
coverage = 'skipped-by-feature-flag';
} else if (!backupService.backupPathIncluded(row, config)) {
// Same gate the walker uses — feature flags (incl. the UI's
// backup_include_archives alias) and the What-to-Backup opt-outs.
coverage = row.feature_flag ? 'skipped-by-feature-flag' : 'skipped-by-setting';
} else if (!stat.exists) {
coverage = 'missing-on-disk';
} else {
@@ -313,6 +316,7 @@ async function getCoverageReport() {
willScanCount: paths.filter((p) => p.coverage === 'will-scan').length,
skippedByToggleCount: paths.filter((p) => p.coverage === 'skipped-by-toggle').length,
skippedByFeatureFlagCount: paths.filter((p) => p.coverage === 'skipped-by-feature-flag').length,
skippedBySettingCount: paths.filter((p) => p.coverage === 'skipped-by-setting').length,
missingOnDiskCount: paths.filter((p) => p.coverage === 'missing-on-disk').length,
driftCount: unconfiguredOnDisk.length,
tableMissingFallbackInUse: fallback,
+192 -68
View File
@@ -7,6 +7,7 @@ const os = require('os');
const { promisify } = require('util');
const cron = require('node-cron');
const cronParser = require('cron-parser');
const { db } = require('../database/db');
const { queueEmail } = require('./emailProcessor');
const logger = require('../utils/logger');
@@ -348,7 +349,10 @@ async function getDatabaseBackupInfoInternal() {
return {
type: recent.backup_type || 'unknown',
backupFile: recent.file_path,
size: recent.file_size_bytes,
// file_size_bytes is a bigInteger column — node-postgres returns int8
// as a STRING, and `backedUpSize += size` then concatenates instead of
// adding (issue #871: "167.6 TB" dashboard size). Coerce at the source.
size: Number(recent.file_size_bytes) || 0,
checksum: recent.checksum,
hasChanged,
backupTime: recent.completed_at,
@@ -389,7 +393,11 @@ async function scanDirectory(dirPath, fileList, basePath, excludePatterns = [])
const isExcluded = excludePatterns.some(pattern => {
if (pattern.includes('*')) {
const regex = new RegExp(`^${pattern.replace(/\*/g, '.*')}$`);
// Escape regex metacharacters before expanding the glob star — the
// raw replace turned '.nfs*' into /^.nfs.*$/ whose leading dot
// matched any character (e.g. 'anfs-photo.jpg' was excluded too).
const escaped = pattern.replace(/[.+?^${}()|[\]\\]/g, '\\$&').replace(/\*/g, '.*');
const regex = new RegExp(`^${escaped}$`);
return regex.test(entry.name);
}
return entry.name === pattern;
@@ -437,6 +445,26 @@ const LEGACY_BACKUP_PATHS = [
{ path: 'business-docs', feature_flag: null },
];
// "What to Backup" opt-OUT toggles written by BackupConfiguration.tsx.
// Default-ON semantics: only an explicit false excludes the path, so
// installs that never saved the backup form keep backing up everything
// (issue #871: unchecking Thumbnails had no effect because these keys
// were stored but never read).
const OPT_OUT_FLAGS = {
'events/active': 'backup_include_photos',
'thumbnails': 'backup_include_thumbnails',
};
// The UI "Archives" checkbox writes backup_include_archives (plural) while
// the feature_flag rows use backup_include_archived — accept both.
const FLAG_ALIASES = {
backup_include_archived: 'backup_include_archives',
};
// Filesystem noise that must never land in a backup: NFS silly-rename
// artifacts (issue #871 showed .nfs* files uploaded to S3) and OS metadata.
const DEFAULT_EXCLUDE_PATTERNS = ['.nfs*', '.DS_Store', 'Thumbs.db'];
/**
* Resolve the walker's target subdirectories from `backup_paths`.
*
@@ -456,34 +484,84 @@ const LEGACY_BACKUP_PATHS = [
* Used to evaluate feature_flag gates.
* @returns {Promise<Array<{ path: string, feature_flag: string|null }>>}
*/
async function resolveBackupPaths(config) {
let rows;
async function loadBackupPathRows({ includeDisabled = false } = {}) {
try {
if (!(await db.schema.hasTable('backup_paths'))) {
logger.warn('backup_paths table missing — falling back to LEGACY_BACKUP_PATHS');
rows = LEGACY_BACKUP_PATHS;
} else {
rows = await db('backup_paths')
.where('include_in_default', formatBoolean(true))
.orderBy('display_order', 'asc')
.select('path', 'feature_flag');
if (!rows.length) {
logger.warn('backup_paths has no rows with include_in_default=true — falling back to LEGACY_BACKUP_PATHS');
rows = LEGACY_BACKUP_PATHS;
}
return LEGACY_BACKUP_PATHS;
}
let query = db('backup_paths')
.orderBy('display_order', 'asc')
.select('path', 'feature_flag', 'include_in_default');
if (!includeDisabled) {
query = query.where('include_in_default', formatBoolean(true));
}
const rows = await query;
if (!rows.filter((r) => normalizeBoolean(r.include_in_default)).length) {
logger.warn('backup_paths has no rows with include_in_default=true — falling back to LEGACY_BACKUP_PATHS');
return LEGACY_BACKUP_PATHS;
}
return rows;
} catch (err) {
logger.warn(`Failed to query backup_paths (${err.message}) — falling back to LEGACY_BACKUP_PATHS`);
rows = LEGACY_BACKUP_PATHS;
return LEGACY_BACKUP_PATHS;
}
}
// Apply feature_flag gating. A row with feature_flag='backup_include_archived'
// requires config.backup_include_archived to be truthy (same semantics as
// the historical `includeArchived` parameter).
// Per-row gate. Applies the UI opt-out toggles first, then feature_flag
// gating: a row with feature_flag='backup_include_archived' requires the
// corresponding config key to be truthy (same semantics as the historical
// `includeArchived` parameter).
function backupPathIncluded(row, config) {
const optOutKey = OPT_OUT_FLAGS[row.path];
if (optOutKey && config) {
const optOutValue = config[optOutKey];
if (optOutValue !== undefined && optOutValue !== null && normalizeBoolean(optOutValue) === false) {
return false;
}
}
if (!row.feature_flag) return true;
let flagValue;
if (config) {
// The alias (backup_include_archives) is what the current UI writes;
// the canonical singular key is seeded true by migration on every
// install, so the UI value must take precedence or the checkbox can
// never turn the flag off.
const alias = FLAG_ALIASES[row.feature_flag];
if (alias && config[alias] !== undefined && config[alias] !== null) {
flagValue = config[alias];
} else {
flagValue = config[row.feature_flag];
}
}
return normalizeBoolean(flagValue);
}
// The raw config value the gate actually consulted for a row's feature
// flag (alias-aware) — the coverage report shows it next to the status,
// so it must not display the shadowed seeded key.
function effectiveFlagValue(row, config) {
if (!row.feature_flag || !config) return undefined;
const alias = FLAG_ALIASES[row.feature_flag];
if (alias && config[alias] !== undefined && config[alias] !== null) {
return config[alias];
}
return config[row.feature_flag];
}
async function resolveBackupPaths(config) {
return (await loadBackupPathRows()).filter((row) => backupPathIncluded(row, config));
}
// The rows the admin de-selected — the rsync destination needs them as
// --exclude filters because it syncs the whole storage root rather than
// the walker's file list. Includes rows with include_in_default=false,
// which the enabled-only loader would otherwise hide from rsync entirely.
async function resolveExcludedBackupPaths(config) {
const rows = await loadBackupPathRows({ includeDisabled: true });
return rows.filter((row) => {
if (!row.feature_flag) return true;
const flagValue = config ? config[row.feature_flag] : undefined;
return normalizeBoolean(flagValue);
const disabled = row.include_in_default !== undefined && !normalizeBoolean(row.include_in_default);
return disabled || !backupPathIncluded(row, config);
});
}
@@ -574,6 +652,14 @@ async function getFilesToBackupInternal(configOrIncludeArchived = true) {
const targets = await resolveBackupPaths(config);
// backup_exclude_patterns was only honored by the rsync destination
// (as --exclude args); the local/S3 walker ignored it. Merge it with
// the always-on noise filters here so every destination agrees.
const configuredExcludes = Array.isArray(config.backup_exclude_patterns)
? config.backup_exclude_patterns
: [];
const excludePatterns = [...new Set([...DEFAULT_EXCLUDE_PATTERNS, ...configuredExcludes])];
for (const target of targets) {
// CRM document estate is special-cased in the comment block below
// because it's the most expensive omission to recover from:
@@ -589,7 +675,7 @@ async function getFilesToBackupInternal(configOrIncludeArchived = true) {
// those values refer to do not, leaving every CRM *_path column a
// broken FK. scanDirectory short-circuits on ENOENT so installs
// that never used CRM features won't error.
await scanDirectory(path.join(storagePath, target.path), files, storagePath);
await scanDirectory(path.join(storagePath, target.path), files, storagePath, excludePatterns);
}
return files;
@@ -686,7 +772,7 @@ function validateRsyncParam(value, label) {
return value;
}
function buildRsyncArgs(config) {
function buildRsyncArgs(config, extraExcludes = []) {
const storagePath = getStoragePath();
const host = validateRsyncParam(config.backup_rsync_host, 'host');
const remotePath = validateRsyncParam(config.backup_rsync_path, 'remote path');
@@ -713,7 +799,14 @@ function buildRsyncArgs(config) {
args.push('-e', `ssh -i ${sshKey} -o StrictHostKeyChecking=no`);
}
const excludePatterns = config.backup_exclude_patterns || [];
// Same noise filters as the walker, plus the de-selected backup paths
// (extraExcludes) — rsync syncs the whole storage root, so this is the
// only place the What-to-Backup selection can take effect for rsync.
const excludePatterns = [...new Set([
...DEFAULT_EXCLUDE_PATTERNS,
...(Array.isArray(config.backup_exclude_patterns) ? config.backup_exclude_patterns : []),
...extraExcludes,
])];
excludePatterns.forEach(pattern => args.push('--exclude', pattern));
const source = `${storagePath}/`;
@@ -752,7 +845,11 @@ function parseRsyncStats(output) {
async function performRsyncBackup(config, files) {
const { spawnAsync } = require('../utils/safeExec');
const rsyncArgs = buildRsyncArgs(config);
// Anchored excludes for the de-selected What-to-Backup paths; rsync
// otherwise transfers the whole storage root regardless of the walker's
// file list (which only feeds manifests and file state).
const excludedPaths = await resolveExcludedBackupPaths(config);
const rsyncArgs = buildRsyncArgs(config, excludedPaths.map((row) => `/${row.path}/`));
const { stdout } = await spawnAsync('rsync', rsyncArgs);
const stats = parseRsyncStats(stdout);
@@ -1175,6 +1272,50 @@ async function runBackupInternal(isManual = false) {
}
}
// Two settings cooperate here:
// - backup_schedule — UI label like "daily" / "weekly" / "custom"
// - backup_schedule_cron — actual cron expression (custom schedules)
// Older startup code read backup_schedule and crashed when it found a label
// instead of a cron expression. Resolution order: explicit cron field, then
// map known labels, then fall back to default.
const NAMED_SCHEDULES = {
hourly: '0 * * * *',
daily: '0 2 * * *',
weekly: '0 3 * * 0', // Sunday 03:00
monthly: '0 4 1 * *',
};
function resolveScheduleCron(config) {
const isCronExpression = (s) => typeof s === 'string' && /^\s*\S+(\s+\S+){4}\s*$/.test(s);
const readSetting = (key) => {
if (config && Object.prototype.hasOwnProperty.call(config, key)) {
return String(config[key] ?? '').trim();
}
if (config?.__raw && Object.prototype.hasOwnProperty.call(config.__raw, key)) {
return String(parseSettingValue(config.__raw[key]) ?? '').trim();
}
return '';
};
let schedule = '0 2 * * *';
const cronCandidate = readSetting('backup_schedule_cron');
const labelCandidate = readSetting('backup_schedule');
// A named label wins over the cron field: the UI always used to send its
// default cron ('0 3 * * *') alongside e.g. backup_schedule='weekly', which
// silently turned weekly schedules into daily ones (issue #871). The cron
// field only applies for 'custom' (or when no known label is set).
if (labelCandidate && labelCandidate.toLowerCase() !== 'custom' && NAMED_SCHEDULES[labelCandidate.toLowerCase()]) {
schedule = NAMED_SCHEDULES[labelCandidate.toLowerCase()];
} else if (cronCandidate && isCronExpression(cronCandidate)) {
schedule = cronCandidate;
} else if (labelCandidate && isCronExpression(labelCandidate)) {
// Back-compat: a deployment that wrote a cron expression directly into
// backup_schedule (no _cron field) still works.
schedule = labelCandidate;
}
return schedule;
}
async function startBackupService() {
try {
const config = await resolveConfigWithFallback();
@@ -1192,42 +1333,7 @@ async function startBackupService() {
backupJob = null;
}
// Two settings cooperate here:
// - backup_schedule — UI label like "daily" / "weekly" / "custom"
// - backup_schedule_cron — actual cron expression
// The frontend writes both (BackupConfiguration.jsx). Older startup code
// here read backup_schedule and crashed when it found a label instead of
// a cron expression. Resolution order: explicit cron field, then map known
// labels, then fall back to default.
const NAMED_SCHEDULES = {
hourly: '0 * * * *',
daily: '0 2 * * *',
weekly: '0 3 * * 0', // Sunday 03:00
monthly: '0 4 1 * *',
};
const isCronExpression = (s) => typeof s === 'string' && /^\s*\S+(\s+\S+){4}\s*$/.test(s);
const readSetting = (key) => {
if (config && Object.prototype.hasOwnProperty.call(config, key)) {
return String(config[key] ?? '').trim();
}
if (config?.__raw && Object.prototype.hasOwnProperty.call(config.__raw, key)) {
return String(parseSettingValue(config.__raw[key]) ?? '').trim();
}
return '';
};
let schedule = '0 2 * * *';
const cronCandidate = readSetting('backup_schedule_cron');
const labelCandidate = readSetting('backup_schedule');
if (cronCandidate && isCronExpression(cronCandidate)) {
schedule = cronCandidate;
} else if (labelCandidate && NAMED_SCHEDULES[labelCandidate.toLowerCase()]) {
schedule = NAMED_SCHEDULES[labelCandidate.toLowerCase()];
} else if (labelCandidate && isCronExpression(labelCandidate)) {
// Back-compat: a deployment that wrote a cron expression directly into
// backup_schedule (no _cron field) still works.
schedule = labelCandidate;
}
const schedule = resolveScheduleCron(config);
backupJob = cron.schedule(schedule, async () => {
logger.info('Starting scheduled backup');
@@ -1319,6 +1425,8 @@ async function getBackupStatus(limit = 10) {
// ago looked identical to a successful one. Same "silent failure
// not surfaced" class Stage A was designed to fight.
const lastSuccessful = runs.find(r => r.status === 'completed') || null;
const config = await getBackupConfigInternal();
const nextRun = getNextScheduledRun(config);
// Detect zombie running rows (started >30min ago, never updated)
// — these are processes that died without writing a completed_at.
// Surface them so the admin can tell at a glance vs a live run.
@@ -1339,7 +1447,8 @@ async function getBackupStatus(limit = 10) {
recentRuns: runs,
recentBackups: runs, // Alias for frontend compatibility
totalBackups: runs.filter(r => r.status === 'completed').length,
nextScheduledRun: getNextScheduledRun()
nextScheduledRun: nextRun,
nextBackup: nextRun // BackupManagement.tsx reads this name
};
} catch (error) {
logger.error('Failed to get backup status:', error);
@@ -1351,12 +1460,19 @@ async function getBackupStatus(limit = 10) {
}
}
function getNextScheduledRun() {
const now = new Date();
const next = new Date(now);
next.setDate(now.getDate() + 1);
next.setHours(2, 0, 0, 0);
return next.toISOString();
function getNextScheduledRun(config) {
// null → the UI shows "Not scheduled". Only a real, enabled schedule
// produces a date (issue #871: this used to be a hardcoded "tomorrow
// 02:00" that ignored the configured schedule entirely).
if (!config || !normalizeBoolean(config.backup_enabled)) {
return null;
}
try {
return cronParser.parseExpression(resolveScheduleCron(config)).next().toISOString();
} catch (error) {
logger.warn(`Could not compute next backup run: ${error.message}`);
return null;
}
}
async function cleanupOldBackupRuns(retentionDays = 30) {
@@ -1569,5 +1685,13 @@ service.getBackupStatus = getBackupStatus;
service.cleanupOldBackupRuns = cleanupOldBackupRuns;
service.getBackupManifest = getBackupManifest;
service.validateBackupManifest = validateBackupManifest;
service.resolveBackupPaths = resolveBackupPaths;
service.resolveExcludedBackupPaths = resolveExcludedBackupPaths;
service.backupPathIncluded = backupPathIncluded;
service.effectiveFlagValue = effectiveFlagValue;
service.normalizeBoolean = normalizeBoolean;
service.buildRsyncArgs = buildRsyncArgs;
service.resolveScheduleCron = resolveScheduleCron;
service.getNextScheduledRun = getNextScheduledRun;
module.exports = service;
+4 -1
View File
@@ -298,7 +298,10 @@ class RestoreService {
this.log('info', 'Applying post-restore migrations to restored database...');
this.updateProgress('Applying any post-backup migrations...');
const backendRoot = path.join(__dirname, '..', '..');
const { stderr } = await spawnAsync('npm', ['run', 'migrate:safe'], {
// Invoked via node directly — the runtime image ships no npm
// (see Dockerfile), and an ENOENT here would be swallowed by the
// non-fatal catch below, silently skipping post-restore migrations.
const { stderr } = await spawnAsync('node', ['migrations/run-migrations-safe.js'], {
cwd: backendRoot,
env: { ...process.env },
});
+5 -3
View File
@@ -120,12 +120,14 @@ echo "Ensuring storage directories exist..."
STORAGE_BASE="${STORAGE_PATH:-/app/storage}"
mkdir -p "$STORAGE_BASE/events/active" "$STORAGE_BASE/events/archived" "$STORAGE_BASE/thumbnails" 2>/dev/null || true
# Run migrations (use safe runner in production)
# Run migrations (use safe runner in production). Invoked via node directly —
# the runtime image no longer ships npm (see Dockerfile: its bundled deps kept
# tripping CVE scanners while npm itself never runs in production).
echo "Running database migrations..."
if [ "$NODE_ENV" = "production" ]; then
npm run migrate:safe
node migrations/run-migrations-safe.js
else
npm run migrate
node migrations/run-migrations.js
fi
# Execute the main command
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "picpeak-frontend",
"private": true,
"version": "3.45.5",
"version": "3.45.8",
"type": "module",
"scripts": {
"dev": "vite",
@@ -5,7 +5,6 @@ import {
Server,
Cloud,
HardDrive,
AlertCircle,
Eye,
EyeOff,
Wifi,
@@ -102,7 +101,9 @@ export const BackupConfiguration: React.FC<BackupConfigurationProps> = ({ config
backup_include_database: true,
backup_include_photos: true,
backup_include_archives: true,
backup_include_thumbnails: false,
// Matches the backend never-saved fallback (include everything) so the
// form does not show "off" while thumbnails are in fact being backed up.
backup_include_thumbnails: true,
backup_include_temp: false,
backup_compression: true,
backup_encryption: false,
@@ -111,8 +112,7 @@ export const BackupConfiguration: React.FC<BackupConfigurationProps> = ({ config
const [showSecrets, setShowSecrets] = useState({
s3_secret_key: false,
ssh_key: false,
encryption_passphrase: false
ssh_key: false
});
const [testingConnection, setTestingConnection] = useState(false);
@@ -152,6 +152,15 @@ export const BackupConfiguration: React.FC<BackupConfigurationProps> = ({ config
return;
}
// A custom schedule needs a real 5-field cron — the backend silently
// falls back to daily 02:00 otherwise. For named schedules the stored
// cron is kept (the backend prefers the label), so switching back to
// Custom keeps the previously saved expression.
if (formData.backup_schedule === 'custom' && !/^\s*\S+(\s+\S+){4}\s*$/.test(formData.backup_schedule_cron)) {
toast.error(t('backup.configuration.messages.invalidCron', 'Please enter a valid cron expression (5 fields)'));
return;
}
onSave(formData);
};
@@ -546,69 +555,6 @@ export const BackupConfiguration: React.FC<BackupConfigurationProps> = ({ config
</div>
</Card>
{/* Advanced Options */}
<Card className="p-6">
<h3 className="text-lg font-semibold text-neutral-900 dark:text-neutral-100 mb-4">{t('backup.configuration.advancedOptions.title')}</h3>
<div className="space-y-4">
<label className="flex items-center">
<input
type="checkbox"
checked={formData.backup_compression}
onChange={(e) => handleChange('backup_compression', e.target.checked)}
className="h-4 w-4 text-primary focus:ring-primary border-neutral-300 dark:border-neutral-600 rounded bg-white dark:bg-neutral-700"
/>
<div className="ml-3">
<span className="text-sm font-medium text-neutral-700 dark:text-neutral-300">{t('backup.configuration.advancedOptions.compression')}</span>
<p className="text-xs text-neutral-500 dark:text-neutral-400">{t('backup.configuration.advancedOptions.compressionHelp')}</p>
</div>
</label>
<div>
<label className="flex items-center mb-3">
<input
type="checkbox"
checked={formData.backup_encryption}
onChange={(e) => handleChange('backup_encryption', e.target.checked)}
className="h-4 w-4 text-primary focus:ring-primary border-neutral-300 dark:border-neutral-600 rounded bg-white dark:bg-neutral-700"
/>
<div className="ml-3">
<span className="text-sm font-medium text-neutral-700 dark:text-neutral-300">{t('backup.configuration.advancedOptions.encryption')}</span>
<p className="text-xs text-neutral-500 dark:text-neutral-400">{t('backup.configuration.advancedOptions.encryptionHelp')}</p>
</div>
</label>
{formData.backup_encryption && (
<div className="ml-7">
<label className="block text-sm font-medium text-neutral-700 dark:text-neutral-300 mb-1">
{t('backup.configuration.advancedOptions.encryptionPassphrase')}
</label>
<div className="relative">
<Input
type={showSecrets.encryption_passphrase ? 'text' : 'password'}
value={formData.backup_encryption_passphrase}
onChange={(e) => handleChange('backup_encryption_passphrase', e.target.value)}
placeholder={t('backup.configuration.advancedOptions.encryptionPassphraseHelp')}
required={formData.backup_encryption}
/>
<button
type="button"
onClick={() => setShowSecrets(prev => ({ ...prev, encryption_passphrase: !prev.encryption_passphrase }))}
className="absolute top-1/2 -translate-y-1/2 right-2 text-neutral-400 hover:text-neutral-600 dark:hover:text-neutral-300"
>
{showSecrets.encryption_passphrase ? <EyeOff size={20} /> : <Eye size={20} />}
</button>
</div>
<p className="mt-1 text-xs text-red-600">
<AlertCircle className="inline h-3 w-3 mr-1" />
{t('backup.configuration.advancedOptions.encryptionPassphraseHelp')}
</p>
</div>
)}
</div>
</div>
</Card>
{/* Save Button */}
<div className="flex justify-end">
<Button
@@ -258,6 +258,12 @@ const SummaryCard: React.FC<{
value={String(summary.skippedByFeatureFlagCount)}
/>
)}
{summary.skippedBySettingCount > 0 && (
<Row
label={t('backup.coverage.summary.skippedBySetting', 'Skipped (backup settings)')}
value={String(summary.skippedBySettingCount)}
/>
)}
{summary.missingOnDiskCount > 0 && (
<Row
label={t('backup.coverage.summary.missingOnDisk', 'Missing on disk')}
@@ -380,6 +386,10 @@ const CoverageBadge: React.FC<{ coverage: BackupPathCoverage }> = ({ coverage })
tone: 'neutral',
label: t('backup.coverage.coverage.skippedByFlag', 'Gated off'),
},
'skipped-by-setting': {
tone: 'neutral',
label: t('backup.coverage.coverage.skippedBySetting', 'Off (backup settings)'),
},
'missing-on-disk': {
tone: 'amber',
label: t('backup.coverage.coverage.missingOnDisk', 'Missing on disk'),
+2
View File
@@ -416,6 +416,7 @@
"willScan": "Wird gescannt",
"skippedByToggle": "Übersprungen (Schalter aus)",
"skippedByFlag": "Übersprungen (Feature-Flag)",
"skippedBySetting": "Übersprungen (Backup-Einstellungen)",
"missingOnDisk": "Auf Festplatte fehlend",
"drift": "Nicht konfiguriert auf Festplatte (Drift)"
},
@@ -430,6 +431,7 @@
"willScan": "Wird gescannt",
"skippedByToggle": "Aus",
"skippedByFlag": "Per Flag aus",
"skippedBySetting": "In Backup-Einstellungen aus",
"missingOnDisk": "Auf Festplatte fehlend"
},
"drift": {
+2
View File
@@ -2862,6 +2862,7 @@
"willScan": "Will scan",
"skippedByToggle": "Skipped (toggle off)",
"skippedByFlag": "Skipped (feature flag)",
"skippedBySetting": "Skipped (backup settings)",
"missingOnDisk": "Missing on disk",
"drift": "Unconfigured on disk (drift)"
},
@@ -2876,6 +2877,7 @@
"willScan": "Will scan",
"skippedByToggle": "Off",
"skippedByFlag": "Gated off",
"skippedBySetting": "Off (backup settings)",
"missingOnDisk": "Missing on disk"
},
"drift": {
@@ -128,7 +128,7 @@ export const BackupManagement: React.FC = () => {
<div className="flex items-center space-x-2">
<Clock className="h-5 w-5 text-neutral-400" />
<span className="text-sm text-neutral-600 dark:text-neutral-400">
{t('backup.status.nextBackup')}: {backupStatus?.nextBackup || t('backup.status.notScheduled')}
{t('backup.status.nextBackup')}: {backupStatus?.nextBackup ? fmtDateTime(backupStatus.nextBackup) : t('backup.status.notScheduled')}
</span>
</div>
)}
+2
View File
@@ -251,6 +251,7 @@ export type BackupPathCoverage =
| 'will-scan'
| 'skipped-by-toggle'
| 'skipped-by-feature-flag'
| 'skipped-by-setting'
| 'missing-on-disk';
export interface BackupCoveragePath {
@@ -289,6 +290,7 @@ export interface BackupCoverageReport {
willScanCount: number;
skippedByToggleCount: number;
skippedByFeatureFlagCount: number;
skippedBySettingCount: number;
missingOnDiskCount: number;
driftCount: number;
tableMissingFallbackInUse: boolean;