paul/picpeak
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

feat(backup): admin endpoint to verify CRM document-artefact integrity

Browse Source 
Diagnostic for the bug fixed in a9280ea — confirms every *_path
column on quotes / contracts / invoices points at a file that
actually exists on disk and (where a *_sha256 column is set) the
file's bytes still hash to the expected value. Read-only;
on-demand only; no scheduler.
Per the design decisions locked in this PR's design call:
  D1 — on-demand only for v1; scheduling deferred until we have
       runtime data on large installs
  D2 — not auto-triggered after restore; surface a "verify
       integrity now" CTA on the restore-completed screen instead
  D3 — wet-upload contracts hash-verified same as system-rendered
       (signed_pdf_sha256 is computed at upload time, no special
       case needed in the verifier)
Coverage (single source of truth in backupIntegrityService.CHECKS):
  quotes.pdf_path                           existence
  contracts.pdf_path + pdf_sha256           existence + hash
  contracts.signed_pdf_path + signed_pdf_sha256  existence + hash
  contracts.signed_customer_signature_path  existence  (PNG/JPG, no hash)
  contracts.signed_admin_signature_path     existence  (PNG/JPG, no hash)
  invoices.pdf_path                         existence
  invoices.imported_pdf_path                existence  (admin-uploaded scans)
Report shape buckets each row into verifiedOk / missing /
hashMismatches / existsButNoHash so callers can distinguish hash-
verified from existence-only — the latter is weaker evidence in
a legal dispute and the UI should reflect that.
Route GET /api/admin/system-health/backup-integrity accepts an
optional ?scope= CSV filter (quote | contract | contract-signature
| invoice). Unknown scope tokens are rejected with a 400 +
BACKUP_INTEGRITY_UNKNOWN_SCOPE code rather than silently scanning
everything.
Frontend half (BackupIntegrityCard on a System Health page) is
deferred until backlog #11 (System Health page) is scaffolded.
The endpoint is independently useful via curl in the meantime.
This commit is contained in:
Luca
2026-05-29 13:00:18 +02:00
parent a9280ea9ba
commit 4812fcdec3
5 changed files with 649 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/src/routes/adminSystemHealth.js
+64
View File
@@ -0,0 +1,64 @@
/**
* Admin → System Health
*
* Endpoint mounted at /api/admin/system-health. The "Backup
* integrity" sub-endpoint is the on-demand verifier for CRM
* document artefacts — confirms every `*_path` column on quotes /
* contracts / invoices points at a file that actually exists on
* disk and (where a `*_sha256` column is set) the file's bytes
* still hash to the expected value.
*
* Per the design decisions locked with the maintainer:
* - On-demand only; no scheduler (D1)
* - Not auto-triggered after restore (D2)
* - Wet-upload contracts are hash-verified same as system-rendered (D3)
*
* Read-only. Returns a JSON report — never mutates DB or fs.
*/
const express = require('express');
const { query } = require('express-validator');
const { adminAuth } = require('../middleware/auth');
const { requirePermission } = require('../middleware/permissions');
const { handleAsync, validateRequest, successResponse } = require('../utils/routeHelpers');
const { verifyDocumentArtefacts } = require('../services/backupIntegrityService');
const router = express.Router();
router.use(adminAuth);
const VALID_SCOPES = ['quote', 'contract', 'contract-signature', 'invoice'];
router.get(
'/backup-integrity',
requirePermission('settings.view'),
[
// CSV string like `?scope=contract,invoice`. Each member must be
// one of the four known scopes. Empty / omitted means full scan.
query('scope').optional({ values: 'falsy' }).isString().isLength({ max: 128 }),
],
handleAsync(async (req, res) => {
validateRequest(req);
let scope;
if (req.query.scope) {
scope = String(req.query.scope)
.split(',')
.map((s) => s.trim())
.filter(Boolean);
// Defense-in-depth: reject unknown scope tokens so a typo doesn't
// silently scan everything when the caller wanted just one slice.
const unknown = scope.filter((s) => !VALID_SCOPES.includes(s));
if (unknown.length > 0) {
return res.status(400).json({
error: `Unknown scope(s): ${unknown.join(', ')}`,
code: 'BACKUP_INTEGRITY_UNKNOWN_SCOPE',
validScopes: VALID_SCOPES,
});
}
}
const report = await verifyDocumentArtefacts({ scope });
return successResponse(res, { report });
}),
);
module.exports = router;
backend/src/services/backupIntegrityService.js
+228
View File
@@ -0,0 +1,228 @@
/**
* Backup-integrity verifier — walks every CRM document-artefact path
* column and confirms (a) the file exists on disk, (b) when a SHA-256
* is stored, the file's actual bytes hash to the stored value.
*
* **Why this is a separate service**
*
* The audit trail captured at issue / sign time (signed_customer_ip,
* signed_by_customer_at, signed_pdf_sha256, signed_*_signature_path,
* issue_date, etc.) is worth exactly nothing on its own — what makes
* it legally meaningful is being able to produce the document the
* audit trail refers to. A backup that captures the DB but skips
* `storage/business-docs/` (the bug fixed in this same PR) leaves
* every `*_path` column a broken FK and every `*_sha256` column with
* nothing to verify against. This service is the diagnostic for
* exactly that drift — runs on demand, surfaces missing files +
* hash mismatches without making any changes.
*
* **Verification modes**
*
* - existence — file at `*_path` must exist on disk
* - sha256 — file at `*_path` must exist AND its sha256 must
* equal `*_sha256` column (when that column is set)
*
* Per-table coverage (lines reference migrations/core/107_crm_consolidated.js):
*
* quotes.pdf_path line 844 (existence)
* contracts.pdf_path line 1245 (existence + sha256 via contracts.pdf_sha256)
* contracts.signed_pdf_path line 1246 (existence + sha256 via contracts.signed_pdf_sha256)
* contracts.signed_customer_signature_path line 1269 (existence — drawn signatures, no hash column)
* contracts.signed_admin_signature_path line 1273 (existence — admin counter-signature drawing)
* invoices.pdf_path line 1026 (existence)
* invoices.imported_pdf_path line 1020 (existence — admin-uploaded scans)
*
* Wet uploads (`contracts.signed_pdf_is_wet_upload = true`) DO have a
* `signed_pdf_sha256` computed at upload time (contractService.js
* upload route), so they're hash-verified the same as system-rendered
* contracts — no special case here.
*
* **What this service does NOT do**
*
* - Does not write anything (no DB mutations, no fs touches)
* - Does not fail the request when a mismatch is found — the
* report shape carries the data, the caller decides what to do
* - Does not auto-trigger after restore (D2 decision: surface a
* CTA on the restore-completed screen instead)
* - Does not run on a schedule (D1 decision: on-demand v1; revisit
* once we have runtime data on large installs)
*/
const fs = require('fs');
const crypto = require('crypto');
const path = require('path');
const { db } = require('../database/db');
const logger = require('../utils/logger');
const STORAGE_ROOT = () => process.env.STORAGE_PATH || path.join(process.cwd(), 'storage');
/**
* Every column the verifier walks, declared once so the test suite
* and the service share a single source of truth. Order is the order
* the report lists rows in — table-major, then column-by-column.
*/
const CHECKS = [
{ table: 'quotes', pathColumn: 'pdf_path', shaColumn: null, scope: 'quote' },
{ table: 'contracts', pathColumn: 'pdf_path', shaColumn: 'pdf_sha256', scope: 'contract' },
{ table: 'contracts', pathColumn: 'signed_pdf_path', shaColumn: 'signed_pdf_sha256', scope: 'contract' },
{ table: 'contracts', pathColumn: 'signed_customer_signature_path', shaColumn: null, scope: 'contract-signature' },
{ table: 'contracts', pathColumn: 'signed_admin_signature_path', shaColumn: null, scope: 'contract-signature' },
{ table: 'invoices', pathColumn: 'pdf_path', shaColumn: null, scope: 'invoice' },
{ table: 'invoices', pathColumn: 'imported_pdf_path', shaColumn: null, scope: 'invoice' },
];
/** Stream-hash a file to sha256 hex without buffering the whole thing. */
function hashFile(absPath) {
return new Promise((resolve, reject) => {
const hash = crypto.createHash('sha256');
const stream = fs.createReadStream(absPath);
stream.on('error', reject);
stream.on('data', (chunk) => hash.update(chunk));
stream.on('end', () => resolve(hash.digest('hex')));
});
}
/**
* @param {object} [options]
* @param {string[]} [options.scope] Filter checks by scope tag:
* 'quote' | 'contract' | 'contract-signature' | 'invoice'.
* Defaults to all four (full scan).
* @returns {Promise<{
* scannedAt: string,
* scopes: string[],
* summary: {
* totalRows: number,
* verifiedOk: number,
* missingFiles: number,
* hashMismatches: number,
* existsButNoHash: number,
* },
* missing: Array<{ table, rowId, column, expectedPath }>,
* hashMismatches: Array<{ table, rowId, column, expectedPath, expectedSha, actualSha }>,
* existsButNoHash: Array<{ table, rowId, column, path }>,
* }>}
*
* `existsButNoHash` is the existence-only-verified bucket — the file
* was found but no `*_sha256` column exists for it (quote/invoice PDFs,
* signature PNGs). Surfaced separately so admins can distinguish
* "verified by hash" from "verified by existence only" — the latter
* is weaker evidence in a legal dispute.
*/
async function verifyDocumentArtefacts(options = {}) {
const scopes = Array.isArray(options.scope) && options.scope.length > 0
? options.scope.slice()
: Array.from(new Set(CHECKS.map((c) => c.scope)));
const checksToRun = CHECKS.filter((c) => scopes.includes(c.scope));
const storageRoot = STORAGE_ROOT();
const missing = [];
const hashMismatches = [];
const existsButNoHash = [];
let totalRows = 0;
let verifiedOk = 0;
for (const check of checksToRun) {
// Skip the check cleanly when the column or table doesn't exist
// on this install — keeps the verifier safe to run on partial
// migrations or installs that have features disabled.
if (!(await db.schema.hasTable(check.table))) continue;
if (!(await db.schema.hasColumn(check.table, check.pathColumn))) continue;
const select = ['id', check.pathColumn];
const hasHashColumn = check.shaColumn
&& (await db.schema.hasColumn(check.table, check.shaColumn));
if (hasHashColumn) select.push(check.shaColumn);
const rows = await db(check.table)
.whereNotNull(check.pathColumn)
.select(...select);
for (const row of rows) {
totalRows += 1;
const storedPath = row[check.pathColumn];
// Stored paths can be absolute (older rows) or relative-to-
// storage (newer rows). Normalize: resolve relative paths
// against STORAGE_PATH; absolute paths are used verbatim.
const absPath = path.isAbsolute(storedPath)
? storedPath
: path.join(storageRoot, storedPath);
let exists = false;
try {
exists = fs.existsSync(absPath);
} catch (_) { exists = false; }
if (!exists) {
missing.push({
table: check.table,
rowId: row.id,
column: check.pathColumn,
expectedPath: storedPath,
});
continue;
}
const expectedSha = hasHashColumn ? row[check.shaColumn] : null;
if (!expectedSha) {
// File exists but we have no hash to verify it against.
existsButNoHash.push({
table: check.table,
rowId: row.id,
column: check.pathColumn,
path: storedPath,
});
continue;
}
let actualSha;
try {
actualSha = await hashFile(absPath);
} catch (err) {
logger.warn(`backupIntegrity: failed to hash ${absPath}: ${err.message}`);
missing.push({
table: check.table,
rowId: row.id,
column: check.pathColumn,
expectedPath: storedPath,
});
continue;
}
if (actualSha !== expectedSha) {
hashMismatches.push({
table: check.table,
rowId: row.id,
column: check.pathColumn,
expectedPath: storedPath,
expectedSha,
actualSha,
});
continue;
}
verifiedOk += 1;
}
}
return {
scannedAt: new Date().toISOString(),
scopes,
summary: {
totalRows,
verifiedOk,
missingFiles: missing.length,
hashMismatches: hashMismatches.length,
existsButNoHash: existsButNoHash.length,
},
missing,
hashMismatches,
existsButNoHash,
};
}
module.exports = {
verifyDocumentArtefacts,
// Exported for tests; not part of the route API.
_internal: { CHECKS, hashFile },
};