fix(accounting): PR #622 blockers — CSV formula injection + IMAP double-ingest race

Blocker 1 — CSV/Banana formula injection. Neither csvEscape (ledgerService) nor
the tax-report CSV escape nor the unquoted tab-separated Banana cell formatter
prefixed risky leading chars, so an admin-/sender-controlled cell beginning with
= + - @ TAB CR executes as a formula when the Treuhänder opens the export. New
shared util neutralizeSpreadsheetFormula() prepends a single quote; wired into
all three sinks (quoted CSV + unquoted Banana). Unit test pins one of each char.

Blocker 2 — IMAP intake double-ingest race. received_emails.message_id was
INDEX, not UNIQUE, and the poller ingested attachments BEFORE writing the audit
row, so a second replica / rolling-deploy overlap double-ingested the same mail.
Migration 128 makes message_id UNIQUE (nulls stay distinct); the intake now
CLAIMS the message row (status='processing') BEFORE ingesting — a concurrent
claim hits the unique constraint and skips cleanly (shared isUniqueViolation
helper). Stale 'processing' rows (worker crashed mid-ingest) are reclaimed after
10 min so no attachment is orphaned.

NOT done (deliberate): the suggested UNIQUE on inbound_documents.file_sha256 —
that column is a SOFT dedup key by design (manual re-uploads are kept as flagged
'duplicate' rows + duplicate_of_id for the Duplikat disposition); a unique index
would break that feature. The file race only yields an extra 'unsorted' row (a
data-quality nit, caught by the existing manual Duplikat backstop), not a
double-count. Rationale to be added to the PR reply.
This commit is contained in:
Luca
2026-06-16 18:21:02 +02:00
parent a7c19135bb
commit cd6d57839b
7 changed files with 134 additions and 16 deletions
+7 -1
View File
@@ -42,7 +42,13 @@ exports.up = async function (knex) {
table.integer('inbound_document_id').unsigned();
table.text('error');
table.timestamp('created_at').defaultTo(knex.fn.now());
table.index(['message_id']);
// UNIQUE (not just INDEX): message_id is the dedup/claim key for the IMAP
// poller. The in-process `polling` lock serialises within one backend, but a
// second replica / rolling-deploy overlap would otherwise let two workers
// both pass the check-then-insert and double-ingest the same mail. NULLs stay
// distinct (Postgres + SQLite) so no-Message-ID rows aren't blocked. The
// intake claims this row BEFORE ingesting.
table.unique(['message_id']);
table.index(['status']);
});
}