fix(accounting): PR #622 blockers — CSV formula injection + IMAP double-ingest race
Blocker 1 — CSV/Banana formula injection. Neither csvEscape (ledgerService) nor the tax-report CSV escape nor the unquoted tab-separated Banana cell formatter prefixed risky leading chars, so an admin-/sender-controlled cell beginning with = + - @ TAB CR executes as a formula when the Treuhänder opens the export. New shared util neutralizeSpreadsheetFormula() prepends a single quote; wired into all three sinks (quoted CSV + unquoted Banana). Unit test pins one of each char. Blocker 2 — IMAP intake double-ingest race. received_emails.message_id was INDEX, not UNIQUE, and the poller ingested attachments BEFORE writing the audit row, so a second replica / rolling-deploy overlap double-ingested the same mail. Migration 128 makes message_id UNIQUE (nulls stay distinct); the intake now CLAIMS the message row (status='processing') BEFORE ingesting — a concurrent claim hits the unique constraint and skips cleanly (shared isUniqueViolation helper). Stale 'processing' rows (worker crashed mid-ingest) are reclaimed after 10 min so no attachment is orphaned. NOT done (deliberate): the suggested UNIQUE on inbound_documents.file_sha256 — that column is a SOFT dedup key by design (manual re-uploads are kept as flagged 'duplicate' rows + duplicate_of_id for the Duplikat disposition); a unique index would break that feature. The file race only yields an extra 'unsorted' row (a data-quality nit, caught by the existing manual Duplikat backstop), not a double-count. Rationale to be added to the PR reply.
This commit is contained in:
@@ -42,7 +42,13 @@ exports.up = async function (knex) {
|
||||
table.integer('inbound_document_id').unsigned();
|
||||
table.text('error');
|
||||
table.timestamp('created_at').defaultTo(knex.fn.now());
|
||||
table.index(['message_id']);
|
||||
// UNIQUE (not just INDEX): message_id is the dedup/claim key for the IMAP
|
||||
// poller. The in-process `polling` lock serialises within one backend, but a
|
||||
// second replica / rolling-deploy overlap would otherwise let two workers
|
||||
// both pass the check-then-insert and double-ingest the same mail. NULLs stay
|
||||
// distinct (Postgres + SQLite) so no-Message-ID rows aren't blocked. The
|
||||
// intake claims this row BEFORE ingesting.
|
||||
table.unique(['message_id']);
|
||||
table.index(['status']);
|
||||
});
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user