fix(backup): incremental backups against S3 + jsonb stats parsing
Three fixes uncovered while bringing the backup-s3 integration suite to 12/12 against MinIO + Postgres: - backupService.getDatabaseBackupInfo: pg's jsonb driver auto-parses `statistics` / `table_checksums` to objects; the old JSON.parse() then threw "[object Object]" is not valid JSON and the manifest dropped database info silently. Accept both string and object inputs. - backupService.runBackup: incremental path called backupManifest.loadManifest() with an s3:// URI directly, which falls through to fs.readFile() and ENOENTs — every "incremental" backup silently downgraded to a full one. Added loadManifestFromAnywhere() helper that downloads s3:// to a tmp file before delegating. - backupManifest.generateIncrementalManifest: attached the `incremental` section AFTER generateManifest() had already stamped verification.total_checksum, so every incremental manifest failed validateManifest() on read-back. Recompute the checksum after. Test side: updated assertions to the current manifest shape (`incremental.changes.modified_files_count`), Number()-coerce bigint columns from pg, and gate the logger mock on UNMOCK_LOGGER for diagnosing similar silent-failure modes in the future.
This commit is contained in:
@@ -80,10 +80,12 @@ describe('S3 Backup Integration Tests', () => {
|
||||
await setupTestData();
|
||||
|
||||
// Mock logger to reduce noise
|
||||
logger.info = jest.fn();
|
||||
logger.debug = jest.fn();
|
||||
logger.warn = jest.fn();
|
||||
logger.error = jest.fn();
|
||||
if (process.env.UNMOCK_LOGGER !== 'true') {
|
||||
logger.info = jest.fn();
|
||||
logger.debug = jest.fn();
|
||||
logger.warn = jest.fn();
|
||||
logger.error = jest.fn();
|
||||
}
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
@@ -281,13 +283,16 @@ describe('S3 Backup Integration Tests', () => {
|
||||
.first();
|
||||
|
||||
expect(secondRun.id).not.toBe(firstRun.id);
|
||||
expect(secondRun.files_backed_up).toBe(1); // Only modified file
|
||||
expect(Number(secondRun.files_backed_up)).toBe(1); // Only modified file
|
||||
|
||||
// Check manifest indicates incremental
|
||||
// Check manifest indicates incremental. The current manifest schema
|
||||
// groups counts under `incremental.changes.*` (added/modified/deleted/
|
||||
// unchanged + size_difference) — see backupManifest.generateIncrementalManifest.
|
||||
if (secondRun.manifest_path) {
|
||||
const manifest = await backupService.getBackupManifest(secondRun.id);
|
||||
expect(manifest.manifest.incremental).toBeDefined();
|
||||
expect(manifest.manifest.incremental.modified_files_count).toBe(1);
|
||||
expect(manifest.manifest.incremental.changes).toBeDefined();
|
||||
expect(manifest.manifest.incremental.changes.modified_files_count).toBe(1);
|
||||
}
|
||||
});
|
||||
|
||||
|
||||
Reference in New Issue
Block a user