fix(archives): sort and total on real archive sizes, escape LIKE wildcards
Closes the three trade-offs the server-side archives query deliberately
accepted.
C1 -- the sorted number and the displayed number are now the same one.
There was no archive_size column, so the Size column came from a per-row
fs.stat done after pagination while the sort fell back to summed photo bytes:
the list could be ordered by a number the user was not looking at. Adds
events.archive_size (bigInteger -- int4's 2.1GB ceiling is the same limit
that forced the restore path off adm-zip), written at archive time from
archive.pointer(), which is the exact byte count the completion email already
reports. The route now sorts and displays that column and no longer touches
the filesystem. The migration backfills by stat-ing every archive_path where
the column is null, outside the column guard so a half-finished run
self-heals; unstatable rows (missing zip, S3-backed storage) stay null, order
last via COALESCE and display 0 -- exactly what the old fs.stat produced for
a file it could not read. Restore nulls it alongside archive_path.
Accepted: the list no longer notices a zip deleted out of band and shows the
last recorded size. The detail route still stats the real file.
C2 -- escape \ % _ in the bound value plus an explicit ESCAPE '\'. The
ESCAPE clause is load-bearing rather than decorative: SQLite has no default
LIKE escape character, so without it the escaped pattern matches literal
backslashes and the search silently returns nothing on SQLite while working
on Postgres. The value stays bound; no interpolation.
C3 -- the four stat cards aggregated only the current page, so every total
was wrong for any dataset past page one. The list response now carries
totals { archives, photos, archiveSize } computed with the same applyFilters()
closure as pagination.total, so cards and footer cannot drift. Two aggregate
queries: archive_size sums on the unjoined events query (joining photos
multiplies it by photo count) and photos count on the joined one, both read
back through Number() for pg's bigint-as-string. The "Showing X of Y" line
moved out of the totalPages > 1 guard so it survives a single-page result,
now gated on total > 0 so a zero-result search does not render
"Showing 1 to 0 of 0"; only the page controls stay conditional.
Test fixtures deliberately order zip sizes differently from summed photo
bytes, so the sort test can only pass on the right column.
Refs testplan REPORT.md C1, C2, C3.
This commit is contained in:
@@ -0,0 +1,63 @@
|
||||
/**
|
||||
* `events.archive_size` — the archive zip's own byte size, recorded once at
|
||||
* archive time.
|
||||
*
|
||||
* The archives list showed a Size column produced by a per-row `fs.stat` run
|
||||
* AFTER pagination, so the number was only ever known for the 20 rows on
|
||||
* screen. That made a server-side "sort by size" impossible without statting
|
||||
* every archive on every request, and the sort fell back to
|
||||
* `SUM(photos.size_bytes)` — the archived *content* size, a different number
|
||||
* from the one in the column. Rows whose zip is missing or whose compression
|
||||
* ratio differs sorted by a value the admin was not looking at.
|
||||
*
|
||||
* With the size on the row, the sorted number and the displayed number are the
|
||||
* same number, and the list stops touching the filesystem at all.
|
||||
*
|
||||
* bigInteger, not integer: a real wedding archive crosses int4's 2.1 GB
|
||||
* ceiling routinely — that limit is why the restore path had to move off
|
||||
* adm-zip. Read it back through `Number()`, the pg driver hands bigints out as
|
||||
* strings.
|
||||
*/
|
||||
|
||||
const fs = require('fs').promises;
|
||||
const path = require('path');
|
||||
|
||||
exports.up = async function (knex) {
|
||||
if (!(await knex.schema.hasColumn('events', 'archive_size'))) {
|
||||
await knex.schema.alterTable('events', (table) => {
|
||||
table.bigInteger('archive_size');
|
||||
});
|
||||
}
|
||||
|
||||
// Backfill deliberately OUTSIDE the column guard: a run that died after the
|
||||
// alterTable but partway through the stats would leave the column present
|
||||
// and half the rows empty, and the re-run would skip both. `whereNull` makes
|
||||
// this idempotent and self-healing.
|
||||
//
|
||||
// Same storage-root resolution the routes use, so the migration reads the
|
||||
// files the app writes.
|
||||
const storagePath = process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
|
||||
const rows = await knex('events')
|
||||
.whereNotNull('archive_path')
|
||||
.whereNull('archive_size')
|
||||
.select('id', 'archive_path');
|
||||
|
||||
for (const row of rows) {
|
||||
try {
|
||||
const stats = await fs.stat(path.join(storagePath, row.archive_path));
|
||||
await knex('events').where('id', row.id).update({ archive_size: stats.size });
|
||||
} catch (_) {
|
||||
// Zip gone, or on a storage backend this process cannot stat (S3). Left
|
||||
// NULL, which the list renders and orders as 0 — exactly what the per-row
|
||||
// fs.stat this replaces already did for a file it could not read.
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
exports.down = async function (knex) {
|
||||
if (await knex.schema.hasColumn('events', 'archive_size')) {
|
||||
await knex.schema.alterTable('events', (table) => {
|
||||
table.dropColumn('archive_size');
|
||||
});
|
||||
}
|
||||
};
|
||||
Reference in New Issue
Block a user