paul/picpeak
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

fix(archives): sort and total on real archive sizes, escape LIKE wildcards

Browse Source 
Closes the three trade-offs the server-side archives query deliberately
accepted.

C1 -- the sorted number and the displayed number are now the same one.
There was no archive_size column, so the Size column came from a per-row
fs.stat done after pagination while the sort fell back to summed photo bytes:
the list could be ordered by a number the user was not looking at. Adds
events.archive_size (bigInteger -- int4's 2.1GB ceiling is the same limit
that forced the restore path off adm-zip), written at archive time from
archive.pointer(), which is the exact byte count the completion email already
reports. The route now sorts and displays that column and no longer touches
the filesystem. The migration backfills by stat-ing every archive_path where
the column is null, outside the column guard so a half-finished run
self-heals; unstatable rows (missing zip, S3-backed storage) stay null, order
last via COALESCE and display 0 -- exactly what the old fs.stat produced for
a file it could not read. Restore nulls it alongside archive_path.
Accepted: the list no longer notices a zip deleted out of band and shows the
last recorded size. The detail route still stats the real file.

C2 -- escape \ % _ in the bound value plus an explicit ESCAPE '\'. The
ESCAPE clause is load-bearing rather than decorative: SQLite has no default
LIKE escape character, so without it the escaped pattern matches literal
backslashes and the search silently returns nothing on SQLite while working
on Postgres. The value stays bound; no interpolation.

C3 -- the four stat cards aggregated only the current page, so every total
was wrong for any dataset past page one. The list response now carries
totals { archives, photos, archiveSize } computed with the same applyFilters()
closure as pagination.total, so cards and footer cannot drift. Two aggregate
queries: archive_size sums on the unjoined events query (joining photos
multiplies it by photo count) and photos count on the joined one, both read
back through Number() for pg's bigint-as-string. The "Showing X of Y" line
moved out of the totalPages > 1 guard so it survives a single-page result,
now gated on total > 0 so a zero-result search does not render
"Showing 1 to 0 of 0"; only the page controls stay conditional.

Test fixtures deliberately order zip sizes differently from summed photo
bytes, so the sort test can only pass on the right column.

Refs testplan REPORT.md C1, C2, C3.
This commit is contained in:
Paul Nothaft
2026-09-02 09:43:10 +02:00
parent 4c2eeab2f4
commit da6e34d6a3
7 changed files with 380 additions and 84 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/src/routes/adminArchives.js
+49 -27
View File
@@ -22,18 +22,23 @@ router.get('/', adminAuth, requirePermission('archives.view'), async (req, res)
const type = typeof req.query.type === 'string' ? req.query.type.trim() : '';
const sortBy = ['date', 'name', 'size'].includes(req.query.sortBy) ? req.query.sortBy : 'date';
// A literal % or _ typed into the search box has to match itself rather
// than act as a wildcard. Escape the escape character first, then the two
// wildcards. utils/sqlSecurity.js's escapeLikePattern() is not usable
// here: it also doubles single quotes, which corrupts a BOUND value —
// "Sarah's Birthday" would be searched for as "Sarah''s Birthday".
const escapeLike = (value) => value.replace(/[\\%_]/g, '\\$&');
// Search and type filtering run in SQL so both the returned rows and
// the total count cover the whole archive table, not just the page the
// client happens to be on. Values are bound, never interpolated.
// % and _ are wildcards to LIKE but literal characters to the client-side
// `includes()` this replaced, so searching for "100%" would otherwise match
// every archive and report a nonsense total. The ESCAPE clause is
// load-bearing rather than decorative: SQLite has no default LIKE escape
// character, so without it the escaped pattern matches literal backslashes
// there while working on Postgres.
const escapeLike = (value) => value.replace(/[\\%_]/g, '\\$&');
const applyFilters = (query) => {
if (search) {
// The ESCAPE clause is explicit because the two engines disagree
// without it: Postgres treats a backslash in a LIKE pattern as an
// escape by default, SQLite has no default escape character at all and
// would match the backslash literally. Naming it makes the escaping
// above mean the same thing on both.
query.whereRaw(
'LOWER(events.event_name) LIKE ? ESCAPE \'\\\'',
[`%${escapeLike(search.toLowerCase())}%`]
@@ -45,11 +50,24 @@ router.get('/', adminAuth, requirePermission('archives.view'), async (req, res)
return query;
};
// Get total count (of the filtered set, so pagination stays truthful)
// Totals for the filtered set, so pagination AND the stat cards describe
// the whole result rather than the page. Unjoined: joining photos here
// would multiply archive_size by the event's photo count.
const totalCount = await applyFilters(
db('events').where('events.is_archived', formatBoolean(true))
)
.count('events.id as count')
.sum({ archive_size: 'events.archive_size' })
.first();
// Photo total needs the join, so it is its own query for the same reason.
// count(photos.id) rather than count(*): a left-joined event with no
// photos must contribute 0, not 1.
const photoTotal = await applyFilters(
db('events').where('events.is_archived', formatBoolean(true))
)
.leftJoin('photos', 'events.id', 'photos.event_id')
.count('photos.id as count')
.first();
// Get archived events
@@ -67,29 +85,20 @@ router.get('/', adminAuth, requirePermission('archives.view'), async (req, res)
if (sortBy === 'name') {
archivesQuery.orderBy('events.event_name', 'asc');
} else if (sortBy === 'size') {
// The zip's on-disk size is only known after the per-row fs.stat below,
// so a global size sort has to use the archived content size instead.
archivesQuery.orderByRaw('COALESCE(SUM(photos.size_bytes), 0) desc');
// events.archive_size — the same number the Size column renders. It was
// the archived *content* size (SUM(photos.size_bytes)) until the column
// existed, which meant the list could be ordered by a value that is not
// on screen. NULL is an archive whose zip could not be measured (missing
// file, or a storage backend the backfill could not stat); it sorts as 0,
// i.e. last, which is also what it displays as.
archivesQuery.orderByRaw('COALESCE(events.archive_size, 0) desc');
} else {
archivesQuery.orderBy('events.archived_at', 'desc');
}
const archives = await archivesQuery.limit(limit).offset(offset);
// Check if archive files exist and get their sizes
const storagePath = process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
const archivesWithFileInfo = await Promise.all(archives.map(async (archive) => {
let archiveFileSize = 0;
if (archive.archive_path) {
try {
const fullArchivePath = path.join(storagePath, archive.archive_path);
const stats = await fs.stat(fullArchivePath);
archiveFileSize = stats.size;
} catch (error) {
logger.error(`Archive file not found: ${archive.archive_path}`);
}
}
const archivesWithFileInfo = archives.map((archive) => {
return {
id: archive.id,
slug: archive.slug,
@@ -101,10 +110,11 @@ router.get('/', adminAuth, requirePermission('archives.view'), async (req, res)
expiresAt: archive.expires_at ? new Date(archive.expires_at).toISOString() : null,
photoCount: archive.photo_count || 0,
originalSize: archive.total_size || 0,
archiveSize: archiveFileSize,
// Number(): bigInteger comes back from the pg driver as a string.
archiveSize: Number(archive.archive_size) || 0,
archivePath: archive.archive_path
};
}));
});
res.json({
archives: archivesWithFileInfo,
@@ -113,6 +123,14 @@ router.get('/', adminAuth, requirePermission('archives.view'), async (req, res)
limit,
total: totalCount.count,
totalPages: Math.ceil(totalCount.count / limit)
},
// Aggregates over the FILTERED set — the stat cards used to sum the rows
// the client could see, so every figure was page-scoped while the footer
// beside them reported the real total.
totals: {
archives: Number(totalCount.count) || 0,
photos: Number(photoTotal.count) || 0,
archiveSize: Number(totalCount.archive_size) || 0
}
});
} catch (error) {
@@ -533,6 +551,10 @@ router.post('/:id/restore', adminAuth, requirePermission('archives.restore'), re
is_archived: false,
is_active: true,
archive_path: null,
// Cleared with the path it measures — a restored event has no zip, and
// a stale size would be re-shown verbatim if it is archived again
// before the new archive finishes writing.
archive_size: null,
archived_at: null,
expires_at: thirtyDaysFromNow.toISOString() // Reset expiration - works on both DBs
});
backend/src/services/archiveService.js
+4
View File
@@ -166,6 +166,10 @@ async function archiveEvent(event) {
await db('events').where('id', event.id).update({
is_archived: true,
archive_path: archiveRelKey,
// The zip's own byte size — the same number the completion email
// reports below. Persisted so the archives list can sort and display it
// without statting every archive on every request.
archive_size: totalBytes,
archived_at: new Date(),
});