paul/picpeak
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

fix(archives): take the restored category from the manifest (#1240)

Browse Source 
* fix(archives): take the restored category from the manifest

The archive writer already persists `category_name` per photo in
photos_manifest.json — that is why the manifest exists, and the comment
above it says so: "(and category linkage) can't be derived from the
extracted files alone". The restore route then read only
`original_filename` out of it and kept deriving the category from the
ZIP's first path segment.

Archives store photos exactly as they sit on disk, so an event whose
photos live in the gallery root produces a FLAT zip. `path.dirname()` is
'.' for every entry, no category is resolved, and every restored photo
lands with `category_id = null` — silently, behind a 200.

Seen on a real restore: 596 photos back, 0 with a category, while the
nine category rows sat untouched in the table.

Now the manifest is the source of truth and the first path segment is
the fallback, so foldered archives and legacy archives without a
manifest behave exactly as before. The find-or-create is pulled into
`resolveCategoryId` so both paths share it and each name is resolved
once per restore.

Tests: __tests__/integration/adminArchives.restoreCategories.test.js
builds real ZIPs (flat with manifest, flat with an existing category
row, foldered without manifest) and drives POST /:id/restore. Without
this change the two manifest cases fail and the foldered one passes —
the fallback is unchanged.

* fix(archives): let the manifest be authoritative when it says "no category"

Review follow-up on #1240, pushed with the author's agreement.

The manifest won for "category X" but not for "none": an entry with a null
category_name fell through to the directory fallback, so a photo the archive
recorded as uncategorized came back filed under a category anyway.

That matters because the directory is not a category. Archive entry names are
the storage key minus `events/active/{slug}`, and that layout is
`individual/{filename}` / `collages/{filename}` — categories have never been
directories there. Reading the first path segment on a real archive invents
categories literally named "individual" and "collages", so the fallback was
overriding an accurate record with a junk one.

The fallback is now confined to photos with NO manifest entry at all: archives
written before the manifest existed, where the directory is the only signal
left and inventing those names still beats losing every category.

Tests: the legacy case now uses `individual/`, the shape a real archive
actually has, instead of a category-shaped folder no archive produces — so it
documents what the fallback really does. Plus a new case pinning that a
manifest saying uncategorized leaves the photo uncategorized and creates no
category row. It fails without this change; the legacy fallback keeps passing.

---------

Co-authored-by: Paul Nothaft <paul@MacStudio-von-Paul.local>
This commit is contained in:
Marian
2026-08-30 15:02:14 +00:00
committed by GitHub
parent 44a8c416c7
commit 0d340f4e81
2 changed files with 259 additions and 33 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/__tests__/integration/adminArchives.restoreCategories.test.js
+207
View File
@@ -0,0 +1,207 @@
/**
* Restoring an archive must put the photos back into their categories.
*
* The archive writer already persists `category_name` per photo in
* `photos_manifest.json` — that is why the manifest exists, and the comment
* above it says so: "(and category linkage) can't be derived from the
* extracted files alone". The restore route then read only
* `original_filename` from it and kept deriving the category from the ZIP's
* first path segment.
*
* Archives store photos exactly as they sit on disk, so an event whose photos
* live in the gallery root produces a FLAT zip. `path.dirname()` is '.' for
* every entry, no category is resolved, and every restored photo lands with
* `category_id = null` — silently, with a 200 response.
*
* These pin the manifest as the source of truth, with the directory as the
* fallback that keeps foldered and legacy archives working.
*/
const fs = require('fs');
const path = require('path');
const os = require('os');
const express = require('express');
const request = require('supertest');
describe('archive restore restores categories (flat archives included)', () => {
let tmpDir; let db; let cleanup; let app; let storagePath;
beforeAll(async () => {
tmpDir = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'picpeak-restore-cat-'));
storagePath = path.join(tmpDir, 'storage');
process.env.NODE_ENV = 'test';
process.env.TEST_DATABASE_PATH = path.join(tmpDir, 'data', 'test.db');
process.env.STORAGE_PATH = storagePath;
await fs.promises.mkdir(path.dirname(process.env.TEST_DATABASE_PATH), { recursive: true });
await fs.promises.mkdir(path.join(storagePath, 'archives'), { recursive: true });
jest.resetModules();
jest.doMock('../../src/middleware/auth', () => ({
adminAuth: (req, _res, next) => { req.admin = { id: 1, username: 'tester' }; next(); },
}));
jest.doMock('../../src/middleware/permissions', () => ({
requirePermission: () => (_req, _res, next) => next(),
}));
jest.doMock('../../src/middleware/ownership', () => ({
requireEventOwnership: (_req, _res, next) => next(),
}));
({ db, cleanup } = await require('./helpers/crmDb').bootCrmDb());
// bootCrmDb points STORAGE_PATH at its own tmp dir; follow it rather than
// fighting it, so the archives the tests write are where the route looks.
storagePath = process.env.STORAGE_PATH;
await fs.promises.mkdir(path.join(storagePath, 'archives'), { recursive: true });
app = express();
app.use(express.json());
app.use('/admin/archives', require('../../src/routes/adminArchives'));
}, 180000);
afterAll(async () => {
if (cleanup) await cleanup();
await fs.promises.rm(tmpDir, { recursive: true, force: true }).catch(() => {});
});
beforeEach(async () => {
await db('photos').del();
await db('photo_categories').del();
await db('events').del();
});
/** A one-pixel JPEG is enough; the route only stats the extracted file. */
const PIXEL = Buffer.from(
'/9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0a'
+ 'HBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/wAALCAABAAEBAREA/8QAFAABAAAAAAAA'
+ 'AAAAAAAAAAAACf/EABQQAQAAAAAAAAAAAAAAAAAAAAD/2gAIAQEAAD8AKp//2Q==',
'base64',
);
async function writeArchive(name, entries) {
// Required lazily: the suite calls jest.resetModules() in beforeAll, and
// archiver's readable-stream copy does not survive being split across the
// two module registries.
const archiver = require('archiver');
const archivePath = path.join(storagePath, 'archives', name);
await new Promise((resolve, reject) => {
const output = fs.createWriteStream(archivePath);
const zip = archiver('zip', { zlib: { level: 0 } });
output.on('close', resolve);
zip.on('error', reject);
zip.pipe(output);
for (const [entryName, buffer] of Object.entries(entries)) {
zip.append(buffer, { name: entryName });
}
zip.finalize();
});
return path.join('archives', name);
}
async function seedArchivedEvent(archiveRelPath, slug) {
const [row] = await db('events').insert({
slug,
event_type: 'wedding',
event_name: slug,
event_date: '2026-06-27',
host_email: 'h@example.com',
admin_email: 'a@example.com',
password_hash: 'x',
share_link: `${slug}-share`,
expires_at: new Date().toISOString(),
is_archived: 1, // sqlite stores booleans as 0/1, see utils/dbCompat
archive_path: archiveRelPath,
}).returning('id');
return typeof row === 'object' ? row.id : row;
}
const categoryOf = async (filename) => {
const photo = await db('photos').where('filename', filename).first();
if (!photo || !photo.category_id) return null;
const category = await db('photo_categories').where('id', photo.category_id).first();
return category ? category.name : null;
};
it('takes the category from the manifest when the archive is flat', async () => {
// Exactly the shape a gallery-root event archives to: no directories.
const manifest = JSON.stringify([
{ filename: 'a.jpg', original_filename: 'DSC_0001.jpg', category_name: 'Polterabend' },
{ filename: 'b.jpg', original_filename: 'DSC_0002.jpg', category_name: 'Ceremony' },
]);
const archiveRelPath = await writeArchive('flat.zip', {
'a.jpg': PIXEL,
'b.jpg': PIXEL,
'photos_manifest.json': Buffer.from(manifest, 'utf8'),
});
const eventId = await seedArchivedEvent(archiveRelPath, 'flat-event');
const res = await request(app).post(`/admin/archives/${eventId}/restore`).send({});
expect(res.status).toBe(200);
// The whole bug: both of these used to be null.
expect(await categoryOf('a.jpg')).toBe('Polterabend');
expect(await categoryOf('b.jpg')).toBe('Ceremony');
});
it('reuses an existing category row instead of creating a duplicate', async () => {
const archiveRelPath = await writeArchive('reuse.zip', {
'c.jpg': PIXEL,
'photos_manifest.json': Buffer.from(JSON.stringify([
{ filename: 'c.jpg', original_filename: 'DSC_0003.jpg', category_name: 'Party' },
]), 'utf8'),
});
const eventId = await seedArchivedEvent(archiveRelPath, 'reuse-event');
await db('photo_categories').insert({
event_id: eventId, name: 'Party', slug: 'party', created_at: new Date(),
});
const res = await request(app).post(`/admin/archives/${eventId}/restore`).send({});
expect(res.status).toBe(200);
expect(await categoryOf('c.jpg')).toBe('Party');
const rows = await db('photo_categories').where({ event_id: eventId, name: 'Party' });
expect(rows).toHaveLength(1);
});
it('still falls back to the directory for legacy archives with no manifest', async () => {
// No manifest at all — the shape every archive had before the manifest
// landed. The directory is the only signal left, and it must keep working.
//
// `individual/` is what a REAL archive contains: entry names are the
// storage key minus `events/active/{slug}`, and that layout is
// `individual/` / `collages/`. Categories have never been directories, so
// the fallback invents a category with that name — not useful, but better
// than losing every category, and this pins what actually happens rather
// than a category-shaped folder no archive produces.
const archiveRelPath = await writeArchive('foldered.zip', {
'individual/d.jpg': PIXEL,
});
const eventId = await seedArchivedEvent(archiveRelPath, 'foldered-event');
const res = await request(app).post(`/admin/archives/${eventId}/restore`).send({});
expect(res.status).toBe(200);
expect(await categoryOf('d.jpg')).toBe('individual');
});
it('honours a manifest that says UNCATEGORIZED, instead of inventing one from the directory', async () => {
// The case the manifest-first change was for. A real archive puts every
// photo under `individual/`, so a photo the manifest records as having no
// category used to come back filed under a category called "individual" —
// the manifest being authoritative for "category X" but not for "none".
const manifest = JSON.stringify([
{ filename: 'u.jpg', original_filename: 'DSC_7000.jpg', category_name: null },
]);
const archiveRelPath = await writeArchive('uncategorized.zip', {
'individual/u.jpg': PIXEL,
'photos_manifest.json': Buffer.from(manifest, 'utf8'),
});
const eventId = await seedArchivedEvent(archiveRelPath, 'uncategorized-event');
const res = await request(app).post(`/admin/archives/${eventId}/restore`).send({});
expect(res.status).toBe(200);
expect(await categoryOf('u.jpg')).toBeNull();
// And no junk category row was created as a side effect.
const rows = await db('photo_categories').where({ event_id: eventId });
expect(rows).toHaveLength(0);
});
});
backend/src/routes/adminArchives.js
+52 -33
View File
@@ -226,9 +226,35 @@ router.post('/:id/restore', adminAuth, requirePermission('archives.restore'), re
// Get list of extracted files to update database
const extractedPhotos = [];
// First, collect all category information from the ZIP structure
// Category name -> id, resolved once per name for the whole restore.
const categoriesMap = new Map();
// Find-or-create the category by name, scoped to this event.
const resolveCategoryId = async (categoryName) => {
if (!categoryName) return null;
if (categoriesMap.has(categoryName)) return categoriesMap.get(categoryName);
const existingCategory = await db('photo_categories')
.where('event_id', archive.id)
.where('name', categoryName)
.first();
if (existingCategory) {
categoriesMap.set(categoryName, existingCategory.id);
} else {
const insertResult = await db('photo_categories').insert({
event_id: archive.id,
name: categoryName,
slug: slugify(categoryName),
created_at: new Date()
}).returning('id');
categoriesMap.set(categoryName, insertResult[0]?.id || insertResult[0]);
}
return categoriesMap.get(categoryName);
};
for (const entry of entries) {
if (!entry.isDirectory && entry.name.match(/\.(jpg|jpeg|png|gif|webp)$/i)) {
const filename = path.basename(entry.name);
@@ -238,39 +264,33 @@ router.post('/:id/restore', adminAuth, requirePermission('archives.restore'), re
try {
// Check if file was extracted successfully
const stats = await fs.stat(actualFilePath);
// Determine category from directory structure
const manifestEntry = manifestByFilename.get(filename);
// The manifest is the only faithful source for the category, and
// it is authoritative INCLUDING when it says "none". A manifest
// entry with a null category_name means the photo was genuinely
// uncategorized, so falling through to the directory would
// contradict the very record being restored from.
//
// That matters because the directory is not a category. Archive
// entry names are the storage key minus `events/active/{slug}`,
// and that layout is `individual/{filename}` / `collages/…` —
// categories have never been directories there. Reading the first
// path segment on a real archive therefore invents categories
// literally named "individual" and "collages".
//
// So the fallback is confined to photos with NO manifest entry at
// all: archives written before the manifest existed, where the
// directory is the only signal left and inventing those two names
// is still better than losing every category.
let categoryId = null;
if (dirPath && dirPath !== '.') {
// Get the first level directory as category
const categoryName = dirPath.split(path.sep)[0];
if (!categoriesMap.has(categoryName)) {
// Check if this category exists in the database
const existingCategory = await db('photo_categories')
.where('event_id', archive.id)
.where('name', categoryName)
.first();
if (existingCategory) {
categoriesMap.set(categoryName, existingCategory.id);
} else {
// Create the category if it doesn't exist
const insertResult = await db('photo_categories').insert({
event_id: archive.id,
name: categoryName,
slug: slugify(categoryName),
created_at: new Date()
}).returning('id');
const newCategoryId = insertResult[0]?.id || insertResult[0];
categoriesMap.set(categoryName, newCategoryId);
}
}
categoryId = categoriesMap.get(categoryName);
if (manifestEntry) {
categoryId = await resolveCategoryId(manifestEntry.category_name);
} else if (dirPath && dirPath !== '.') {
categoryId = await resolveCategoryId(dirPath.split(path.sep)[0]);
}
// Check if photo already exists in database
const existingPhoto = await db('photos')
.where('event_id', archive.id)
@@ -280,7 +300,6 @@ router.post('/:id/restore', adminAuth, requirePermission('archives.restore'), re
if (!existingPhoto) {
// Store relative path from storage root
const relativePath = path.relative(storagePath, actualFilePath);
const manifestEntry = manifestByFilename.get(filename);
extractedPhotos.push({
event_id: archive.id,
filename: filename,