feat: increase file upload limit from 20 to 500 with performance optimizations
Test and Lint / backend-test (push) Successful in 1m10s
Test and Lint / frontend-test (push) Successful in 2m22s
continuous-integration/drone/push Build is passing
Version and Release / version-bump (push) Successful in 37s
Version and Release / trigger-drone (push) Successful in 3s

Backend changes:
- Update multer configuration to accept up to 500 files per upload
- Implement batch processing (10 files per transaction) for better performance
- Add memory-efficient Sharp configuration for thumbnail generation
- Increase Express body parser limits to handle large payloads
- Add proper error handling and reporting for partial upload failures

Frontend changes:
- Update validation to allow 500 files maximum
- Implement chunked uploads (50 files per chunk) to prevent timeouts
- Add progress tracking with chunk information display
- Update error messages and translations (EN/DE)

Performance optimizations:
- Disable Sharp cache to prevent memory buildup
- Limit Sharp concurrency to 2 operations
- Use sequential read for large images
- Process files in database transaction batches
- Return detailed upload results including success/failure counts

This implementation ensures the application can handle large photo uploads
efficiently without running into memory or timeout issues.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
2025-07-14 21:11:59 +02:00
parent ac48bfdd0d
commit 6906c8bcf7
6 changed files with 228 additions and 118 deletions
+26 -9
View File
@@ -2,6 +2,10 @@ const sharp = require('sharp');
const path = require('path');
const fs = require('fs').promises;
// Configure sharp for better memory management with large batches
sharp.cache(false); // Disable cache to prevent memory buildup
sharp.concurrency(2); // Limit concurrent operations
const THUMBNAIL_WIDTH = 300;
const getStoragePath = () => process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
const getThumbnailPath = () => path.join(getStoragePath(), 'thumbnails');
@@ -15,16 +19,29 @@ async function generateThumbnail(imagePath) {
// Ensure thumbnail directory exists
await fs.mkdir(thumbnailDir, { recursive: true });
// Generate thumbnail
await sharp(imagePath)
.resize(THUMBNAIL_WIDTH, null, {
withoutEnlargement: true,
fit: 'inside'
try {
// Generate thumbnail with memory-efficient settings
await sharp(imagePath, {
limitInputPixels: 268402689, // ~16k x 16k max
sequentialRead: true // More memory efficient for large images
})
.jpeg({ quality: 80 })
.toFile(thumbnailPath);
return path.relative(getStoragePath(), thumbnailPath);
.resize(THUMBNAIL_WIDTH, null, {
withoutEnlargement: true,
fit: 'inside'
})
.jpeg({
quality: 80,
progressive: true, // Progressive JPEG for better loading
mozjpeg: true // Better compression
})
.toFile(thumbnailPath);
return path.relative(getStoragePath(), thumbnailPath);
} catch (error) {
console.error(`Failed to generate thumbnail for ${filename}:`, error);
// Return null if thumbnail generation fails, don't fail the whole upload
return null;
}
}
module.exports = { generateThumbnail };