feat: increase file upload limit from 20 to 500 with performance optimizations
Test and Lint / backend-test (push) Successful in 1m10s
Test and Lint / frontend-test (push) Successful in 2m22s
continuous-integration/drone/push Build is passing
Version and Release / version-bump (push) Successful in 37s
Version and Release / trigger-drone (push) Successful in 3s
Test and Lint / backend-test (push) Successful in 1m10s
Test and Lint / frontend-test (push) Successful in 2m22s
continuous-integration/drone/push Build is passing
Version and Release / version-bump (push) Successful in 37s
Version and Release / trigger-drone (push) Successful in 3s
Backend changes: - Update multer configuration to accept up to 500 files per upload - Implement batch processing (10 files per transaction) for better performance - Add memory-efficient Sharp configuration for thumbnail generation - Increase Express body parser limits to handle large payloads - Add proper error handling and reporting for partial upload failures Frontend changes: - Update validation to allow 500 files maximum - Implement chunked uploads (50 files per chunk) to prevent timeouts - Add progress tracking with chunk information display - Update error messages and translations (EN/DE) Performance optimizations: - Disable Sharp cache to prevent memory buildup - Limit Sharp concurrency to 2 operations - Use sequential read for large images - Process files in database transaction batches - Return detailed upload results including success/failure counts This implementation ensures the application can handle large photo uploads efficiently without running into memory or timeout issues. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -2,6 +2,10 @@ const sharp = require('sharp');
|
||||
const path = require('path');
|
||||
const fs = require('fs').promises;
|
||||
|
||||
// Configure sharp for better memory management with large batches
|
||||
sharp.cache(false); // Disable cache to prevent memory buildup
|
||||
sharp.concurrency(2); // Limit concurrent operations
|
||||
|
||||
const THUMBNAIL_WIDTH = 300;
|
||||
const getStoragePath = () => process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
|
||||
const getThumbnailPath = () => path.join(getStoragePath(), 'thumbnails');
|
||||
@@ -15,16 +19,29 @@ async function generateThumbnail(imagePath) {
|
||||
// Ensure thumbnail directory exists
|
||||
await fs.mkdir(thumbnailDir, { recursive: true });
|
||||
|
||||
// Generate thumbnail
|
||||
await sharp(imagePath)
|
||||
.resize(THUMBNAIL_WIDTH, null, {
|
||||
withoutEnlargement: true,
|
||||
fit: 'inside'
|
||||
try {
|
||||
// Generate thumbnail with memory-efficient settings
|
||||
await sharp(imagePath, {
|
||||
limitInputPixels: 268402689, // ~16k x 16k max
|
||||
sequentialRead: true // More memory efficient for large images
|
||||
})
|
||||
.jpeg({ quality: 80 })
|
||||
.toFile(thumbnailPath);
|
||||
|
||||
return path.relative(getStoragePath(), thumbnailPath);
|
||||
.resize(THUMBNAIL_WIDTH, null, {
|
||||
withoutEnlargement: true,
|
||||
fit: 'inside'
|
||||
})
|
||||
.jpeg({
|
||||
quality: 80,
|
||||
progressive: true, // Progressive JPEG for better loading
|
||||
mozjpeg: true // Better compression
|
||||
})
|
||||
.toFile(thumbnailPath);
|
||||
|
||||
return path.relative(getStoragePath(), thumbnailPath);
|
||||
} catch (error) {
|
||||
console.error(`Failed to generate thumbnail for ${filename}:`, error);
|
||||
// Return null if thumbnail generation fails, don't fail the whole upload
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { generateThumbnail };
|
||||
|
||||
Reference in New Issue
Block a user