diff --git a/CLAUDE.md b/CLAUDE.md
index 4ead341..ae193dd 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -117,6 +117,7 @@ Background services run as separate processes:
- **archiveService**: Creates ZIP archives of expired events
- **expirationChecker**: Cron job for expiration warnings
- **fileWatcher**: Monitors for new photo uploads
+- **backupService**: Scheduled backups with checksum-based change detection
### API Structure
- `/api/admin/*` - Admin panel endpoints (requires adminAuth)
@@ -294,6 +295,42 @@ const { theme, setTheme, setThemeByName } = useTheme();
--border-radius: 0.5rem;
```
+## Backup Service
+
+### Overview
+The backup service provides automated, scheduled backups of all photo data with checksum-based change detection to minimize transfer overhead.
+
+### Features
+- **Multiple Destinations**: Local directory, remote server (rsync), S3-compatible storage
+- **Change Detection**: SHA256 checksums track file changes, only modified files are backed up
+- **Scheduled Execution**: Configurable cron-based scheduling (default: 2 AM daily)
+- **Email Notifications**: Alerts on backup failure, optional success notifications
+- **Retention Management**: Automatic cleanup of old backup runs based on retention policy
+- **Progress Tracking**: Database storage of backup history, file states, and statistics
+
+### Configuration
+Backup settings are stored in `app_settings` table with `backup_` prefix:
+- `backup_enabled`: Enable/disable the service
+- `backup_schedule`: Cron expression (e.g., '0 2 * * *')
+- `backup_destination_type`: 'local', 'rsync', or 's3'
+- `backup_retention_days`: How long to keep backup history
+- `backup_include_archived`: Whether to backup archived events
+- `backup_exclude_patterns`: File patterns to exclude
+
+### API Endpoints
+- `GET /api/admin/backup/config` - Get current configuration
+- `PUT /api/admin/backup/config` - Update configuration
+- `GET /api/admin/backup/status` - Get backup status and history
+- `POST /api/admin/backup/run` - Trigger manual backup
+- `POST /api/admin/backup/test-connection` - Test destination connectivity
+
+### Testing
+Run backup service test: `npm run test-backup`
+
+### Database Tables
+- `backup_runs`: Tracks each backup execution with statistics
+- `backup_file_states`: Stores file checksums for change detection
+
## Success Metrics (from PRD)
- Time to generate gallery: <2 minutes
- Guest satisfaction: >90%
diff --git a/README.md b/README.md
index c94a63c..ee541f3 100644
--- a/README.md
+++ b/README.md
@@ -159,6 +159,20 @@ Organize and manage your photo galleries with intuitive event management tools.
+## 🗺️ Roadmap
+
+We're constantly improving PicPeak and welcome contributions from our community! If you have ideas for new features or want to help implement existing ones, please open an issue or submit a pull request. Your contributions help make PicPeak better for everyone.
+
+| Feature | Description | Priority | Status |
+|---------|-------------|----------|---------|
+| **Backup & Restore** | Comprehensive backup system with S3/MinIO support, automated scheduling, and safe restore functionality | High | ✅ Implemented |
+| **Gallery Templates** | Additional gallery layouts and themes (masonry, slideshow, story-style) for different event types | Medium | 🔄 Open |
+| **Face Recognition** | AI-powered face detection to help guests find their photos and create automatic person-based albums | Low | 🔄 Open |
+| **Gallery Feedback** | Allow guests to like, rate, and comment on photos with admin notifications and moderation | Medium | 🔄 Open |
+| **Video Support** | Upload and display videos alongside photos in galleries with streaming support | Low | 🔄 Open |
+
+**Status Legend:** ✅ Implemented | 🚧 In Progress | 🔄 Open | 📋 Planned
+
## 🙏 Acknowledgments
PicPeak is inspired by the best features of commercial platforms while remaining completely open source. Special thanks to all contributors who make this project possible.
diff --git a/backend/__tests__/README.md b/backend/__tests__/README.md
new file mode 100644
index 0000000..c4bda50
--- /dev/null
+++ b/backend/__tests__/README.md
@@ -0,0 +1,229 @@
+# Enhanced Backup System Test Suite
+
+This directory contains comprehensive tests for the enhanced backup system with S3 support.
+
+## Test Structure
+
+### Unit Tests
+- `services/backupService.enhanced.test.js` - Unit tests for the enhanced backup service
+ - Configuration management
+ - S3 backup functionality
+ - Manifest generation
+ - Error handling and recovery
+ - Backward compatibility (local and rsync)
+ - Service lifecycle management
+
+### Integration Tests
+- `integration/backup-s3.test.js` - Integration tests for S3 backups
+ - Real S3/MinIO connection tests
+ - Full backup process with actual files
+ - Incremental backup verification
+ - Manifest storage and retrieval
+ - Error recovery scenarios
+
+### Manual Integration Test Script
+- `../scripts/test-backup-integration.js` - Comprehensive manual testing script
+ - Can test against MinIO, AWS S3, or any S3-compatible service
+ - Tests all backup types (S3, local, rsync)
+ - Performance testing with large files
+ - Detailed progress reporting
+
+## Running Tests
+
+### Prerequisites
+
+1. **For Unit Tests**: No special setup required, all dependencies are mocked.
+
+2. **For Integration Tests**: Requires a running S3-compatible service (MinIO recommended)
+ ```bash
+ # Start MinIO using Docker
+ docker run -d \
+ -p 9000:9000 \
+ -p 9001:9001 \
+ --name minio-test \
+ -e MINIO_ROOT_USER=minioadmin \
+ -e MINIO_ROOT_PASSWORD=minioadmin \
+ minio/minio server /data --console-address ":9001"
+ ```
+
+3. **Environment Variables** (for integration tests):
+ ```bash
+ # Optional - defaults work with local MinIO
+ export TEST_S3_ENDPOINT=http://localhost:9000
+ export TEST_S3_ACCESS_KEY=minioadmin
+ export TEST_S3_SECRET_KEY=minioadmin
+
+ # Skip S3 tests if no S3 service available
+ export SKIP_S3_TESTS=true
+ ```
+
+### Running Unit Tests
+
+```bash
+# Run all backup service tests
+npm test -- __tests__/services/backupService.enhanced.test.js
+
+# Run specific test suite
+npm test -- __tests__/services/backupService.enhanced.test.js -t "S3 Backup Functionality"
+
+# Run with coverage
+npm test -- --coverage __tests__/services/backupService.enhanced.test.js
+```
+
+### Running Integration Tests
+
+```bash
+# Ensure MinIO is running first!
+
+# Run S3 integration tests
+npm test -- __tests__/integration/backup-s3.test.js
+
+# Run with verbose output
+npm test -- __tests__/integration/backup-s3.test.js --verbose
+
+# Skip S3 tests if needed
+SKIP_S3_TESTS=true npm test -- __tests__/integration/backup-s3.test.js
+```
+
+### Running Manual Integration Tests
+
+```bash
+# Test with local MinIO (default)
+node scripts/test-backup-integration.js
+
+# Test with AWS S3
+node scripts/test-backup-integration.js \
+ --endpoint https://s3.amazonaws.com \
+ --access-key YOUR_ACCESS_KEY \
+ --secret-key YOUR_SECRET_KEY \
+ --bucket your-test-bucket
+
+# Test local backup
+node scripts/test-backup-integration.js --type local
+
+# Test with cleanup after completion
+node scripts/test-backup-integration.js --cleanup
+
+# Verbose output
+node scripts/test-backup-integration.js --verbose
+```
+
+## Test Coverage
+
+The test suite covers:
+
+### Configuration
+- ✅ Database configuration retrieval
+- ✅ JSON parsing and error handling
+- ✅ Configuration validation
+- ✅ Required field validation
+
+### S3 Functionality
+- ✅ S3 client initialization
+- ✅ Connection testing
+- ✅ File upload with progress tracking
+- ✅ Large file handling (multipart upload)
+- ✅ Metadata and custom headers
+- ✅ Error handling and retries
+
+### Backup Process
+- ✅ Full backup execution
+- ✅ Incremental backup (changed files only)
+- ✅ File checksum calculation and comparison
+- ✅ Database backup inclusion
+- ✅ Archive inclusion toggle
+- ✅ File size limits
+
+### Manifest Generation
+- ✅ Full manifest generation
+- ✅ Incremental manifest with parent reference
+- ✅ JSON and YAML format support
+- ✅ Manifest validation
+- ✅ S3 manifest storage and retrieval
+- ✅ Checksum verification
+
+### Error Handling
+- ✅ S3 connection failures
+- ✅ File read errors
+- ✅ Individual file failure recovery
+- ✅ Retry logic with exponential backoff
+- ✅ Email notifications on failure
+- ✅ Concurrent backup prevention
+
+### Backward Compatibility
+- ✅ Local directory backup
+- ✅ Rsync backup
+- ✅ Existing manifest format support
+
+### Service Management
+- ✅ Cron job scheduling
+- ✅ Service start/stop
+- ✅ Manual backup triggering
+- ✅ Backup history and status
+
+## Mock Setup
+
+The unit tests use comprehensive mocking:
+
+```javascript
+// Database mocking
+jest.mock('../../src/database/db');
+
+// S3 client mocking
+jest.mock('../../src/services/storage/s3Storage');
+
+// File system mocking
+const mockFs = require('mock-fs');
+
+// Cron job mocking
+jest.mock('node-cron');
+```
+
+## CI/CD Integration
+
+To run tests in CI/CD pipeline:
+
+```yaml
+# Example GitHub Actions
+- name: Run Unit Tests
+ run: npm test -- __tests__/services/backupService.enhanced.test.js
+
+- name: Start MinIO
+ run: |
+ docker run -d \
+ -p 9000:9000 \
+ --name minio-test \
+ -e MINIO_ROOT_USER=minioadmin \
+ -e MINIO_ROOT_PASSWORD=minioadmin \
+ minio/minio server /data
+
+- name: Run Integration Tests
+ run: npm test -- __tests__/integration/backup-s3.test.js
+```
+
+## Debugging Tests
+
+```bash
+# Run tests in debug mode
+node --inspect-brk ./node_modules/.bin/jest __tests__/services/backupService.enhanced.test.js
+
+# Run single test with console output
+npm test -- __tests__/services/backupService.enhanced.test.js -t "should perform S3 backup" --verbose
+```
+
+## Performance Considerations
+
+- Integration tests create real files and S3 objects
+- Each test run creates a unique S3 bucket to avoid conflicts
+- Cleanup is automatic but can be disabled for debugging
+- Large file tests (10MB+) are included but can be slow
+
+## Adding New Tests
+
+When adding new backup features:
+
+1. Add unit tests to `backupService.enhanced.test.js`
+2. Add integration tests to `backup-s3.test.js` if S3-specific
+3. Update manual test script for comprehensive testing
+4. Ensure mocks are properly configured
+5. Document any new environment requirements
\ No newline at end of file
diff --git a/backend/__tests__/integration/backup-s3.test.js b/backend/__tests__/integration/backup-s3.test.js
new file mode 100644
index 0000000..a300c47
--- /dev/null
+++ b/backend/__tests__/integration/backup-s3.test.js
@@ -0,0 +1,506 @@
+const { describe, it, expect, jest, beforeAll, afterAll, beforeEach, afterEach } = require('@jest/globals');
+const { S3Client, CreateBucketCommand, DeleteBucketCommand, ListObjectsV2Command, DeleteObjectsCommand } = require('@aws-sdk/client-s3');
+const path = require('path');
+const fs = require('fs').promises;
+const crypto = require('crypto');
+
+// Load services
+const backupService = require('../../src/services/backupService');
+const S3StorageAdapter = require('../../src/services/storage/s3Storage');
+const { db, initialize: initDb } = require('../../src/database/db');
+const logger = require('../../src/utils/logger');
+
+// Test configuration
+const TEST_CONFIG = {
+ endpoint: process.env.TEST_S3_ENDPOINT || 'http://localhost:9000',
+ accessKeyId: process.env.TEST_S3_ACCESS_KEY || 'minioadmin',
+ secretAccessKey: process.env.TEST_S3_SECRET_KEY || 'minioadmin',
+ bucket: 'test-backup-bucket-' + Date.now(),
+ region: 'us-east-1'
+};
+
+describe('S3 Backup Integration Tests', () => {
+ let s3Client;
+ let testStoragePath;
+ let originalEnv;
+
+ beforeAll(async () => {
+ // Skip if no S3 endpoint configured
+ if (process.env.SKIP_S3_TESTS === 'true') {
+ console.log('Skipping S3 integration tests (SKIP_S3_TESTS=true)');
+ return;
+ }
+
+ // Save original environment
+ originalEnv = { ...process.env };
+
+ // Initialize S3 client for test setup
+ s3Client = new S3Client({
+ endpoint: TEST_CONFIG.endpoint,
+ region: TEST_CONFIG.region,
+ credentials: {
+ accessKeyId: TEST_CONFIG.accessKeyId,
+ secretAccessKey: TEST_CONFIG.secretAccessKey
+ },
+ forcePathStyle: true
+ });
+
+ // Create test bucket
+ try {
+ await s3Client.send(new CreateBucketCommand({ Bucket: TEST_CONFIG.bucket }));
+ console.log(`Created test bucket: ${TEST_CONFIG.bucket}`);
+ } catch (error) {
+ if (error.name !== 'BucketAlreadyOwnedByYou') {
+ console.error('Failed to create test bucket:', error);
+ throw error;
+ }
+ }
+
+ // Initialize database
+ await initDb();
+ await db.migrate.latest();
+
+ // Create test storage directory
+ testStoragePath = path.join(__dirname, '../fixtures/test-storage');
+ await fs.mkdir(testStoragePath, { recursive: true });
+ process.env.STORAGE_PATH = testStoragePath;
+
+ // Set up test data
+ await setupTestData();
+
+ // Mock logger to reduce noise
+ logger.info = jest.fn();
+ logger.debug = jest.fn();
+ logger.warn = jest.fn();
+ logger.error = jest.fn();
+ });
+
+ afterAll(async () => {
+ if (process.env.SKIP_S3_TESTS === 'true') return;
+
+ try {
+ // Clean up S3 bucket
+ await cleanupS3Bucket();
+ await s3Client.send(new DeleteBucketCommand({ Bucket: TEST_CONFIG.bucket }));
+ console.log(`Deleted test bucket: ${TEST_CONFIG.bucket}`);
+ } catch (error) {
+ console.error('Failed to cleanup S3 bucket:', error);
+ }
+
+ // Clean up test storage
+ await fs.rm(testStoragePath, { recursive: true, force: true });
+
+ // Restore environment
+ process.env = originalEnv;
+
+ // Close database
+ await db.destroy();
+ });
+
+ beforeEach(async () => {
+ if (process.env.SKIP_S3_TESTS === 'true') {
+ return;
+ }
+
+ // Clean backup tables
+ await db('backup_runs').del();
+ await db('backup_file_states').del();
+ await db('database_backup_runs').del();
+
+ // Configure S3 backup settings
+ await configureS3Backup();
+ });
+
+ afterEach(async () => {
+ if (process.env.SKIP_S3_TESTS === 'true') return;
+
+ // Clean up S3 objects created during test
+ await cleanupS3Bucket();
+ });
+
+ describe('S3 Connection and Configuration', () => {
+ it('should successfully connect to S3-compatible storage', async () => {
+ if (process.env.SKIP_S3_TESTS === 'true') return;
+
+ const s3Adapter = new S3StorageAdapter({
+ ...TEST_CONFIG,
+ bucket: TEST_CONFIG.bucket,
+ forcePathStyle: true,
+ sslEnabled: false
+ });
+
+ const connected = await s3Adapter.testConnection();
+ expect(connected).toBe(true);
+ });
+
+ it('should validate S3 configuration before backup', async () => {
+ if (process.env.SKIP_S3_TESTS === 'true') return;
+
+ // Remove required configuration
+ await db('app_settings')
+ .where('setting_key', 'backup_s3_secret_key')
+ .del();
+
+ await backupService.runBackup();
+
+ const lastRun = await db('backup_runs')
+ .orderBy('started_at', 'desc')
+ .first();
+
+ expect(lastRun.status).toBe('failed');
+ expect(lastRun.error_message).toContain('S3 backup configuration incomplete');
+ });
+ });
+
+ describe('Full S3 Backup Process', () => {
+ it('should perform complete S3 backup with all file types', async () => {
+ if (process.env.SKIP_S3_TESTS === 'true') return;
+
+ // Run backup
+ await backupService.runBackup();
+
+ // Verify backup run completed
+ const backupRun = await db('backup_runs')
+ .orderBy('started_at', 'desc')
+ .first();
+
+ expect(backupRun.status).toBe('completed');
+ expect(backupRun.files_backed_up).toBeGreaterThan(0);
+ expect(backupRun.total_size_bytes).toBeGreaterThan(0);
+
+ // Verify files in S3
+ const s3Objects = await listS3Objects();
+ expect(s3Objects.length).toBeGreaterThan(0);
+
+ // Check for expected file types
+ const hasPhotos = s3Objects.some(obj => obj.Key.includes('events/active'));
+ const hasThumbnails = s3Objects.some(obj => obj.Key.includes('thumbnails'));
+ const hasManifest = s3Objects.some(obj => obj.Key.includes('backup-manifest'));
+ const hasSummary = s3Objects.some(obj => obj.Key.includes('backup-summary.json'));
+
+ expect(hasPhotos).toBe(true);
+ expect(hasThumbnails).toBe(true);
+ expect(hasManifest).toBe(true);
+ expect(hasSummary).toBe(true);
+ });
+
+ it('should handle large file uploads with multipart', async () => {
+ if (process.env.SKIP_S3_TESTS === 'true') return;
+
+ // Create a large test file (15MB)
+ const largeFilePath = path.join(testStoragePath, 'events/active/large-photo.jpg');
+ const largeFileSize = 15 * 1024 * 1024; // 15MB
+ const largeFileContent = Buffer.alloc(largeFileSize, 'x');
+ await fs.writeFile(largeFilePath, largeFileContent);
+
+ // Run backup
+ await backupService.runBackup();
+
+ // Verify large file was uploaded
+ const s3Objects = await listS3Objects();
+ const largeFileUploaded = s3Objects.some(obj =>
+ obj.Key.includes('large-photo.jpg') && obj.Size === largeFileSize
+ );
+
+ expect(largeFileUploaded).toBe(true);
+ });
+
+ it('should include database backup when available', async () => {
+ if (process.env.SKIP_S3_TESTS === 'true') return;
+
+ // Create a mock database backup
+ const dbBackupPath = path.join(testStoragePath, 'backups/db-backup.sql');
+ await fs.mkdir(path.dirname(dbBackupPath), { recursive: true });
+ await fs.writeFile(dbBackupPath, 'CREATE TABLE test (id INT);');
+
+ // Record database backup
+ await db('database_backup_runs').insert({
+ started_at: new Date(),
+ completed_at: new Date(),
+ status: 'completed',
+ backup_type: 'sqlite',
+ file_path: dbBackupPath,
+ file_size_bytes: 100,
+ checksum: 'test123',
+ statistics: JSON.stringify({ tables: {} }),
+ table_checksums: JSON.stringify({})
+ });
+
+ // Configure to include database
+ await db('app_settings')
+ .where('setting_key', 'backup_include_database')
+ .update({ setting_value: 'true' });
+
+ // Run backup
+ await backupService.runBackup();
+
+ // Verify database backup in S3
+ const s3Objects = await listS3Objects();
+ const hasDbBackup = s3Objects.some(obj => obj.Key.includes('database/db-backup.sql'));
+ expect(hasDbBackup).toBe(true);
+ });
+ });
+
+ describe('Incremental Backup', () => {
+ it('should only upload changed files in incremental backup', async () => {
+ if (process.env.SKIP_S3_TESTS === 'true') return;
+
+ // First backup - full
+ await backupService.runBackup();
+
+ const firstRun = await db('backup_runs')
+ .orderBy('started_at', 'desc')
+ .first();
+
+ const firstObjectCount = (await listS3Objects()).length;
+
+ // Wait a moment to ensure different timestamps
+ await new Promise(resolve => setTimeout(resolve, 100));
+
+ // Modify one file
+ const modifiedFile = path.join(testStoragePath, 'events/active/event1/photo1.jpg');
+ await fs.writeFile(modifiedFile, 'modified content');
+
+ // Second backup - incremental
+ await backupService.runBackup();
+
+ const secondRun = await db('backup_runs')
+ .orderBy('started_at', 'desc')
+ .first();
+
+ expect(secondRun.id).not.toBe(firstRun.id);
+ expect(secondRun.files_backed_up).toBe(1); // Only modified file
+
+ // Check manifest indicates incremental
+ if (secondRun.manifest_path) {
+ const manifest = await backupService.getBackupManifest(secondRun.id);
+ expect(manifest.manifest.incremental).toBeDefined();
+ expect(manifest.manifest.incremental.modified_files_count).toBe(1);
+ }
+ });
+
+ it('should track file states across backups', async () => {
+ if (process.env.SKIP_S3_TESTS === 'true') return;
+
+ await backupService.runBackup();
+
+ // Check file states are recorded
+ const fileStates = await db('backup_file_states').select('*');
+ expect(fileStates.length).toBeGreaterThan(0);
+
+ // Verify checksums are stored
+ const hasChecksums = fileStates.every(state => state.checksum !== null);
+ expect(hasChecksums).toBe(true);
+ });
+ });
+
+ describe('S3 Manifest Storage', () => {
+ it('should upload manifest to S3 and retrieve it', async () => {
+ if (process.env.SKIP_S3_TESTS === 'true') return;
+
+ // Configure YAML manifest format
+ await db('app_settings')
+ .where('setting_key', 'backup_manifest_format')
+ .update({ setting_value: '"yaml"' });
+
+ await backupService.runBackup();
+
+ const backupRun = await db('backup_runs')
+ .orderBy('started_at', 'desc')
+ .first();
+
+ expect(backupRun.manifest_path).toMatch(/^s3:\/\//);
+
+ // Retrieve manifest
+ const { manifest, summary } = await backupService.getBackupManifest(backupRun.id);
+
+ expect(manifest).toBeDefined();
+ expect(manifest.backup.id).toBeDefined();
+ expect(summary).toContain('BACKUP MANIFEST SUMMARY');
+ });
+
+ it('should validate manifest integrity', async () => {
+ if (process.env.SKIP_S3_TESTS === 'true') return;
+
+ await backupService.runBackup();
+
+ const backupRun = await db('backup_runs')
+ .orderBy('started_at', 'desc')
+ .first();
+
+ const validationResult = await backupService.validateBackupManifest(backupRun.manifest_path);
+
+ expect(validationResult.valid).toBe(true);
+ expect(validationResult.manifest).toBeDefined();
+ });
+ });
+
+ describe('Error Recovery', () => {
+ it('should handle S3 connection failures gracefully', async () => {
+ if (process.env.SKIP_S3_TESTS === 'true') return;
+
+ // Configure with invalid endpoint
+ await db('app_settings')
+ .where('setting_key', 'backup_s3_endpoint')
+ .update({ setting_value: '"http://invalid-endpoint:9999"' });
+
+ await backupService.runBackup();
+
+ const backupRun = await db('backup_runs')
+ .orderBy('started_at', 'desc')
+ .first();
+
+ expect(backupRun.status).toBe('failed');
+ expect(backupRun.error_message).toBeDefined();
+ });
+
+ it('should continue backup despite individual file failures', async () => {
+ if (process.env.SKIP_S3_TESTS === 'true') return;
+
+ // Create a file that will be deleted during backup
+ const tempFile = path.join(testStoragePath, 'events/active/temp.jpg');
+ await fs.writeFile(tempFile, 'temporary');
+
+ // Mock file deletion during backup
+ const originalUpload = S3StorageAdapter.prototype.upload;
+ let callCount = 0;
+ S3StorageAdapter.prototype.upload = jest.fn(async function(localPath, s3Key, options) {
+ callCount++;
+ if (callCount === 2) {
+ // Delete the temp file to cause an error
+ await fs.unlink(tempFile).catch(() => {});
+ }
+ return originalUpload.call(this, localPath, s3Key, options);
+ });
+
+ await backupService.runBackup();
+
+ const backupRun = await db('backup_runs')
+ .orderBy('started_at', 'desc')
+ .first();
+
+ // Should complete despite one file error
+ expect(backupRun.status).toBe('completed');
+ expect(backupRun.files_backed_up).toBeGreaterThan(0);
+
+ // Restore original method
+ S3StorageAdapter.prototype.upload = originalUpload;
+ });
+
+ it('should retry failed uploads with exponential backoff', async () => {
+ if (process.env.SKIP_S3_TESTS === 'true') return;
+
+ // Mock S3 upload to fail twice then succeed
+ const originalUpload = S3StorageAdapter.prototype.upload;
+ let attemptCount = 0;
+ S3StorageAdapter.prototype.upload = jest.fn(async function(localPath, s3Key, options) {
+ attemptCount++;
+ if (attemptCount <= 2) {
+ const error = new Error('Network timeout');
+ error.code = 'ETIMEDOUT';
+ throw error;
+ }
+ return originalUpload.call(this, localPath, s3Key, options);
+ });
+
+ await backupService.runBackup();
+
+ const backupRun = await db('backup_runs')
+ .orderBy('started_at', 'desc')
+ .first();
+
+ // Should succeed after retries
+ expect(backupRun.status).toBe('completed');
+ expect(attemptCount).toBeGreaterThan(2);
+
+ // Restore original method
+ S3StorageAdapter.prototype.upload = originalUpload;
+ });
+ });
+
+ // Helper functions
+
+ async function setupTestData() {
+ // Create test directory structure
+ const dirs = [
+ 'events/active/event1',
+ 'events/active/event2',
+ 'events/archived',
+ 'thumbnails',
+ 'uploads'
+ ];
+
+ for (const dir of dirs) {
+ await fs.mkdir(path.join(testStoragePath, dir), { recursive: true });
+ }
+
+ // Create test files
+ const files = [
+ { path: 'events/active/event1/photo1.jpg', content: 'photo1 content' },
+ { path: 'events/active/event1/photo2.jpg', content: 'photo2 content' },
+ { path: 'events/active/event2/photo3.jpg', content: 'photo3 content' },
+ { path: 'events/archived/old-event.zip', content: 'archived content' },
+ { path: 'thumbnails/thumb1.jpg', content: 'thumbnail content' },
+ { path: 'uploads/logo.png', content: 'logo content' }
+ ];
+
+ for (const file of files) {
+ await fs.writeFile(
+ path.join(testStoragePath, file.path),
+ file.content
+ );
+ }
+ }
+
+ async function configureS3Backup() {
+ const settings = [
+ { setting_key: 'backup_enabled', setting_value: 'true' },
+ { setting_key: 'backup_destination_type', setting_value: '"s3"' },
+ { setting_key: 'backup_s3_bucket', setting_value: `"${TEST_CONFIG.bucket}"` },
+ { setting_key: 'backup_s3_region', setting_value: `"${TEST_CONFIG.region}"` },
+ { setting_key: 'backup_s3_endpoint', setting_value: `"${TEST_CONFIG.endpoint}"` },
+ { setting_key: 'backup_s3_access_key', setting_value: `"${TEST_CONFIG.accessKeyId}"` },
+ { setting_key: 'backup_s3_secret_key', setting_value: `"${TEST_CONFIG.secretAccessKey}"` },
+ { setting_key: 'backup_s3_force_path_style', setting_value: 'true' },
+ { setting_key: 'backup_s3_ssl_enabled', setting_value: 'false' },
+ { setting_key: 'backup_include_archived', setting_value: 'true' },
+ { setting_key: 'backup_incremental', setting_value: 'true' },
+ { setting_key: 'backup_max_file_size_mb', setting_value: '100' }
+ ];
+
+ for (const setting of settings) {
+ await db('app_settings')
+ .insert({
+ setting_type: 'backup',
+ ...setting,
+ created_at: new Date(),
+ updated_at: new Date()
+ })
+ .onConflict(['setting_type', 'setting_key'])
+ .merge();
+ }
+ }
+
+ async function listS3Objects() {
+ const response = await s3Client.send(new ListObjectsV2Command({
+ Bucket: TEST_CONFIG.bucket
+ }));
+ return response.Contents || [];
+ }
+
+ async function cleanupS3Bucket() {
+ try {
+ const objects = await listS3Objects();
+ if (objects.length > 0) {
+ await s3Client.send(new DeleteObjectsCommand({
+ Bucket: TEST_CONFIG.bucket,
+ Delete: {
+ Objects: objects.map(obj => ({ Key: obj.Key }))
+ }
+ }));
+ }
+ } catch (error) {
+ console.error('Failed to cleanup S3 objects:', error);
+ }
+ }
+});
\ No newline at end of file
diff --git a/backend/__tests__/services/backupService.enhanced.test.js b/backend/__tests__/services/backupService.enhanced.test.js
new file mode 100644
index 0000000..e680e71
--- /dev/null
+++ b/backend/__tests__/services/backupService.enhanced.test.js
@@ -0,0 +1,751 @@
+const { describe, it, expect, jest, beforeEach, afterEach } = require('@jest/globals');
+const mockFs = require('mock-fs');
+const path = require('path');
+const crypto = require('crypto');
+const { EventEmitter } = require('events');
+
+// Mock dependencies before requiring the module
+jest.mock('../../src/database/db');
+jest.mock('../../src/utils/logger');
+jest.mock('../../src/services/emailProcessor');
+jest.mock('node-cron');
+jest.mock('../../src/services/backupManifest');
+jest.mock('../../src/services/storage/s3Storage');
+
+const backupService = require('../../src/services/backupService');
+const { db } = require('../../src/database/db');
+const logger = require('../../src/utils/logger');
+const { queueEmail } = require('../../src/services/emailProcessor');
+const cron = require('node-cron');
+const backupManifest = require('../../src/services/backupManifest');
+const S3StorageAdapter = require('../../src/services/storage/s3Storage');
+
+describe('Enhanced Backup Service Tests', () => {
+ let mockDb;
+ let mockS3Client;
+ let mockCronJob;
+
+ beforeEach(() => {
+ // Reset all mocks
+ jest.clearAllMocks();
+
+ // Mock database
+ mockDb = {
+ select: jest.fn().mockReturnThis(),
+ where: jest.fn().mockReturnThis(),
+ orderBy: jest.fn().mockReturnThis(),
+ limit: jest.fn().mockReturnThis(),
+ first: jest.fn(),
+ insert: jest.fn(),
+ update: jest.fn(),
+ delete: jest.fn()
+ };
+ db.mockReturnValue(mockDb);
+
+ // Mock cron job
+ mockCronJob = {
+ stop: jest.fn()
+ };
+ cron.schedule.mockReturnValue(mockCronJob);
+
+ // Mock S3 client
+ mockS3Client = {
+ testConnection: jest.fn().mockResolvedValue(true),
+ upload: jest.fn().mockResolvedValue({ Location: 's3://bucket/key' }),
+ uploadStream: jest.fn().mockResolvedValue({ Location: 's3://bucket/key' }),
+ download: jest.fn().mockResolvedValue(),
+ exists: jest.fn().mockResolvedValue(false),
+ delete: jest.fn().mockResolvedValue(),
+ list: jest.fn().mockResolvedValue({ Contents: [] })
+ };
+ S3StorageAdapter.mockImplementation(() => mockS3Client);
+
+ // Mock backup manifest
+ backupManifest.generateManifest = jest.fn().mockResolvedValue({
+ backup: { id: 'test-backup-123' },
+ version: '2.0'
+ });
+ backupManifest.saveManifest = jest.fn().mockResolvedValue('/path/to/manifest.json');
+ backupManifest.loadManifest = jest.fn().mockResolvedValue({});
+ backupManifest.validateManifest = jest.fn();
+ backupManifest.generateSummaryReport = jest.fn().mockReturnValue('Summary report');
+
+ // Mock logger
+ logger.info = jest.fn();
+ logger.error = jest.fn();
+ logger.warn = jest.fn();
+ logger.debug = jest.fn();
+ });
+
+ afterEach(() => {
+ mockFs.restore();
+ });
+
+ describe('getBackupConfig', () => {
+ it('should retrieve and parse backup configuration from database', async () => {
+ const mockSettings = [
+ { setting_key: 'backup_enabled', setting_value: 'true' },
+ { setting_key: 'backup_destination_type', setting_value: '"s3"' },
+ { setting_key: 'backup_s3_bucket', setting_value: '"test-bucket"' },
+ { setting_key: 'backup_retention_days', setting_value: '30' }
+ ];
+
+ mockDb.select.mockResolvedValue(mockSettings);
+
+ const config = await backupService.getBackupConfig();
+
+ expect(config).toEqual({
+ backup_enabled: true,
+ backup_destination_type: 's3',
+ backup_s3_bucket: 'test-bucket',
+ backup_retention_days: 30
+ });
+
+ expect(db).toHaveBeenCalledWith('app_settings');
+ expect(mockDb.where).toHaveBeenCalledWith('setting_type', 'backup');
+ });
+
+ it('should handle JSON parse errors gracefully', async () => {
+ const mockSettings = [
+ { setting_key: 'backup_enabled', setting_value: 'invalid-json' }
+ ];
+
+ mockDb.select.mockResolvedValue(mockSettings);
+
+ const config = await backupService.getBackupConfig();
+
+ expect(config).toEqual({
+ backup_enabled: 'invalid-json'
+ });
+ });
+
+ it('should return null on database error', async () => {
+ mockDb.select.mockRejectedValue(new Error('Database error'));
+
+ const config = await backupService.getBackupConfig();
+
+ expect(config).toBeNull();
+ expect(logger.error).toHaveBeenCalled();
+ });
+ });
+
+ describe('S3 Backup Functionality', () => {
+ beforeEach(() => {
+ // Mock file system
+ mockFs({
+ '/storage/events/active/event1': {
+ 'photo1.jpg': Buffer.from('photo1 content'),
+ 'photo2.jpg': Buffer.from('photo2 content')
+ },
+ '/storage/events/archived/event2.zip': Buffer.from('archived content'),
+ '/storage/thumbnails': {
+ 'thumb1.jpg': Buffer.from('thumb1 content')
+ },
+ '/storage/uploads': {
+ 'logo.png': Buffer.from('logo content')
+ }
+ });
+
+ process.env.STORAGE_PATH = '/storage';
+ });
+
+ it('should perform S3 backup with correct configuration', async () => {
+ const config = {
+ backup_enabled: true,
+ backup_destination_type: 's3',
+ backup_s3_bucket: 'test-bucket',
+ backup_s3_region: 'us-east-1',
+ backup_s3_endpoint: 'https://s3.amazonaws.com',
+ backup_s3_access_key: 'test-key',
+ backup_s3_secret_key: 'test-secret',
+ backup_include_archived: true,
+ backup_max_file_size_mb: 100
+ };
+
+ mockDb.select.mockResolvedValue([]);
+ mockDb.where.mockReturnThis();
+ mockDb.first.mockResolvedValue(null);
+ mockDb.insert.mockResolvedValue([1]);
+
+ jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
+ jest.spyOn(backupService, 'getDatabaseBackupInfo').mockResolvedValue({
+ type: 'sqlite',
+ backupFile: null,
+ hasChanged: true
+ });
+
+ await backupService.runBackup();
+
+ expect(S3StorageAdapter).toHaveBeenCalledWith({
+ bucket: 'test-bucket',
+ region: 'us-east-1',
+ endpoint: 'https://s3.amazonaws.com',
+ accessKeyId: 'test-key',
+ secretAccessKey: 'test-secret',
+ forcePathStyle: false,
+ sslEnabled: true,
+ maxRetries: 3,
+ retryDelay: 1000
+ });
+
+ expect(mockS3Client.testConnection).toHaveBeenCalled();
+ expect(mockS3Client.upload).toHaveBeenCalled();
+ });
+
+ it('should handle S3 upload failures gracefully', async () => {
+ const config = {
+ backup_enabled: true,
+ backup_destination_type: 's3',
+ backup_s3_bucket: 'test-bucket',
+ backup_s3_access_key: 'test-key',
+ backup_s3_secret_key: 'test-secret'
+ };
+
+ mockDb.select.mockResolvedValue([]);
+ mockDb.insert.mockResolvedValue([1]);
+ mockDb.first.mockResolvedValue(null);
+
+ jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
+
+ mockS3Client.testConnection.mockRejectedValue(new Error('Connection failed'));
+
+ await backupService.runBackup();
+
+ expect(logger.error).toHaveBeenCalledWith('S3 backup failed:', expect.any(Error));
+ expect(mockDb.update).toHaveBeenCalledWith(expect.objectContaining({
+ status: 'failed',
+ error_message: expect.stringContaining('Connection failed')
+ }));
+ });
+
+ it('should skip unchanged files in incremental backup', async () => {
+ const config = {
+ backup_enabled: true,
+ backup_destination_type: 's3',
+ backup_s3_bucket: 'test-bucket',
+ backup_s3_access_key: 'test-key',
+ backup_s3_secret_key: 'test-secret',
+ backup_incremental: true
+ };
+
+ // Mock existing file state
+ mockDb.first.mockImplementation((query) => {
+ if (query === undefined) {
+ return Promise.resolve({
+ file_path: 'events/active/event1/photo1.jpg',
+ checksum: crypto.createHash('sha256').update('photo1 content').digest('hex')
+ });
+ }
+ return Promise.resolve(null);
+ });
+
+ mockDb.select.mockResolvedValue([]);
+ mockDb.insert.mockResolvedValue([1]);
+
+ jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
+
+ await backupService.runBackup();
+
+ // Should skip unchanged file
+ const uploadCalls = mockS3Client.upload.mock.calls;
+ const photo1Uploaded = uploadCalls.some(call =>
+ call[1].includes('photo1.jpg')
+ );
+ expect(photo1Uploaded).toBe(false);
+ });
+
+ it('should include database backup when configured', async () => {
+ const config = {
+ backup_enabled: true,
+ backup_destination_type: 's3',
+ backup_s3_bucket: 'test-bucket',
+ backup_s3_access_key: 'test-key',
+ backup_s3_secret_key: 'test-secret',
+ backup_include_database: true
+ };
+
+ mockDb.select.mockResolvedValue([]);
+ mockDb.insert.mockResolvedValue([1]);
+
+ jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
+ jest.spyOn(backupService, 'getDatabaseBackupInfo').mockResolvedValue({
+ type: 'sqlite',
+ backupFile: '/backup/db-backup.sql',
+ size: 1024000,
+ checksum: 'abc123',
+ hasChanged: false
+ });
+
+ // Mock database backup file
+ mockFs({
+ '/storage/events/active': {},
+ '/backup/db-backup.sql': Buffer.from('database backup content')
+ });
+
+ await backupService.runBackup();
+
+ // Verify database backup was uploaded
+ const uploadCalls = mockS3Client.upload.mock.calls;
+ const dbBackupUploaded = uploadCalls.some(call =>
+ call[1].includes('database/db-backup.sql')
+ );
+ expect(dbBackupUploaded).toBe(true);
+ });
+
+ it('should validate required S3 configuration', async () => {
+ const config = {
+ backup_enabled: true,
+ backup_destination_type: 's3',
+ backup_s3_bucket: 'test-bucket'
+ // Missing access key and secret key
+ };
+
+ mockDb.select.mockResolvedValue([]);
+ mockDb.insert.mockResolvedValue([1]);
+
+ jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
+
+ await backupService.runBackup();
+
+ expect(logger.error).toHaveBeenCalledWith(
+ 'S3 backup failed:',
+ expect.objectContaining({
+ message: expect.stringContaining('S3 backup configuration incomplete')
+ })
+ );
+ });
+ });
+
+ describe('Manifest Generation', () => {
+ it('should generate and save manifest for successful backup', async () => {
+ const config = {
+ backup_enabled: true,
+ backup_destination_type: 'local',
+ backup_destination_path: '/backup',
+ backup_manifest_format: 'json'
+ };
+
+ mockDb.select.mockResolvedValue([]);
+ mockDb.insert.mockResolvedValue([1]);
+ mockDb.first.mockResolvedValue(null);
+
+ jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
+
+ mockFs({
+ '/storage/events/active/event1': {
+ 'photo1.jpg': Buffer.from('photo1 content')
+ },
+ '/backup': {}
+ });
+
+ await backupService.runBackup();
+
+ expect(backupManifest.generateManifest).toHaveBeenCalledWith(
+ expect.objectContaining({
+ backupType: 'full',
+ backupPath: '/backup',
+ format: 'json'
+ })
+ );
+
+ expect(backupManifest.saveManifest).toHaveBeenCalled();
+ });
+
+ it('should generate incremental manifest when parent exists', async () => {
+ const config = {
+ backup_enabled: true,
+ backup_destination_type: 'local',
+ backup_destination_path: '/backup'
+ };
+
+ const lastBackup = {
+ id: 1,
+ manifest_path: '/backup/manifests/previous.json',
+ manifest_id: 'previous-backup-123'
+ };
+
+ mockDb.select.mockResolvedValue([]);
+ mockDb.insert.mockResolvedValue([2]);
+ mockDb.first.mockImplementation(() => Promise.resolve(lastBackup));
+ mockDb.orderBy.mockReturnThis();
+ mockDb.where.mockReturnThis();
+ mockDb.whereNot = jest.fn().mockReturnThis();
+
+ jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
+
+ mockFs({
+ '/storage/events/active': {},
+ '/backup': {}
+ });
+
+ await backupService.runBackup();
+
+ expect(backupManifest.loadManifest).toHaveBeenCalledWith('/backup/manifests/previous.json');
+ expect(backupManifest.generateIncrementalManifest).toHaveBeenCalled();
+ });
+
+ it('should upload manifest to S3 for S3 backups', async () => {
+ const config = {
+ backup_enabled: true,
+ backup_destination_type: 's3',
+ backup_s3_bucket: 'test-bucket',
+ backup_s3_access_key: 'test-key',
+ backup_s3_secret_key: 'test-secret',
+ backup_manifest_format: 'yaml'
+ };
+
+ mockDb.select.mockResolvedValue([]);
+ mockDb.insert.mockResolvedValue([1]);
+ mockDb.first.mockResolvedValue(null);
+
+ jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
+
+ const manifest = {
+ backup: { id: 'backup-123' },
+ version: '2.0'
+ };
+ backupManifest.generateManifest.mockResolvedValue(manifest);
+
+ mockFs({
+ '/storage/events/active': {},
+ '/storage/temp': {}
+ });
+
+ await backupService.runBackup();
+
+ // Verify manifest was uploaded to S3
+ const uploadCalls = mockS3Client.upload.mock.calls;
+ const manifestUploaded = uploadCalls.some(call =>
+ call[1].includes('manifests/backup-manifest-backup-123.yaml')
+ );
+ expect(manifestUploaded).toBe(true);
+ });
+ });
+
+ describe('Backward Compatibility', () => {
+ it('should support local backup destination', async () => {
+ const config = {
+ backup_enabled: true,
+ backup_destination_type: 'local',
+ backup_destination_path: '/backup/local'
+ };
+
+ mockDb.select.mockResolvedValue([]);
+ mockDb.insert.mockResolvedValue([1]);
+ mockDb.first.mockResolvedValue(null);
+
+ jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
+
+ mockFs({
+ '/storage/events/active/event1': {
+ 'photo1.jpg': Buffer.from('photo1 content')
+ },
+ '/backup/local': {}
+ });
+
+ await backupService.runBackup();
+
+ // Verify files were copied to local destination
+ const fs = require('fs');
+ const destPath = '/backup/local/events/active/event1/photo1.jpg';
+ expect(fs.existsSync(destPath)).toBe(true);
+ });
+
+ it('should support rsync backup destination', async () => {
+ const config = {
+ backup_enabled: true,
+ backup_destination_type: 'rsync',
+ backup_rsync_host: 'backup.example.com',
+ backup_rsync_user: 'backup',
+ backup_rsync_path: '/remote/backup'
+ };
+
+ mockDb.select.mockResolvedValue([]);
+ mockDb.insert.mockResolvedValue([1]);
+ mockDb.first.mockResolvedValue(null);
+
+ jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
+
+ // Mock exec for rsync
+ const { exec } = require('child_process');
+ const mockExec = jest.fn((cmd, callback) => {
+ callback(null, { stdout: 'Number of files transferred: 1\nTotal file size: 1024 bytes' });
+ });
+ exec.mockImplementation(mockExec);
+
+ mockFs({
+ '/storage/events/active': {}
+ });
+
+ await backupService.runBackup();
+
+ expect(mockExec).toHaveBeenCalledWith(
+ expect.stringContaining('rsync'),
+ expect.any(Function)
+ );
+ });
+ });
+
+ describe('Error Handling and Recovery', () => {
+ it('should handle file read errors gracefully', async () => {
+ const config = {
+ backup_enabled: true,
+ backup_destination_type: 's3',
+ backup_s3_bucket: 'test-bucket',
+ backup_s3_access_key: 'test-key',
+ backup_s3_secret_key: 'test-secret'
+ };
+
+ mockDb.select.mockResolvedValue([]);
+ mockDb.insert.mockResolvedValue([1]);
+ mockDb.first.mockResolvedValue(null);
+
+ jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
+
+ // Mock file that throws error on read
+ const fs = require('fs');
+ const originalCreateReadStream = fs.createReadStream;
+ fs.createReadStream = jest.fn((path) => {
+ if (path.includes('error.jpg')) {
+ const stream = new EventEmitter();
+ process.nextTick(() => stream.emit('error', new Error('File read error')));
+ return stream;
+ }
+ return originalCreateReadStream(path);
+ });
+
+ mockFs({
+ '/storage/events/active': {
+ 'error.jpg': Buffer.from('content'),
+ 'good.jpg': Buffer.from('content')
+ }
+ });
+
+ await backupService.runBackup();
+
+ // Should continue with other files despite error
+ expect(logger.error).toHaveBeenCalledWith(
+ expect.stringContaining('Failed to backup file'),
+ expect.any(Error)
+ );
+
+ fs.createReadStream = originalCreateReadStream;
+ });
+
+ it('should send failure email on backup error', async () => {
+ const config = {
+ backup_enabled: true,
+ backup_destination_type: 's3',
+ backup_s3_bucket: 'test-bucket',
+ backup_email_on_failure: true
+ };
+
+ const admins = [
+ { email: 'admin1@example.com', is_active: true },
+ { email: 'admin2@example.com', is_active: true }
+ ];
+
+ mockDb.select.mockResolvedValue([]);
+ mockDb.insert.mockResolvedValue([1]);
+ mockDb.where.mockReturnThis();
+
+ jest.spyOn(backupService, 'getBackupConfig')
+ .mockResolvedValueOnce(config)
+ .mockResolvedValueOnce(config);
+
+ // Force an error
+ jest.spyOn(backupService, 'getFilesToBackup').mockRejectedValue(new Error('Storage error'));
+
+ // Mock admin users query
+ db.mockImplementation((table) => {
+ if (table === 'admin_users') {
+ return {
+ where: jest.fn().mockResolvedValue(admins)
+ };
+ }
+ return mockDb;
+ });
+
+ await backupService.runBackup();
+
+ expect(queueEmail).toHaveBeenCalledTimes(2);
+ expect(queueEmail).toHaveBeenCalledWith(
+ null,
+ 'admin1@example.com',
+ 'backup_failed',
+ expect.objectContaining({
+ error_message: 'Storage error'
+ })
+ );
+ });
+
+ it('should handle concurrent backup attempts', async () => {
+ const config = {
+ backup_enabled: true,
+ backup_destination_type: 'local',
+ backup_destination_path: '/backup'
+ };
+
+ jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
+
+ mockFs({
+ '/storage/events/active': {},
+ '/backup': {}
+ });
+
+ // Start two backups concurrently
+ const backup1 = backupService.runBackup();
+ const backup2 = backupService.runBackup();
+
+ await Promise.all([backup1, backup2]);
+
+ // Second backup should be skipped
+ expect(logger.warn).toHaveBeenCalledWith('Backup already running, skipping');
+ });
+ });
+
+ describe('Service Lifecycle', () => {
+ it('should start backup service with cron schedule', async () => {
+ const config = {
+ backup_enabled: true,
+ backup_schedule: '0 3 * * *' // 3 AM daily
+ };
+
+ mockDb.select.mockResolvedValue(
+ Object.entries(config).map(([key, value]) => ({
+ setting_key: key,
+ setting_value: value.toString()
+ }))
+ );
+
+ await backupService.startBackupService();
+
+ expect(cron.schedule).toHaveBeenCalledWith('0 3 * * *', expect.any(Function));
+ expect(logger.info).toHaveBeenCalledWith('Backup service started with schedule: 0 3 * * *');
+ });
+
+ it('should stop existing job when restarting service', async () => {
+ const config = {
+ backup_enabled: true,
+ backup_schedule: '0 2 * * *'
+ };
+
+ mockDb.select.mockResolvedValue(
+ Object.entries(config).map(([key, value]) => ({
+ setting_key: key,
+ setting_value: value.toString()
+ }))
+ );
+
+ // Start service twice
+ await backupService.startBackupService();
+ await backupService.startBackupService();
+
+ expect(mockCronJob.stop).toHaveBeenCalled();
+ });
+
+ it('should not start service when backup is disabled', async () => {
+ const config = {
+ backup_enabled: false
+ };
+
+ mockDb.select.mockResolvedValue([
+ { setting_key: 'backup_enabled', setting_value: 'false' }
+ ]);
+
+ await backupService.startBackupService();
+
+ expect(cron.schedule).not.toHaveBeenCalled();
+ expect(logger.info).toHaveBeenCalledWith('Backup service is disabled');
+ });
+ });
+
+ describe('Backup Status and History', () => {
+ it('should return backup status with recent runs', async () => {
+ const recentRuns = [
+ {
+ id: 1,
+ started_at: new Date(),
+ completed_at: new Date(),
+ status: 'completed',
+ files_backed_up: 100,
+ total_size_bytes: 1024000,
+ manifest_path: '/backup/manifest.json'
+ }
+ ];
+
+ mockDb.limit.mockResolvedValue(recentRuns);
+
+ backupManifest.validateManifest.mockImplementation(() => true);
+
+ const status = await backupService.getBackupStatus();
+
+ expect(status).toEqual({
+ isRunning: false,
+ isHealthy: true,
+ lastRun: expect.objectContaining({
+ ...recentRuns[0],
+ manifestValid: true
+ }),
+ recentRuns: recentRuns,
+ nextScheduledRun: expect.any(String)
+ });
+ });
+
+ it('should clean up old backup runs', async () => {
+ mockDb.delete.mockResolvedValue(5);
+
+ await backupService.cleanupOldBackupRuns(30);
+
+ expect(mockDb.where).toHaveBeenCalledWith('started_at', '<', expect.any(Date));
+ expect(mockDb.delete).toHaveBeenCalled();
+ expect(logger.info).toHaveBeenCalledWith('Cleaned up 5 old backup runs');
+ });
+ });
+
+ describe('getBackupManifest', () => {
+ it('should retrieve manifest from local filesystem', async () => {
+ const backupRun = {
+ id: 1,
+ manifest_path: '/backup/manifests/backup-123.json'
+ };
+
+ mockDb.first.mockResolvedValue(backupRun);
+
+ const manifest = { backup: { id: 'backup-123' } };
+ backupManifest.loadManifest.mockResolvedValue(manifest);
+ backupManifest.generateSummaryReport.mockReturnValue('Summary');
+
+ const result = await backupService.getBackupManifest(1);
+
+ expect(result).toEqual({
+ manifest: manifest,
+ summary: 'Summary'
+ });
+ });
+
+ it('should retrieve manifest from S3', async () => {
+ const backupRun = {
+ id: 1,
+ manifest_path: 's3://test-bucket/backups/manifests/backup-123.json'
+ };
+
+ mockDb.first.mockResolvedValue(backupRun);
+ mockDb.select.mockResolvedValue([
+ { setting_key: 'backup_s3_access_key', setting_value: '"test-key"' },
+ { setting_key: 'backup_s3_secret_key', setting_value: '"test-secret"' }
+ ]);
+
+ const manifest = { backup: { id: 'backup-123' } };
+ backupManifest.loadManifest.mockResolvedValue(manifest);
+
+ await backupService.getBackupManifest(1);
+
+ expect(S3StorageAdapter).toHaveBeenCalled();
+ expect(mockS3Client.download).toHaveBeenCalledWith(
+ 'backups/manifests/backup-123.json',
+ expect.any(String)
+ );
+ });
+ });
+});
\ No newline at end of file
diff --git a/backend/data/photo_sharing.db b/backend/data/photo_sharing.db
index 9e37c51..84e6fe5 100644
Binary files a/backend/data/photo_sharing.db and b/backend/data/photo_sharing.db differ
diff --git a/backend/migrations/029_add_backup_service_tables.js b/backend/migrations/029_add_backup_service_tables.js
new file mode 100644
index 0000000..10d697d
--- /dev/null
+++ b/backend/migrations/029_add_backup_service_tables.js
@@ -0,0 +1,243 @@
+const { db } = require('../src/database/db');
+
+async function up() {
+ console.log('Adding backup service tables and settings...');
+
+ // Create backup_runs table to track backup history
+ const hasBackupRunsTable = await db.schema.hasTable('backup_runs');
+ if (!hasBackupRunsTable) {
+ await db.schema.createTable('backup_runs', (table) => {
+ table.increments('id').primary();
+ table.datetime('started_at').notNullable();
+ table.datetime('completed_at');
+ table.string('status').defaultTo('running'); // running, completed, failed
+ table.string('backup_type'); // full, incremental
+ table.integer('files_backed_up').defaultTo(0);
+ table.bigInteger('total_size_bytes').defaultTo(0);
+ table.integer('duration_seconds');
+ table.text('error_message');
+ table.json('statistics'); // Detailed stats about the backup
+ table.json('file_checksums'); // Store checksums for change detection
+ });
+ }
+
+ // Create backup_file_states table to track individual file states
+ const hasBackupFileStatesTable = await db.schema.hasTable('backup_file_states');
+ if (!hasBackupFileStatesTable) {
+ await db.schema.createTable('backup_file_states', (table) => {
+ table.increments('id').primary();
+ table.string('file_path').notNullable();
+ table.string('checksum').notNullable();
+ table.bigInteger('size_bytes');
+ table.datetime('last_modified');
+ table.datetime('last_backed_up');
+ table.boolean('is_archived').defaultTo(false);
+ table.index(['file_path'], 'idx_backup_file_path');
+ table.index(['checksum'], 'idx_backup_checksum');
+ });
+ }
+
+ // Add backup-related settings to app_settings
+ const backupSettings = [
+ {
+ setting_key: 'backup_enabled',
+ setting_value: JSON.stringify(false),
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_schedule',
+ setting_value: JSON.stringify('0 2 * * *'), // Default: 2 AM daily
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_destination_type',
+ setting_value: JSON.stringify('local'), // local, rsync, s3
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_destination_path',
+ setting_value: JSON.stringify('/backup/picpeak'),
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_rsync_host',
+ setting_value: JSON.stringify(''),
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_rsync_user',
+ setting_value: JSON.stringify(''),
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_rsync_path',
+ setting_value: JSON.stringify(''),
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_rsync_ssh_key',
+ setting_value: JSON.stringify(''),
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_s3_endpoint',
+ setting_value: JSON.stringify(''),
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_s3_bucket',
+ setting_value: JSON.stringify(''),
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_s3_access_key',
+ setting_value: JSON.stringify(''),
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_s3_secret_key',
+ setting_value: JSON.stringify(''),
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_s3_region',
+ setting_value: JSON.stringify('us-east-1'),
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_retention_days',
+ setting_value: JSON.stringify(30),
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_include_archived',
+ setting_value: JSON.stringify(true),
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_compression',
+ setting_value: JSON.stringify(true),
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_email_on_failure',
+ setting_value: JSON.stringify(true),
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_email_on_success',
+ setting_value: JSON.stringify(false),
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_max_file_size_mb',
+ setting_value: JSON.stringify(5000), // Skip files larger than 5GB
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_exclude_patterns',
+ setting_value: JSON.stringify(['*.tmp', '.DS_Store', 'Thumbs.db']),
+ setting_type: 'backup'
+ }
+ ];
+
+ // Insert backup settings if they don't exist
+ for (const setting of backupSettings) {
+ const exists = await db('app_settings')
+ .where('setting_key', setting.setting_key)
+ .first();
+
+ if (!exists) {
+ await db('app_settings').insert(setting);
+ }
+ }
+
+ // Add backup-related email templates
+ const backupEmailTemplates = [
+ {
+ template_key: 'backup_failed',
+ subject_en: 'Backup Failed - Immediate Attention Required',
+ subject_de: 'Backup fehlgeschlagen - Sofortige Aufmerksamkeit erforderlich',
+ body_html_en: `
Backup Failed
+The scheduled backup has failed and requires immediate attention.
+Error Details:
+
+ - Start Time: {{start_time}}
+ - Backup Type: {{backup_type}}
+ - Error: {{error_message}}
+
+Please check the system logs for more details and resolve the issue as soon as possible.
`,
+ body_html_de: `Backup fehlgeschlagen
+Das geplante Backup ist fehlgeschlagen und erfordert sofortige Aufmerksamkeit.
+Fehlerdetails:
+
+ - Startzeit: {{start_time}}
+ - Backup-Typ: {{backup_type}}
+ - Fehler: {{error_message}}
+
+Bitte überprüfen Sie die Systemprotokolle für weitere Details und beheben Sie das Problem so schnell wie möglich.
`,
+ body_text_en: 'Backup Failed\n\nThe scheduled backup has failed and requires immediate attention.\n\nStart Time: {{start_time}}\nBackup Type: {{backup_type}}\nError: {{error_message}}\n\nPlease check the system logs for more details.',
+ body_text_de: 'Backup fehlgeschlagen\n\nDas geplante Backup ist fehlgeschlagen und erfordert sofortige Aufmerksamkeit.\n\nStartzeit: {{start_time}}\nBackup-Typ: {{backup_type}}\nFehler: {{error_message}}\n\nBitte überprüfen Sie die Systemprotokolle für weitere Details.',
+ variables: JSON.stringify(['start_time', 'backup_type', 'error_message'])
+ },
+ {
+ template_key: 'backup_completed',
+ subject_en: 'Backup Completed Successfully',
+ subject_de: 'Backup erfolgreich abgeschlossen',
+ body_html_en: `Backup Completed
+The scheduled backup has been completed successfully.
+Backup Summary:
+
+ - Start Time: {{start_time}}
+ - Duration: {{duration}}
+ - Files Backed Up: {{files_count}}
+ - Total Size: {{total_size}}
+ - Backup Type: {{backup_type}}
+
`,
+ body_html_de: `Backup abgeschlossen
+Das geplante Backup wurde erfolgreich abgeschlossen.
+Backup-Zusammenfassung:
+
+ - Startzeit: {{start_time}}
+ - Dauer: {{duration}}
+ - Gesicherte Dateien: {{files_count}}
+ - Gesamtgröße: {{total_size}}
+ - Backup-Typ: {{backup_type}}
+
`,
+ body_text_en: 'Backup Completed\n\nThe scheduled backup has been completed successfully.\n\nStart Time: {{start_time}}\nDuration: {{duration}}\nFiles Backed Up: {{files_count}}\nTotal Size: {{total_size}}\nBackup Type: {{backup_type}}',
+ body_text_de: 'Backup abgeschlossen\n\nDas geplante Backup wurde erfolgreich abgeschlossen.\n\nStartzeit: {{start_time}}\nDauer: {{duration}}\nGesicherte Dateien: {{files_count}}\nGesamtgröße: {{total_size}}\nBackup-Typ: {{backup_type}}',
+ variables: JSON.stringify(['start_time', 'duration', 'files_count', 'total_size', 'backup_type'])
+ }
+ ];
+
+ // Insert backup email templates if they don't exist
+ for (const template of backupEmailTemplates) {
+ const exists = await db('email_templates')
+ .where('template_key', template.template_key)
+ .first();
+
+ if (!exists) {
+ await db('email_templates').insert(template);
+ }
+ }
+
+ console.log('Backup service tables and settings added successfully');
+}
+
+async function down() {
+ // Remove backup tables
+ await db.schema.dropTableIfExists('backup_file_states');
+ await db.schema.dropTableIfExists('backup_runs');
+
+ // Remove backup settings
+ await db('app_settings')
+ .where('setting_type', 'backup')
+ .delete();
+
+ // Remove backup email templates
+ await db('email_templates')
+ .whereIn('template_key', ['backup_failed', 'backup_completed'])
+ .delete();
+}
+
+module.exports = { up, down };
\ No newline at end of file
diff --git a/backend/migrations/030_add_database_backup_tables.js b/backend/migrations/030_add_database_backup_tables.js
new file mode 100644
index 0000000..cc99100
--- /dev/null
+++ b/backend/migrations/030_add_database_backup_tables.js
@@ -0,0 +1,182 @@
+const { db } = require('../src/database/db');
+
+async function up() {
+ console.log('Adding database backup tables and settings...');
+
+ // Create database_backup_runs table to track database backup history
+ const hasDatabaseBackupRunsTable = await db.schema.hasTable('database_backup_runs');
+ if (!hasDatabaseBackupRunsTable) {
+ await db.schema.createTable('database_backup_runs', (table) => {
+ table.increments('id').primary();
+ table.datetime('started_at').notNullable();
+ table.datetime('completed_at');
+ table.string('status').defaultTo('running'); // running, completed, failed
+ table.string('backup_type'); // sqlite, postgresql
+ table.string('destination_path');
+ table.string('file_path');
+ table.bigInteger('file_size_bytes').defaultTo(0);
+ table.bigInteger('original_size_bytes').defaultTo(0);
+ table.integer('duration_seconds');
+ table.string('checksum'); // SHA256 checksum of backup file
+ table.float('compression_ratio'); // Compression percentage
+ table.json('table_checksums'); // Individual table checksums
+ table.text('error_message');
+ table.json('statistics'); // Detailed stats about the backup
+ table.index(['started_at'], 'idx_db_backup_started');
+ table.index(['status'], 'idx_db_backup_status');
+ });
+ }
+
+ // Add database backup-related settings to app_settings
+ const databaseBackupSettings = [
+ {
+ setting_key: 'database_backup_enabled',
+ setting_value: JSON.stringify(false),
+ setting_type: 'database_backup'
+ },
+ {
+ setting_key: 'database_backup_schedule',
+ setting_value: JSON.stringify('0 3 * * *'), // Default: 3 AM daily
+ setting_type: 'database_backup'
+ },
+ {
+ setting_key: 'database_backup_destination_path',
+ setting_value: JSON.stringify('/backup/database'),
+ setting_type: 'database_backup'
+ },
+ {
+ setting_key: 'database_backup_compress',
+ setting_value: JSON.stringify(true),
+ setting_type: 'database_backup'
+ },
+ {
+ setting_key: 'database_backup_validate_integrity',
+ setting_value: JSON.stringify(true),
+ setting_type: 'database_backup'
+ },
+ {
+ setting_key: 'database_backup_include_checksums',
+ setting_value: JSON.stringify(true),
+ setting_type: 'database_backup'
+ },
+ {
+ setting_key: 'database_backup_retention_days',
+ setting_value: JSON.stringify(30),
+ setting_type: 'database_backup'
+ },
+ {
+ setting_key: 'database_backup_email_on_failure',
+ setting_value: JSON.stringify(true),
+ setting_type: 'database_backup'
+ },
+ {
+ setting_key: 'database_backup_email_on_success',
+ setting_value: JSON.stringify(false),
+ setting_type: 'database_backup'
+ },
+ {
+ setting_key: 'database_backup_max_retries',
+ setting_value: JSON.stringify(3),
+ setting_type: 'database_backup'
+ }
+ ];
+
+ // Insert database backup settings if they don't exist
+ for (const setting of databaseBackupSettings) {
+ const exists = await db('app_settings')
+ .where('setting_key', setting.setting_key)
+ .first();
+
+ if (!exists) {
+ await db('app_settings').insert(setting);
+ }
+ }
+
+ // Add database backup-related email templates
+ const databaseBackupEmailTemplates = [
+ {
+ template_key: 'database_backup_failed',
+ subject_en: 'Database Backup Failed - Critical Alert',
+ subject_de: 'Datenbank-Backup fehlgeschlagen - Kritische Warnung',
+ body_html_en: `Database Backup Failed
+The scheduled database backup has failed and requires immediate attention.
+Error Details:
+
+ - Backup Type: {{backup_type}}
+ - Timestamp: {{timestamp}}
+ - Error: {{error_message}}
+
+This is a critical issue that could affect disaster recovery. Please investigate immediately.
`,
+ body_html_de: `Datenbank-Backup fehlgeschlagen
+Das geplante Datenbank-Backup ist fehlgeschlagen und erfordert sofortige Aufmerksamkeit.
+Fehlerdetails:
+
+ - Backup-Typ: {{backup_type}}
+ - Zeitstempel: {{timestamp}}
+ - Fehler: {{error_message}}
+
+Dies ist ein kritisches Problem, das die Disaster-Recovery beeinträchtigen könnte. Bitte untersuchen Sie es sofort.
`,
+ body_text_en: 'Database Backup Failed\n\nThe scheduled database backup has failed.\n\nBackup Type: {{backup_type}}\nTimestamp: {{timestamp}}\nError: {{error_message}}\n\nThis is critical - please investigate immediately.',
+ body_text_de: 'Datenbank-Backup fehlgeschlagen\n\nDas geplante Datenbank-Backup ist fehlgeschlagen.\n\nBackup-Typ: {{backup_type}}\nZeitstempel: {{timestamp}}\nFehler: {{error_message}}\n\nDies ist kritisch - bitte sofort untersuchen.',
+ variables: JSON.stringify(['backup_type', 'timestamp', 'error_message'])
+ },
+ {
+ template_key: 'database_backup_completed',
+ subject_en: 'Database Backup Completed Successfully',
+ subject_de: 'Datenbank-Backup erfolgreich abgeschlossen',
+ body_html_en: `Database Backup Completed
+The scheduled database backup has been completed successfully.
+Backup Summary:
+
+ - Backup Type: {{backup_type}}
+ - Duration: {{duration}}
+ - File Size: {{file_size}}
+ - Compression Ratio: {{compression_ratio}}
+ - File Path: {{file_path}}
+
`,
+ body_html_de: `Datenbank-Backup abgeschlossen
+Das geplante Datenbank-Backup wurde erfolgreich abgeschlossen.
+Backup-Zusammenfassung:
+
+ - Backup-Typ: {{backup_type}}
+ - Dauer: {{duration}}
+ - Dateigröße: {{file_size}}
+ - Komprimierungsverhältnis: {{compression_ratio}}
+ - Dateipfad: {{file_path}}
+
`,
+ body_text_en: 'Database Backup Completed\n\nThe scheduled database backup has been completed successfully.\n\nBackup Type: {{backup_type}}\nDuration: {{duration}}\nFile Size: {{file_size}}\nCompression Ratio: {{compression_ratio}}\nFile Path: {{file_path}}',
+ body_text_de: 'Datenbank-Backup abgeschlossen\n\nDas geplante Datenbank-Backup wurde erfolgreich abgeschlossen.\n\nBackup-Typ: {{backup_type}}\nDauer: {{duration}}\nDateigröße: {{file_size}}\nKomprimierungsverhältnis: {{compression_ratio}}\nDateipfad: {{file_path}}',
+ variables: JSON.stringify(['backup_type', 'duration', 'file_size', 'compression_ratio', 'file_path'])
+ }
+ ];
+
+ // Insert database backup email templates if they don't exist
+ for (const template of databaseBackupEmailTemplates) {
+ const exists = await db('email_templates')
+ .where('template_key', template.template_key)
+ .first();
+
+ if (!exists) {
+ await db('email_templates').insert(template);
+ }
+ }
+
+ console.log('Database backup tables and settings added successfully');
+}
+
+async function down() {
+ // Remove database backup tables
+ await db.schema.dropTableIfExists('database_backup_runs');
+
+ // Remove database backup settings
+ await db('app_settings')
+ .where('setting_type', 'database_backup')
+ .delete();
+
+ // Remove database backup email templates
+ await db('email_templates')
+ .whereIn('template_key', ['database_backup_failed', 'database_backup_completed'])
+ .delete();
+}
+
+module.exports = { up, down };
\ No newline at end of file
diff --git a/backend/migrations/031_add_backup_manifest_columns.js b/backend/migrations/031_add_backup_manifest_columns.js
new file mode 100644
index 0000000..5f075c4
--- /dev/null
+++ b/backend/migrations/031_add_backup_manifest_columns.js
@@ -0,0 +1,86 @@
+const { db } = require('../src/database/db');
+const logger = require('../src/utils/logger');
+
+async function up() {
+ console.log('Adding backup manifest columns...');
+
+ // Add manifest columns to backup_runs table
+ const hasManifestPath = await db.schema.hasColumn('backup_runs', 'manifest_path');
+ if (!hasManifestPath) {
+ await db.schema.alterTable('backup_runs', (table) => {
+ table.string('manifest_path'); // Path to the manifest file
+ table.string('manifest_id'); // Unique manifest ID
+ table.string('manifest_format').defaultTo('json'); // json or yaml
+ });
+ }
+
+ // Add backup manifest-related settings to app_settings
+ const manifestSettings = [
+ {
+ setting_key: 'backup_manifest_enabled',
+ setting_value: JSON.stringify(true),
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_manifest_format',
+ setting_value: JSON.stringify('json'), // json or yaml
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_manifest_path',
+ setting_value: JSON.stringify('/backup/manifests'),
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_manifest_validate',
+ setting_value: JSON.stringify(true),
+ setting_type: 'backup'
+ },
+ {
+ setting_key: 'backup_manifest_include_checksums',
+ setting_value: JSON.stringify(true),
+ setting_type: 'backup'
+ }
+ ];
+
+ // Insert manifest settings if they don't exist
+ for (const setting of manifestSettings) {
+ const exists = await db('app_settings')
+ .where('setting_key', setting.setting_key)
+ .first();
+
+ if (!exists) {
+ await db('app_settings').insert(setting);
+ }
+ }
+
+ console.log('✓ Backup manifest columns and settings added');
+}
+
+async function down() {
+ // Remove manifest columns from backup_runs table
+ const hasManifestPath = await db.schema.hasColumn('backup_runs', 'manifest_path');
+ if (hasManifestPath) {
+ await db.schema.alterTable('backup_runs', (table) => {
+ table.dropColumn('manifest_path');
+ table.dropColumn('manifest_id');
+ table.dropColumn('manifest_format');
+ });
+ }
+
+ // Remove manifest settings
+ await db('app_settings')
+ .where('setting_type', 'backup')
+ .whereIn('setting_key', [
+ 'backup_manifest_enabled',
+ 'backup_manifest_format',
+ 'backup_manifest_path',
+ 'backup_manifest_validate',
+ 'backup_manifest_include_checksums'
+ ])
+ .delete();
+
+ console.log('✓ Backup manifest columns and settings removed');
+}
+
+module.exports = { up, down };
\ No newline at end of file
diff --git a/backend/migrations/031_enhance_backup_system.js b/backend/migrations/031_enhance_backup_system.js
new file mode 100644
index 0000000..6efa582
--- /dev/null
+++ b/backend/migrations/031_enhance_backup_system.js
@@ -0,0 +1,131 @@
+exports.up = function(knex) {
+ return knex.schema
+ // Add new settings to app_settings table
+ .table('app_settings', table => {
+ // S3 configuration enhancements
+ table.boolean('backup_s3_force_path_style').defaultTo(false).comment('Force path-style S3 URLs (for MinIO/self-hosted)');
+ table.boolean('backup_s3_ssl_enabled').defaultTo(true).comment('Enable SSL/TLS for S3 connections');
+ table.string('backup_s3_prefix', 255).comment('S3 key prefix for organizing backups');
+
+ // Backup features
+ table.boolean('backup_incremental').defaultTo(false).comment('Enable incremental backups');
+ table.boolean('backup_include_database').defaultTo(true).comment('Include database dumps in backups');
+ table.boolean('backup_manifest_enabled').defaultTo(true).comment('Generate backup manifests');
+ table.enum('backup_manifest_format', ['json', 'yaml']).defaultTo('json').comment('Manifest file format');
+ table.boolean('backup_encryption_enabled').defaultTo(false).comment('Enable backup encryption');
+ table.string('backup_database_schedule', 100).comment('Separate cron schedule for database-only backups');
+ })
+
+ // Enhance backup_runs table
+ .table('backup_runs', table => {
+ // Manifest tracking
+ table.string('manifest_path', 500).comment('Path to backup manifest file');
+ table.uuid('manifest_id').comment('Unique identifier for the manifest');
+ table.enum('manifest_format', ['json', 'yaml']).comment('Format of the manifest file');
+
+ // Incremental backup support
+ table.integer('parent_backup_id').unsigned().references('id').inTable('backup_runs').onDelete('SET NULL').comment('Parent backup for incremental backups');
+ table.enum('backup_mode', ['full', 'incremental', 'database']).defaultTo('full').comment('Type of backup performed');
+
+ // Add indexes for better query performance
+ table.index(['backup_mode', 'status'], 'idx_backup_runs_mode_status');
+ table.index(['parent_backup_id'], 'idx_backup_runs_parent');
+ table.index(['created_at', 'backup_mode'], 'idx_backup_runs_created_mode');
+ })
+
+ // Create backup_manifest table for storing detailed manifest metadata
+ .createTable('backup_manifest', table => {
+ table.increments('id').primary();
+ table.integer('backup_run_id').unsigned().notNullable().references('id').inTable('backup_runs').onDelete('CASCADE');
+ table.uuid('manifest_id').notNullable().unique().comment('Unique identifier matching backup_runs.manifest_id');
+ table.string('version', 20).notNullable().defaultTo('1.0.0').comment('Manifest schema version');
+ table.enum('format', ['json', 'yaml']).notNullable().defaultTo('json');
+
+ // Backup metadata
+ table.timestamp('backup_start').notNullable();
+ table.timestamp('backup_end').notNullable();
+ table.bigInteger('total_size').unsigned().comment('Total size of backup in bytes');
+ table.integer('file_count').unsigned().comment('Number of files in backup');
+ table.integer('photo_count').unsigned().comment('Number of photos backed up');
+ table.integer('event_count').unsigned().comment('Number of events backed up');
+
+ // Incremental backup metadata
+ table.boolean('is_incremental').defaultTo(false);
+ table.uuid('parent_manifest_id').comment('Parent manifest ID for incremental backups');
+ table.timestamp('incremental_since').comment('Timestamp for incremental backup baseline');
+
+ // Content checksums
+ table.string('checksum_algorithm', 50).defaultTo('sha256').comment('Algorithm used for checksums');
+ table.text('manifest_checksum').comment('Checksum of the manifest file itself');
+
+ // Storage information
+ table.string('storage_location', 500).comment('Primary storage location (local path or S3 URI)');
+ table.string('storage_provider', 50).comment('Storage provider (local, s3, etc.)');
+
+ // Encryption metadata
+ table.boolean('is_encrypted').defaultTo(false);
+ table.string('encryption_algorithm', 100).comment('Encryption algorithm used');
+ table.string('encryption_key_id', 255).comment('ID of encryption key used');
+
+ // Additional metadata as JSON
+ table.json('metadata').comment('Additional metadata as JSON');
+
+ // Timestamps
+ table.timestamps(true, true);
+
+ // Indexes
+ table.index(['backup_run_id'], 'idx_manifest_backup_run');
+ table.index(['manifest_id'], 'idx_manifest_uuid');
+ table.index(['parent_manifest_id'], 'idx_manifest_parent');
+ table.index(['backup_start', 'backup_end'], 'idx_manifest_time_range');
+ table.index(['is_incremental', 'created_at'], 'idx_manifest_incremental_created');
+ })
+
+ // Add composite indexes for common query patterns
+ .raw(`
+ CREATE INDEX IF NOT EXISTS idx_backup_runs_recent_successful
+ ON backup_runs(created_at DESC)
+ WHERE status = 'completed' AND backup_mode = 'full';
+ `)
+ .raw(`
+ CREATE INDEX IF NOT EXISTS idx_backup_runs_incremental_chain
+ ON backup_runs(parent_backup_id, created_at)
+ WHERE backup_mode = 'incremental';
+ `);
+};
+
+exports.down = function(knex) {
+ return knex.schema
+ // Drop indexes first
+ .raw('DROP INDEX IF EXISTS idx_backup_runs_incremental_chain;')
+ .raw('DROP INDEX IF EXISTS idx_backup_runs_recent_successful;')
+
+ // Drop backup_manifest table
+ .dropTableIfExists('backup_manifest')
+
+ // Remove columns from backup_runs table
+ .table('backup_runs', table => {
+ table.dropIndex(['backup_mode', 'status'], 'idx_backup_runs_mode_status');
+ table.dropIndex(['parent_backup_id'], 'idx_backup_runs_parent');
+ table.dropIndex(['created_at', 'backup_mode'], 'idx_backup_runs_created_mode');
+
+ table.dropColumn('manifest_path');
+ table.dropColumn('manifest_id');
+ table.dropColumn('manifest_format');
+ table.dropColumn('parent_backup_id');
+ table.dropColumn('backup_mode');
+ })
+
+ // Remove columns from app_settings table
+ .table('app_settings', table => {
+ table.dropColumn('backup_s3_force_path_style');
+ table.dropColumn('backup_s3_ssl_enabled');
+ table.dropColumn('backup_s3_prefix');
+ table.dropColumn('backup_incremental');
+ table.dropColumn('backup_include_database');
+ table.dropColumn('backup_manifest_enabled');
+ table.dropColumn('backup_manifest_format');
+ table.dropColumn('backup_encryption_enabled');
+ table.dropColumn('backup_database_schedule');
+ });
+};
\ No newline at end of file
diff --git a/backend/migrations/032_add_restore_runs_table.js b/backend/migrations/032_add_restore_runs_table.js
new file mode 100644
index 0000000..28f115b
--- /dev/null
+++ b/backend/migrations/032_add_restore_runs_table.js
@@ -0,0 +1,249 @@
+const { formatBoolean, parseBoolean } = require('./helpers');
+
+/**
+ * Add restore_runs table for tracking restore operations
+ */
+exports.up = async function(knex) {
+ // Create restore_runs table
+ await knex.schema.createTable('restore_runs', table => {
+ table.increments('id').primary();
+
+ // Timing
+ table.timestamp('started_at').notNullable().defaultTo(knex.fn.now());
+ table.timestamp('completed_at');
+ table.integer('duration_seconds');
+
+ // Status and type
+ table.string('status', 50).notNullable().defaultTo('running');
+ table.string('restore_type', 50).notNullable(); // full, database, files, selective
+
+ // Source information
+ table.string('source', 500).notNullable(); // Backup source path or S3 URL
+ table.string('manifest_path', 500); // Path to manifest file
+
+ // Results
+ table.text('error_message');
+ table.text('statistics'); // JSON object with detailed statistics
+ table.text('restore_log'); // JSON array of log entries
+
+ // Safety backup
+ table.string('pre_restore_backup_path', 500); // Path to pre-restore safety backup
+
+ // Flags
+ table.boolean('is_dry_run').defaultTo(formatBoolean(false));
+ table.boolean('was_rollback_attempted').defaultTo(formatBoolean(false));
+ table.boolean('was_successful').defaultTo(formatBoolean(false));
+
+ // Operator information
+ table.string('operator_type', 50).defaultTo('manual'); // manual, scheduled, api
+ table.integer('operator_user_id').references('id').inTable('admin_users').onDelete('SET NULL');
+ table.string('operator_ip', 50);
+
+ // Metadata
+ table.text('metadata'); // JSON object for additional data
+
+ table.index(['status', 'started_at']);
+ table.index(['restore_type', 'started_at']);
+ });
+
+ // Create restore_file_operations table for tracking individual file operations
+ await knex.schema.createTable('restore_file_operations', table => {
+ table.increments('id').primary();
+
+ table.integer('restore_run_id').notNullable()
+ .references('id').inTable('restore_runs').onDelete('CASCADE');
+
+ table.string('file_path', 500).notNullable();
+ table.string('operation', 50).notNullable(); // restore, skip, error
+ table.string('status', 50).notNullable(); // pending, in_progress, completed, failed
+
+ table.bigInteger('file_size');
+ table.string('checksum', 64);
+ table.boolean('checksum_verified').defaultTo(formatBoolean(false));
+
+ table.text('error_message');
+ table.timestamp('started_at');
+ table.timestamp('completed_at');
+
+ table.index(['restore_run_id', 'status']);
+ table.index(['file_path']);
+ });
+
+ // Create restore_validation_results table
+ await knex.schema.createTable('restore_validation_results', table => {
+ table.increments('id').primary();
+
+ table.integer('restore_run_id').notNullable()
+ .references('id').inTable('restore_runs').onDelete('CASCADE');
+
+ table.string('validation_type', 50).notNullable(); // pre-restore, post-restore
+ table.boolean('is_valid').notNullable();
+
+ table.text('errors'); // JSON array of errors
+ table.text('warnings'); // JSON array of warnings
+ table.text('checksums'); // JSON object with checksum comparisons
+
+ table.timestamp('validated_at').notNullable().defaultTo(knex.fn.now());
+
+ table.index(['restore_run_id', 'validation_type']);
+ });
+
+ // Add restore-related settings to app_settings
+ await knex('app_settings').insert([
+ {
+ setting_key: 'restore_allow_force',
+ setting_value: formatBoolean(false),
+ setting_type: 'restore',
+ description: 'Allow force restore with warnings',
+ created_at: knex.fn.now(),
+ updated_at: knex.fn.now()
+ },
+ {
+ setting_key: 'restore_require_pre_backup',
+ setting_value: formatBoolean(true),
+ setting_type: 'restore',
+ description: 'Require pre-restore backup',
+ created_at: knex.fn.now(),
+ updated_at: knex.fn.now()
+ },
+ {
+ setting_key: 'restore_max_file_size_mb',
+ setting_value: '5000',
+ setting_type: 'restore',
+ description: 'Maximum file size for restore (MB)',
+ created_at: knex.fn.now(),
+ updated_at: knex.fn.now()
+ },
+ {
+ setting_key: 'restore_verify_checksums',
+ setting_value: formatBoolean(true),
+ setting_type: 'restore',
+ description: 'Verify file checksums during restore',
+ created_at: knex.fn.now(),
+ updated_at: knex.fn.now()
+ },
+ {
+ setting_key: 'restore_email_on_completion',
+ setting_value: formatBoolean(true),
+ setting_type: 'restore',
+ description: 'Send email on restore completion',
+ created_at: knex.fn.now(),
+ updated_at: knex.fn.now()
+ },
+ {
+ setting_key: 'restore_retention_days',
+ setting_value: '30',
+ setting_type: 'restore',
+ description: 'Days to retain restore history',
+ created_at: knex.fn.now(),
+ updated_at: knex.fn.now()
+ }
+ ]);
+
+ // Add new email templates for restore notifications
+ const emailTemplates = [
+ {
+ name: 'restore_completed',
+ subject: '✅ Restore Completed Successfully',
+ body: `Restore Operation Completed
+A restore operation has completed successfully.
+
+Details:
+
+ - Restore Type: {{restore_type}}
+ - Duration: {{duration}}
+ - Files Restored: {{files_restored}}
+ - Backup ID: {{backup_id}}
+ - Timestamp: {{timestamp}}
+
+
+Please verify that all systems are functioning correctly after the restore.
`,
+ language: 'en',
+ is_active: formatBoolean(true),
+ created_at: knex.fn.now(),
+ updated_at: knex.fn.now()
+ },
+ {
+ name: 'restore_failed',
+ subject: '❌ Restore Operation Failed',
+ body: `Restore Operation Failed
+A restore operation has failed and requires attention.
+
+Details:
+
+ - Restore Type: {{restore_type}}
+ - Error: {{error_message}}
+ - Timestamp: {{timestamp}}
+
+
+Please check the system logs for more details and take appropriate action.
+
+Important: If a pre-restore backup was created, it may be used for recovery.
`,
+ language: 'en',
+ is_active: formatBoolean(true),
+ created_at: knex.fn.now(),
+ updated_at: knex.fn.now()
+ },
+ {
+ name: 'restore_completed',
+ subject: '✅ Wiederherstellung erfolgreich abgeschlossen',
+ body: `Wiederherstellungsvorgang abgeschlossen
+Ein Wiederherstellungsvorgang wurde erfolgreich abgeschlossen.
+
+Details:
+
+ - Wiederherstellungstyp: {{restore_type}}
+ - Dauer: {{duration}}
+ - Wiederhergestellte Dateien: {{files_restored}}
+ - Backup-ID: {{backup_id}}
+ - Zeitstempel: {{timestamp}}
+
+
+Bitte überprüfen Sie, ob alle Systeme nach der Wiederherstellung ordnungsgemäß funktionieren.
`,
+ language: 'de',
+ is_active: formatBoolean(true),
+ created_at: knex.fn.now(),
+ updated_at: knex.fn.now()
+ },
+ {
+ name: 'restore_failed',
+ subject: '❌ Wiederherstellungsvorgang fehlgeschlagen',
+ body: `Wiederherstellungsvorgang fehlgeschlagen
+Ein Wiederherstellungsvorgang ist fehlgeschlagen und erfordert Ihre Aufmerksamkeit.
+
+Details:
+
+ - Wiederherstellungstyp: {{restore_type}}
+ - Fehler: {{error_message}}
+ - Zeitstempel: {{timestamp}}
+
+
+Bitte überprüfen Sie die Systemprotokolle für weitere Details und ergreifen Sie entsprechende Maßnahmen.
+
+Wichtig: Falls ein Backup vor der Wiederherstellung erstellt wurde, kann es zur Wiederherstellung verwendet werden.
`,
+ language: 'de',
+ is_active: formatBoolean(true),
+ created_at: knex.fn.now(),
+ updated_at: knex.fn.now()
+ }
+ ];
+
+ await knex('email_templates').insert(emailTemplates);
+};
+
+exports.down = async function(knex) {
+ // Remove email templates
+ await knex('email_templates')
+ .whereIn('name', ['restore_completed', 'restore_failed'])
+ .delete();
+
+ // Remove settings
+ await knex('app_settings')
+ .where('setting_type', 'restore')
+ .delete();
+
+ // Drop tables
+ await knex.schema.dropTableIfExists('restore_validation_results');
+ await knex.schema.dropTableIfExists('restore_file_operations');
+ await knex.schema.dropTableIfExists('restore_runs');
+};
\ No newline at end of file
diff --git a/backend/package-lock.json b/backend/package-lock.json
index 758ff61..f16193c 100644
--- a/backend/package-lock.json
+++ b/backend/package-lock.json
@@ -8,6 +8,9 @@
"name": "picpeak-backend",
"version": "1.0.74",
"dependencies": {
+ "@aws-sdk/client-s3": "^3.850.0",
+ "@aws-sdk/lib-storage": "^3.850.0",
+ "@aws-sdk/s3-request-presigner": "^3.850.0",
"adm-zip": "^0.5.16",
"archiver": "^5.3.1",
"axios": "^1.10.0",
@@ -25,8 +28,10 @@
"i18next-browser-languagedetector": "^8.2.0",
"i18next-http-backend": "^3.0.2",
"joi": "^17.9.1",
+ "js-yaml": "^4.1.0",
"jsonwebtoken": "^9.0.0",
"knex": "^2.4.2",
+ "mime-types": "^3.0.1",
"multer": "^2.0.1",
"node-cron": "^3.0.2",
"nodemailer": "^6.9.1",
@@ -59,6 +64,942 @@
"node": ">=6.0.0"
}
},
+ "node_modules/@aws-crypto/crc32": {
+ "version": "5.2.0",
+ "resolved": "https://registry.npmjs.org/@aws-crypto/crc32/-/crc32-5.2.0.tgz",
+ "integrity": "sha512-nLbCWqQNgUiwwtFsen1AdzAtvuLRsQS8rYgMuxCrdKf9kOssamGLuPwyTY9wyYblNr9+1XM8v6zoDTPPSIeANg==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-crypto/util": "^5.2.0",
+ "@aws-sdk/types": "^3.222.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=16.0.0"
+ }
+ },
+ "node_modules/@aws-crypto/crc32c": {
+ "version": "5.2.0",
+ "resolved": "https://registry.npmjs.org/@aws-crypto/crc32c/-/crc32c-5.2.0.tgz",
+ "integrity": "sha512-+iWb8qaHLYKrNvGRbiYRHSdKRWhto5XlZUEBwDjYNf+ly5SVYG6zEoYIdxvf5R3zyeP16w4PLBn3rH1xc74Rag==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-crypto/util": "^5.2.0",
+ "@aws-sdk/types": "^3.222.0",
+ "tslib": "^2.6.2"
+ }
+ },
+ "node_modules/@aws-crypto/sha1-browser": {
+ "version": "5.2.0",
+ "resolved": "https://registry.npmjs.org/@aws-crypto/sha1-browser/-/sha1-browser-5.2.0.tgz",
+ "integrity": "sha512-OH6lveCFfcDjX4dbAvCFSYUjJZjDr/3XJ3xHtjn3Oj5b9RjojQo8npoLeA/bNwkOkrSQ0wgrHzXk4tDRxGKJeg==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-crypto/supports-web-crypto": "^5.2.0",
+ "@aws-crypto/util": "^5.2.0",
+ "@aws-sdk/types": "^3.222.0",
+ "@aws-sdk/util-locate-window": "^3.0.0",
+ "@smithy/util-utf8": "^2.0.0",
+ "tslib": "^2.6.2"
+ }
+ },
+ "node_modules/@aws-crypto/sha1-browser/node_modules/@smithy/is-array-buffer": {
+ "version": "2.2.0",
+ "resolved": "https://registry.npmjs.org/@smithy/is-array-buffer/-/is-array-buffer-2.2.0.tgz",
+ "integrity": "sha512-GGP3O9QFD24uGeAXYUjwSTXARoqpZykHadOmA8G5vfJPK0/DC67qa//0qvqrJzL1xc8WQWX7/yc7fwudjPHPhA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=14.0.0"
+ }
+ },
+ "node_modules/@aws-crypto/sha1-browser/node_modules/@smithy/util-buffer-from": {
+ "version": "2.2.0",
+ "resolved": "https://registry.npmjs.org/@smithy/util-buffer-from/-/util-buffer-from-2.2.0.tgz",
+ "integrity": "sha512-IJdWBbTcMQ6DA0gdNhh/BwrLkDR+ADW5Kr1aZmd4k3DIF6ezMV4R2NIAmT08wQJ3yUK82thHWmC/TnK/wpMMIA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/is-array-buffer": "^2.2.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=14.0.0"
+ }
+ },
+ "node_modules/@aws-crypto/sha1-browser/node_modules/@smithy/util-utf8": {
+ "version": "2.3.0",
+ "resolved": "https://registry.npmjs.org/@smithy/util-utf8/-/util-utf8-2.3.0.tgz",
+ "integrity": "sha512-R8Rdn8Hy72KKcebgLiv8jQcQkXoLMOGGv5uI1/k0l+snqkOzQ1R0ChUBCxWMlBsFMekWjq0wRudIweFs7sKT5A==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/util-buffer-from": "^2.2.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=14.0.0"
+ }
+ },
+ "node_modules/@aws-crypto/sha256-browser": {
+ "version": "5.2.0",
+ "resolved": "https://registry.npmjs.org/@aws-crypto/sha256-browser/-/sha256-browser-5.2.0.tgz",
+ "integrity": "sha512-AXfN/lGotSQwu6HNcEsIASo7kWXZ5HYWvfOmSNKDsEqC4OashTp8alTmaz+F7TC2L083SFv5RdB+qU3Vs1kZqw==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-crypto/sha256-js": "^5.2.0",
+ "@aws-crypto/supports-web-crypto": "^5.2.0",
+ "@aws-crypto/util": "^5.2.0",
+ "@aws-sdk/types": "^3.222.0",
+ "@aws-sdk/util-locate-window": "^3.0.0",
+ "@smithy/util-utf8": "^2.0.0",
+ "tslib": "^2.6.2"
+ }
+ },
+ "node_modules/@aws-crypto/sha256-browser/node_modules/@smithy/is-array-buffer": {
+ "version": "2.2.0",
+ "resolved": "https://registry.npmjs.org/@smithy/is-array-buffer/-/is-array-buffer-2.2.0.tgz",
+ "integrity": "sha512-GGP3O9QFD24uGeAXYUjwSTXARoqpZykHadOmA8G5vfJPK0/DC67qa//0qvqrJzL1xc8WQWX7/yc7fwudjPHPhA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=14.0.0"
+ }
+ },
+ "node_modules/@aws-crypto/sha256-browser/node_modules/@smithy/util-buffer-from": {
+ "version": "2.2.0",
+ "resolved": "https://registry.npmjs.org/@smithy/util-buffer-from/-/util-buffer-from-2.2.0.tgz",
+ "integrity": "sha512-IJdWBbTcMQ6DA0gdNhh/BwrLkDR+ADW5Kr1aZmd4k3DIF6ezMV4R2NIAmT08wQJ3yUK82thHWmC/TnK/wpMMIA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/is-array-buffer": "^2.2.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=14.0.0"
+ }
+ },
+ "node_modules/@aws-crypto/sha256-browser/node_modules/@smithy/util-utf8": {
+ "version": "2.3.0",
+ "resolved": "https://registry.npmjs.org/@smithy/util-utf8/-/util-utf8-2.3.0.tgz",
+ "integrity": "sha512-R8Rdn8Hy72KKcebgLiv8jQcQkXoLMOGGv5uI1/k0l+snqkOzQ1R0ChUBCxWMlBsFMekWjq0wRudIweFs7sKT5A==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/util-buffer-from": "^2.2.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=14.0.0"
+ }
+ },
+ "node_modules/@aws-crypto/sha256-js": {
+ "version": "5.2.0",
+ "resolved": "https://registry.npmjs.org/@aws-crypto/sha256-js/-/sha256-js-5.2.0.tgz",
+ "integrity": "sha512-FFQQyu7edu4ufvIZ+OadFpHHOt+eSTBaYaki44c+akjg7qZg9oOQeLlk77F6tSYqjDAFClrHJk9tMf0HdVyOvA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-crypto/util": "^5.2.0",
+ "@aws-sdk/types": "^3.222.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=16.0.0"
+ }
+ },
+ "node_modules/@aws-crypto/supports-web-crypto": {
+ "version": "5.2.0",
+ "resolved": "https://registry.npmjs.org/@aws-crypto/supports-web-crypto/-/supports-web-crypto-5.2.0.tgz",
+ "integrity": "sha512-iAvUotm021kM33eCdNfwIN//F77/IADDSs58i+MDaOqFrVjZo9bAal0NK7HurRuWLLpF1iLX7gbWrjHjeo+YFg==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "tslib": "^2.6.2"
+ }
+ },
+ "node_modules/@aws-crypto/util": {
+ "version": "5.2.0",
+ "resolved": "https://registry.npmjs.org/@aws-crypto/util/-/util-5.2.0.tgz",
+ "integrity": "sha512-4RkU9EsI6ZpBve5fseQlGNUWKMa1RLPQ1dnjnQoe07ldfIzcsGb5hC5W0Dm7u423KWzawlrpbjXBrXCEv9zazQ==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/types": "^3.222.0",
+ "@smithy/util-utf8": "^2.0.0",
+ "tslib": "^2.6.2"
+ }
+ },
+ "node_modules/@aws-crypto/util/node_modules/@smithy/is-array-buffer": {
+ "version": "2.2.0",
+ "resolved": "https://registry.npmjs.org/@smithy/is-array-buffer/-/is-array-buffer-2.2.0.tgz",
+ "integrity": "sha512-GGP3O9QFD24uGeAXYUjwSTXARoqpZykHadOmA8G5vfJPK0/DC67qa//0qvqrJzL1xc8WQWX7/yc7fwudjPHPhA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=14.0.0"
+ }
+ },
+ "node_modules/@aws-crypto/util/node_modules/@smithy/util-buffer-from": {
+ "version": "2.2.0",
+ "resolved": "https://registry.npmjs.org/@smithy/util-buffer-from/-/util-buffer-from-2.2.0.tgz",
+ "integrity": "sha512-IJdWBbTcMQ6DA0gdNhh/BwrLkDR+ADW5Kr1aZmd4k3DIF6ezMV4R2NIAmT08wQJ3yUK82thHWmC/TnK/wpMMIA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/is-array-buffer": "^2.2.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=14.0.0"
+ }
+ },
+ "node_modules/@aws-crypto/util/node_modules/@smithy/util-utf8": {
+ "version": "2.3.0",
+ "resolved": "https://registry.npmjs.org/@smithy/util-utf8/-/util-utf8-2.3.0.tgz",
+ "integrity": "sha512-R8Rdn8Hy72KKcebgLiv8jQcQkXoLMOGGv5uI1/k0l+snqkOzQ1R0ChUBCxWMlBsFMekWjq0wRudIweFs7sKT5A==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/util-buffer-from": "^2.2.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=14.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/client-s3": {
+ "version": "3.850.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/client-s3/-/client-s3-3.850.0.tgz",
+ "integrity": "sha512-tX5bUfqiLOh6jtAlaiAuOUKFYh8KDG9k9zFLUdgGplC5TP47AYTreUEg+deCTHo4DD3YCvrLuyZ8tIDgKu7neQ==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-crypto/sha1-browser": "5.2.0",
+ "@aws-crypto/sha256-browser": "5.2.0",
+ "@aws-crypto/sha256-js": "5.2.0",
+ "@aws-sdk/core": "3.846.0",
+ "@aws-sdk/credential-provider-node": "3.848.0",
+ "@aws-sdk/middleware-bucket-endpoint": "3.840.0",
+ "@aws-sdk/middleware-expect-continue": "3.840.0",
+ "@aws-sdk/middleware-flexible-checksums": "3.846.0",
+ "@aws-sdk/middleware-host-header": "3.840.0",
+ "@aws-sdk/middleware-location-constraint": "3.840.0",
+ "@aws-sdk/middleware-logger": "3.840.0",
+ "@aws-sdk/middleware-recursion-detection": "3.840.0",
+ "@aws-sdk/middleware-sdk-s3": "3.846.0",
+ "@aws-sdk/middleware-ssec": "3.840.0",
+ "@aws-sdk/middleware-user-agent": "3.848.0",
+ "@aws-sdk/region-config-resolver": "3.840.0",
+ "@aws-sdk/signature-v4-multi-region": "3.846.0",
+ "@aws-sdk/types": "3.840.0",
+ "@aws-sdk/util-endpoints": "3.848.0",
+ "@aws-sdk/util-user-agent-browser": "3.840.0",
+ "@aws-sdk/util-user-agent-node": "3.848.0",
+ "@aws-sdk/xml-builder": "3.821.0",
+ "@smithy/config-resolver": "^4.1.4",
+ "@smithy/core": "^3.7.0",
+ "@smithy/eventstream-serde-browser": "^4.0.4",
+ "@smithy/eventstream-serde-config-resolver": "^4.1.2",
+ "@smithy/eventstream-serde-node": "^4.0.4",
+ "@smithy/fetch-http-handler": "^5.1.0",
+ "@smithy/hash-blob-browser": "^4.0.4",
+ "@smithy/hash-node": "^4.0.4",
+ "@smithy/hash-stream-node": "^4.0.4",
+ "@smithy/invalid-dependency": "^4.0.4",
+ "@smithy/md5-js": "^4.0.4",
+ "@smithy/middleware-content-length": "^4.0.4",
+ "@smithy/middleware-endpoint": "^4.1.15",
+ "@smithy/middleware-retry": "^4.1.16",
+ "@smithy/middleware-serde": "^4.0.8",
+ "@smithy/middleware-stack": "^4.0.4",
+ "@smithy/node-config-provider": "^4.1.3",
+ "@smithy/node-http-handler": "^4.1.0",
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/smithy-client": "^4.4.7",
+ "@smithy/types": "^4.3.1",
+ "@smithy/url-parser": "^4.0.4",
+ "@smithy/util-base64": "^4.0.0",
+ "@smithy/util-body-length-browser": "^4.0.0",
+ "@smithy/util-body-length-node": "^4.0.0",
+ "@smithy/util-defaults-mode-browser": "^4.0.23",
+ "@smithy/util-defaults-mode-node": "^4.0.23",
+ "@smithy/util-endpoints": "^3.0.6",
+ "@smithy/util-middleware": "^4.0.4",
+ "@smithy/util-retry": "^4.0.6",
+ "@smithy/util-stream": "^4.2.3",
+ "@smithy/util-utf8": "^4.0.0",
+ "@smithy/util-waiter": "^4.0.6",
+ "@types/uuid": "^9.0.1",
+ "tslib": "^2.6.2",
+ "uuid": "^9.0.1"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/client-s3/node_modules/uuid": {
+ "version": "9.0.1",
+ "resolved": "https://registry.npmjs.org/uuid/-/uuid-9.0.1.tgz",
+ "integrity": "sha512-b+1eJOlsR9K8HJpow9Ok3fiWOWSIcIzXodvv0rQjVoOVNpWMpxf1wZNpt4y9h10odCNrqnYp1OBzRktckBe3sA==",
+ "funding": [
+ "https://github.com/sponsors/broofa",
+ "https://github.com/sponsors/ctavan"
+ ],
+ "license": "MIT",
+ "bin": {
+ "uuid": "dist/bin/uuid"
+ }
+ },
+ "node_modules/@aws-sdk/client-sso": {
+ "version": "3.848.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/client-sso/-/client-sso-3.848.0.tgz",
+ "integrity": "sha512-mD+gOwoeZQvbecVLGoCmY6pS7kg02BHesbtIxUj+PeBqYoZV5uLvjUOmuGfw1SfoSobKvS11urxC9S7zxU/Maw==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-crypto/sha256-browser": "5.2.0",
+ "@aws-crypto/sha256-js": "5.2.0",
+ "@aws-sdk/core": "3.846.0",
+ "@aws-sdk/middleware-host-header": "3.840.0",
+ "@aws-sdk/middleware-logger": "3.840.0",
+ "@aws-sdk/middleware-recursion-detection": "3.840.0",
+ "@aws-sdk/middleware-user-agent": "3.848.0",
+ "@aws-sdk/region-config-resolver": "3.840.0",
+ "@aws-sdk/types": "3.840.0",
+ "@aws-sdk/util-endpoints": "3.848.0",
+ "@aws-sdk/util-user-agent-browser": "3.840.0",
+ "@aws-sdk/util-user-agent-node": "3.848.0",
+ "@smithy/config-resolver": "^4.1.4",
+ "@smithy/core": "^3.7.0",
+ "@smithy/fetch-http-handler": "^5.1.0",
+ "@smithy/hash-node": "^4.0.4",
+ "@smithy/invalid-dependency": "^4.0.4",
+ "@smithy/middleware-content-length": "^4.0.4",
+ "@smithy/middleware-endpoint": "^4.1.15",
+ "@smithy/middleware-retry": "^4.1.16",
+ "@smithy/middleware-serde": "^4.0.8",
+ "@smithy/middleware-stack": "^4.0.4",
+ "@smithy/node-config-provider": "^4.1.3",
+ "@smithy/node-http-handler": "^4.1.0",
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/smithy-client": "^4.4.7",
+ "@smithy/types": "^4.3.1",
+ "@smithy/url-parser": "^4.0.4",
+ "@smithy/util-base64": "^4.0.0",
+ "@smithy/util-body-length-browser": "^4.0.0",
+ "@smithy/util-body-length-node": "^4.0.0",
+ "@smithy/util-defaults-mode-browser": "^4.0.23",
+ "@smithy/util-defaults-mode-node": "^4.0.23",
+ "@smithy/util-endpoints": "^3.0.6",
+ "@smithy/util-middleware": "^4.0.4",
+ "@smithy/util-retry": "^4.0.6",
+ "@smithy/util-utf8": "^4.0.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/core": {
+ "version": "3.846.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/core/-/core-3.846.0.tgz",
+ "integrity": "sha512-7CX0pM906r4WSS68fCTNMTtBCSkTtf3Wggssmx13gD40gcWEZXsU00KzPp1bYheNRyPlAq3rE22xt4wLPXbuxA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/types": "3.840.0",
+ "@aws-sdk/xml-builder": "3.821.0",
+ "@smithy/core": "^3.7.0",
+ "@smithy/node-config-provider": "^4.1.3",
+ "@smithy/property-provider": "^4.0.4",
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/signature-v4": "^5.1.2",
+ "@smithy/smithy-client": "^4.4.7",
+ "@smithy/types": "^4.3.1",
+ "@smithy/util-base64": "^4.0.0",
+ "@smithy/util-body-length-browser": "^4.0.0",
+ "@smithy/util-middleware": "^4.0.4",
+ "@smithy/util-utf8": "^4.0.0",
+ "fast-xml-parser": "5.2.5",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/credential-provider-env": {
+ "version": "3.846.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/credential-provider-env/-/credential-provider-env-3.846.0.tgz",
+ "integrity": "sha512-QuCQZET9enja7AWVISY+mpFrEIeHzvkx/JEEbHYzHhUkxcnC2Kq2c0bB7hDihGD0AZd3Xsm653hk1O97qu69zg==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/core": "3.846.0",
+ "@aws-sdk/types": "3.840.0",
+ "@smithy/property-provider": "^4.0.4",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/credential-provider-http": {
+ "version": "3.846.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/credential-provider-http/-/credential-provider-http-3.846.0.tgz",
+ "integrity": "sha512-Jh1iKUuepdmtreMYozV2ePsPcOF5W9p3U4tWhi3v6nDvz0GsBjzjAROW+BW8XMz9vAD3I9R+8VC3/aq63p5nlw==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/core": "3.846.0",
+ "@aws-sdk/types": "3.840.0",
+ "@smithy/fetch-http-handler": "^5.1.0",
+ "@smithy/node-http-handler": "^4.1.0",
+ "@smithy/property-provider": "^4.0.4",
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/smithy-client": "^4.4.7",
+ "@smithy/types": "^4.3.1",
+ "@smithy/util-stream": "^4.2.3",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/credential-provider-ini": {
+ "version": "3.848.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/credential-provider-ini/-/credential-provider-ini-3.848.0.tgz",
+ "integrity": "sha512-r6KWOG+En2xujuMhgZu7dzOZV3/M5U/5+PXrG8dLQ3rdPRB3vgp5tc56KMqLwm/EXKRzAOSuw/UE4HfNOAB8Hw==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/core": "3.846.0",
+ "@aws-sdk/credential-provider-env": "3.846.0",
+ "@aws-sdk/credential-provider-http": "3.846.0",
+ "@aws-sdk/credential-provider-process": "3.846.0",
+ "@aws-sdk/credential-provider-sso": "3.848.0",
+ "@aws-sdk/credential-provider-web-identity": "3.848.0",
+ "@aws-sdk/nested-clients": "3.848.0",
+ "@aws-sdk/types": "3.840.0",
+ "@smithy/credential-provider-imds": "^4.0.6",
+ "@smithy/property-provider": "^4.0.4",
+ "@smithy/shared-ini-file-loader": "^4.0.4",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/credential-provider-node": {
+ "version": "3.848.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/credential-provider-node/-/credential-provider-node-3.848.0.tgz",
+ "integrity": "sha512-AblNesOqdzrfyASBCo1xW3uweiSro4Kft9/htdxLeCVU1KVOnFWA5P937MNahViRmIQm2sPBCqL8ZG0u9lnh5g==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/credential-provider-env": "3.846.0",
+ "@aws-sdk/credential-provider-http": "3.846.0",
+ "@aws-sdk/credential-provider-ini": "3.848.0",
+ "@aws-sdk/credential-provider-process": "3.846.0",
+ "@aws-sdk/credential-provider-sso": "3.848.0",
+ "@aws-sdk/credential-provider-web-identity": "3.848.0",
+ "@aws-sdk/types": "3.840.0",
+ "@smithy/credential-provider-imds": "^4.0.6",
+ "@smithy/property-provider": "^4.0.4",
+ "@smithy/shared-ini-file-loader": "^4.0.4",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/credential-provider-process": {
+ "version": "3.846.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/credential-provider-process/-/credential-provider-process-3.846.0.tgz",
+ "integrity": "sha512-mEpwDYarJSH+CIXnnHN0QOe0MXI+HuPStD6gsv3z/7Q6ESl8KRWon3weFZCDnqpiJMUVavlDR0PPlAFg2MQoPg==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/core": "3.846.0",
+ "@aws-sdk/types": "3.840.0",
+ "@smithy/property-provider": "^4.0.4",
+ "@smithy/shared-ini-file-loader": "^4.0.4",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/credential-provider-sso": {
+ "version": "3.848.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/credential-provider-sso/-/credential-provider-sso-3.848.0.tgz",
+ "integrity": "sha512-pozlDXOwJZL0e7w+dqXLgzVDB7oCx4WvtY0sk6l4i07uFliWF/exupb6pIehFWvTUcOvn5aFTTqcQaEzAD5Wsg==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/client-sso": "3.848.0",
+ "@aws-sdk/core": "3.846.0",
+ "@aws-sdk/token-providers": "3.848.0",
+ "@aws-sdk/types": "3.840.0",
+ "@smithy/property-provider": "^4.0.4",
+ "@smithy/shared-ini-file-loader": "^4.0.4",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/credential-provider-web-identity": {
+ "version": "3.848.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/credential-provider-web-identity/-/credential-provider-web-identity-3.848.0.tgz",
+ "integrity": "sha512-D1fRpwPxtVDhcSc/D71exa2gYweV+ocp4D3brF0PgFd//JR3XahZ9W24rVnTQwYEcK9auiBZB89Ltv+WbWN8qw==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/core": "3.846.0",
+ "@aws-sdk/nested-clients": "3.848.0",
+ "@aws-sdk/types": "3.840.0",
+ "@smithy/property-provider": "^4.0.4",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/lib-storage": {
+ "version": "3.850.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/lib-storage/-/lib-storage-3.850.0.tgz",
+ "integrity": "sha512-DKG8mKeUMLRboyqwhKiV9QOiKXN00OYLnGsT21mhlaF1Uc7OZ6Vm+Olw4YrbYSBuDup0rMWtVaWudJ49I+ZCHA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/abort-controller": "^4.0.4",
+ "@smithy/middleware-endpoint": "^4.1.15",
+ "@smithy/smithy-client": "^4.4.7",
+ "buffer": "5.6.0",
+ "events": "3.3.0",
+ "stream-browserify": "3.0.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ },
+ "peerDependencies": {
+ "@aws-sdk/client-s3": "^3.850.0"
+ }
+ },
+ "node_modules/@aws-sdk/lib-storage/node_modules/buffer": {
+ "version": "5.6.0",
+ "resolved": "https://registry.npmjs.org/buffer/-/buffer-5.6.0.tgz",
+ "integrity": "sha512-/gDYp/UtU0eA1ys8bOs9J6a+E/KWIY+DZ+Q2WESNUA0jFRsJOc0SNUO6xJ5SGA1xueg3NL65W6s+NY5l9cunuw==",
+ "license": "MIT",
+ "dependencies": {
+ "base64-js": "^1.0.2",
+ "ieee754": "^1.1.4"
+ }
+ },
+ "node_modules/@aws-sdk/middleware-bucket-endpoint": {
+ "version": "3.840.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/middleware-bucket-endpoint/-/middleware-bucket-endpoint-3.840.0.tgz",
+ "integrity": "sha512-+gkQNtPwcSMmlwBHFd4saVVS11In6ID1HczNzpM3MXKXRBfSlbZJbCt6wN//AZ8HMklZEik4tcEOG0qa9UY8SQ==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/types": "3.840.0",
+ "@aws-sdk/util-arn-parser": "3.804.0",
+ "@smithy/node-config-provider": "^4.1.3",
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/types": "^4.3.1",
+ "@smithy/util-config-provider": "^4.0.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/middleware-expect-continue": {
+ "version": "3.840.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/middleware-expect-continue/-/middleware-expect-continue-3.840.0.tgz",
+ "integrity": "sha512-iJg2r6FKsKKvdiU4oCOuCf7Ro/YE0Q2BT/QyEZN3/Rt8Nr4SAZiQOlcBXOCpGvuIKOEAhvDOUnW3aDHL01PdVw==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/types": "3.840.0",
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/middleware-flexible-checksums": {
+ "version": "3.846.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/middleware-flexible-checksums/-/middleware-flexible-checksums-3.846.0.tgz",
+ "integrity": "sha512-CdkeVfkwt3+bDLhmOwBxvkUf6oY9iUhvosaUnqkoPsOqIiUEN54yTGOnO8A0wLz6mMsZ6aBlfFrQhFnxt3c+yw==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-crypto/crc32": "5.2.0",
+ "@aws-crypto/crc32c": "5.2.0",
+ "@aws-crypto/util": "5.2.0",
+ "@aws-sdk/core": "3.846.0",
+ "@aws-sdk/types": "3.840.0",
+ "@smithy/is-array-buffer": "^4.0.0",
+ "@smithy/node-config-provider": "^4.1.3",
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/types": "^4.3.1",
+ "@smithy/util-middleware": "^4.0.4",
+ "@smithy/util-stream": "^4.2.3",
+ "@smithy/util-utf8": "^4.0.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/middleware-host-header": {
+ "version": "3.840.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/middleware-host-header/-/middleware-host-header-3.840.0.tgz",
+ "integrity": "sha512-ub+hXJAbAje94+Ya6c6eL7sYujoE8D4Bumu1NUI8TXjUhVVn0HzVWQjpRLshdLsUp1AW7XyeJaxyajRaJQ8+Xg==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/types": "3.840.0",
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/middleware-location-constraint": {
+ "version": "3.840.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/middleware-location-constraint/-/middleware-location-constraint-3.840.0.tgz",
+ "integrity": "sha512-KVLD0u0YMF3aQkVF8bdyHAGWSUY6N1Du89htTLgqCcIhSxxAJ9qifrosVZ9jkAzqRW99hcufyt2LylcVU2yoKQ==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/types": "3.840.0",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/middleware-logger": {
+ "version": "3.840.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/middleware-logger/-/middleware-logger-3.840.0.tgz",
+ "integrity": "sha512-lSV8FvjpdllpGaRspywss4CtXV8M7NNNH+2/j86vMH+YCOZ6fu2T/TyFd/tHwZ92vDfHctWkRbQxg0bagqwovA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/types": "3.840.0",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/middleware-recursion-detection": {
+ "version": "3.840.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/middleware-recursion-detection/-/middleware-recursion-detection-3.840.0.tgz",
+ "integrity": "sha512-Gu7lGDyfddyhIkj1Z1JtrY5NHb5+x/CRiB87GjaSrKxkDaydtX2CU977JIABtt69l9wLbcGDIQ+W0uJ5xPof7g==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/types": "3.840.0",
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/middleware-sdk-s3": {
+ "version": "3.846.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/middleware-sdk-s3/-/middleware-sdk-s3-3.846.0.tgz",
+ "integrity": "sha512-jP9x+2Q87J5l8FOP+jlAd7vGLn0cC6G9QGmf386e5OslBPqxXKcl3RjqGLIOKKos2mVItY3ApP5xdXQx7jGTVA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/core": "3.846.0",
+ "@aws-sdk/types": "3.840.0",
+ "@aws-sdk/util-arn-parser": "3.804.0",
+ "@smithy/core": "^3.7.0",
+ "@smithy/node-config-provider": "^4.1.3",
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/signature-v4": "^5.1.2",
+ "@smithy/smithy-client": "^4.4.7",
+ "@smithy/types": "^4.3.1",
+ "@smithy/util-config-provider": "^4.0.0",
+ "@smithy/util-middleware": "^4.0.4",
+ "@smithy/util-stream": "^4.2.3",
+ "@smithy/util-utf8": "^4.0.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/middleware-ssec": {
+ "version": "3.840.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/middleware-ssec/-/middleware-ssec-3.840.0.tgz",
+ "integrity": "sha512-CBZP9t1QbjDFGOrtnUEHL1oAvmnCUUm7p0aPNbIdSzNtH42TNKjPRN3TuEIJDGjkrqpL3MXyDSmNayDcw/XW7Q==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/types": "3.840.0",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/middleware-user-agent": {
+ "version": "3.848.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/middleware-user-agent/-/middleware-user-agent-3.848.0.tgz",
+ "integrity": "sha512-rjMuqSWJEf169/ByxvBqfdei1iaduAnfolTshsZxwcmLIUtbYrFUmts0HrLQqsAG8feGPpDLHA272oPl+NTCCA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/core": "3.846.0",
+ "@aws-sdk/types": "3.840.0",
+ "@aws-sdk/util-endpoints": "3.848.0",
+ "@smithy/core": "^3.7.0",
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/nested-clients": {
+ "version": "3.848.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/nested-clients/-/nested-clients-3.848.0.tgz",
+ "integrity": "sha512-joLsyyo9u61jnZuyYzo1z7kmS7VgWRAkzSGESVzQHfOA1H2PYeUFek6vLT4+c9xMGrX/Z6B0tkRdzfdOPiatLg==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-crypto/sha256-browser": "5.2.0",
+ "@aws-crypto/sha256-js": "5.2.0",
+ "@aws-sdk/core": "3.846.0",
+ "@aws-sdk/middleware-host-header": "3.840.0",
+ "@aws-sdk/middleware-logger": "3.840.0",
+ "@aws-sdk/middleware-recursion-detection": "3.840.0",
+ "@aws-sdk/middleware-user-agent": "3.848.0",
+ "@aws-sdk/region-config-resolver": "3.840.0",
+ "@aws-sdk/types": "3.840.0",
+ "@aws-sdk/util-endpoints": "3.848.0",
+ "@aws-sdk/util-user-agent-browser": "3.840.0",
+ "@aws-sdk/util-user-agent-node": "3.848.0",
+ "@smithy/config-resolver": "^4.1.4",
+ "@smithy/core": "^3.7.0",
+ "@smithy/fetch-http-handler": "^5.1.0",
+ "@smithy/hash-node": "^4.0.4",
+ "@smithy/invalid-dependency": "^4.0.4",
+ "@smithy/middleware-content-length": "^4.0.4",
+ "@smithy/middleware-endpoint": "^4.1.15",
+ "@smithy/middleware-retry": "^4.1.16",
+ "@smithy/middleware-serde": "^4.0.8",
+ "@smithy/middleware-stack": "^4.0.4",
+ "@smithy/node-config-provider": "^4.1.3",
+ "@smithy/node-http-handler": "^4.1.0",
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/smithy-client": "^4.4.7",
+ "@smithy/types": "^4.3.1",
+ "@smithy/url-parser": "^4.0.4",
+ "@smithy/util-base64": "^4.0.0",
+ "@smithy/util-body-length-browser": "^4.0.0",
+ "@smithy/util-body-length-node": "^4.0.0",
+ "@smithy/util-defaults-mode-browser": "^4.0.23",
+ "@smithy/util-defaults-mode-node": "^4.0.23",
+ "@smithy/util-endpoints": "^3.0.6",
+ "@smithy/util-middleware": "^4.0.4",
+ "@smithy/util-retry": "^4.0.6",
+ "@smithy/util-utf8": "^4.0.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/region-config-resolver": {
+ "version": "3.840.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/region-config-resolver/-/region-config-resolver-3.840.0.tgz",
+ "integrity": "sha512-Qjnxd/yDv9KpIMWr90ZDPtRj0v75AqGC92Lm9+oHXZ8p1MjG5JE2CW0HL8JRgK9iKzgKBL7pPQRXI8FkvEVfrA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/types": "3.840.0",
+ "@smithy/node-config-provider": "^4.1.3",
+ "@smithy/types": "^4.3.1",
+ "@smithy/util-config-provider": "^4.0.0",
+ "@smithy/util-middleware": "^4.0.4",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/s3-request-presigner": {
+ "version": "3.850.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/s3-request-presigner/-/s3-request-presigner-3.850.0.tgz",
+ "integrity": "sha512-eFvMUCJXoVTkAxkqHKn125mLMGtNa76+oD3wV97ScXUZuL5liaj+kAN9nSqRiQ5vaCz5gsOeB9t/ba/cTGATjg==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/signature-v4-multi-region": "3.846.0",
+ "@aws-sdk/types": "3.840.0",
+ "@aws-sdk/util-format-url": "3.840.0",
+ "@smithy/middleware-endpoint": "^4.1.15",
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/smithy-client": "^4.4.7",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/signature-v4-multi-region": {
+ "version": "3.846.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/signature-v4-multi-region/-/signature-v4-multi-region-3.846.0.tgz",
+ "integrity": "sha512-ZMfIMxUljqZzPJGOcraC6erwq/z1puNMU35cO1a/WdhB+LdYknMn1lr7SJuH754QwNzzIlZbEgg4hoHw50+DpQ==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/middleware-sdk-s3": "3.846.0",
+ "@aws-sdk/types": "3.840.0",
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/signature-v4": "^5.1.2",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/token-providers": {
+ "version": "3.848.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/token-providers/-/token-providers-3.848.0.tgz",
+ "integrity": "sha512-oNPyM4+Di2Umu0JJRFSxDcKQ35+Chl/rAwD47/bS0cDPI8yrao83mLXLeDqpRPHyQW4sXlP763FZcuAibC0+mg==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/core": "3.846.0",
+ "@aws-sdk/nested-clients": "3.848.0",
+ "@aws-sdk/types": "3.840.0",
+ "@smithy/property-provider": "^4.0.4",
+ "@smithy/shared-ini-file-loader": "^4.0.4",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/types": {
+ "version": "3.840.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/types/-/types-3.840.0.tgz",
+ "integrity": "sha512-xliuHaUFZxEx1NSXeLLZ9Dyu6+EJVQKEoD+yM+zqUo3YDZ7medKJWY6fIOKiPX/N7XbLdBYwajb15Q7IL8KkeA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/util-arn-parser": {
+ "version": "3.804.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/util-arn-parser/-/util-arn-parser-3.804.0.tgz",
+ "integrity": "sha512-wmBJqn1DRXnZu3b4EkE6CWnoWMo1ZMvlfkqU5zPz67xx1GMaXlDCchFvKAXMjk4jn/L1O3tKnoFDNsoLV1kgNQ==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/util-endpoints": {
+ "version": "3.848.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/util-endpoints/-/util-endpoints-3.848.0.tgz",
+ "integrity": "sha512-fY/NuFFCq/78liHvRyFKr+aqq1aA/uuVSANjzr5Ym8c+9Z3HRPE9OrExAHoMrZ6zC8tHerQwlsXYYH5XZ7H+ww==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/types": "3.840.0",
+ "@smithy/types": "^4.3.1",
+ "@smithy/url-parser": "^4.0.4",
+ "@smithy/util-endpoints": "^3.0.6",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/util-format-url": {
+ "version": "3.840.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/util-format-url/-/util-format-url-3.840.0.tgz",
+ "integrity": "sha512-VB1PWyI1TQPiPvg4w7tgUGGQER1xxXPNUqfh3baxUSFi1Oh8wHrDnFywkxLm3NMmgDmnLnSZ5Q326qAoyqKLSg==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/types": "3.840.0",
+ "@smithy/querystring-builder": "^4.0.4",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/util-locate-window": {
+ "version": "3.804.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/util-locate-window/-/util-locate-window-3.804.0.tgz",
+ "integrity": "sha512-zVoRfpmBVPodYlnMjgVjfGoEZagyRF5IPn3Uo6ZvOZp24chnW/FRstH7ESDHDDRga4z3V+ElUQHKpFDXWyBW5A==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@aws-sdk/util-user-agent-browser": {
+ "version": "3.840.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/util-user-agent-browser/-/util-user-agent-browser-3.840.0.tgz",
+ "integrity": "sha512-JdyZM3EhhL4PqwFpttZu1afDpPJCCc3eyZOLi+srpX11LsGj6sThf47TYQN75HT1CarZ7cCdQHGzP2uy3/xHfQ==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/types": "3.840.0",
+ "@smithy/types": "^4.3.1",
+ "bowser": "^2.11.0",
+ "tslib": "^2.6.2"
+ }
+ },
+ "node_modules/@aws-sdk/util-user-agent-node": {
+ "version": "3.848.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/util-user-agent-node/-/util-user-agent-node-3.848.0.tgz",
+ "integrity": "sha512-Zz1ft9NiLqbzNj/M0jVNxaoxI2F4tGXN0ZbZIj+KJ+PbJo+w5+Jo6d0UDAtbj3AEd79pjcCaP4OA9NTVzItUdw==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-sdk/middleware-user-agent": "3.848.0",
+ "@aws-sdk/types": "3.840.0",
+ "@smithy/node-config-provider": "^4.1.3",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ },
+ "peerDependencies": {
+ "aws-crt": ">=1.0.0"
+ },
+ "peerDependenciesMeta": {
+ "aws-crt": {
+ "optional": true
+ }
+ }
+ },
+ "node_modules/@aws-sdk/xml-builder": {
+ "version": "3.821.0",
+ "resolved": "https://registry.npmjs.org/@aws-sdk/xml-builder/-/xml-builder-3.821.0.tgz",
+ "integrity": "sha512-DIIotRnefVL6DiaHtO6/21DhJ4JZnnIwdNbpwiAhdt/AVbttcE4yw925gsjur0OGv5BTYXQXU3YnANBYnZjuQA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
"node_modules/@babel/code-frame": {
"version": "7.27.1",
"resolved": "https://registry.npmjs.org/@babel/code-frame/-/code-frame-7.27.1.tgz",
@@ -1350,6 +2291,738 @@
"@sinonjs/commons": "^3.0.0"
}
},
+ "node_modules/@smithy/abort-controller": {
+ "version": "4.0.4",
+ "resolved": "https://registry.npmjs.org/@smithy/abort-controller/-/abort-controller-4.0.4.tgz",
+ "integrity": "sha512-gJnEjZMvigPDQWHrW3oPrFhQtkrgqBkyjj3pCIdF3A5M6vsZODG93KNlfJprv6bp4245bdT32fsHK4kkH3KYDA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/chunked-blob-reader": {
+ "version": "5.0.0",
+ "resolved": "https://registry.npmjs.org/@smithy/chunked-blob-reader/-/chunked-blob-reader-5.0.0.tgz",
+ "integrity": "sha512-+sKqDBQqb036hh4NPaUiEkYFkTUGYzRsn3EuFhyfQfMy6oGHEUJDurLP9Ufb5dasr/XiAmPNMr6wa9afjQB+Gw==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/chunked-blob-reader-native": {
+ "version": "4.0.0",
+ "resolved": "https://registry.npmjs.org/@smithy/chunked-blob-reader-native/-/chunked-blob-reader-native-4.0.0.tgz",
+ "integrity": "sha512-R9wM2yPmfEMsUmlMlIgSzOyICs0x9uu7UTHoccMyt7BWw8shcGM8HqB355+BZCPBcySvbTYMs62EgEQkNxz2ig==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/util-base64": "^4.0.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/config-resolver": {
+ "version": "4.1.4",
+ "resolved": "https://registry.npmjs.org/@smithy/config-resolver/-/config-resolver-4.1.4.tgz",
+ "integrity": "sha512-prmU+rDddxHOH0oNcwemL+SwnzcG65sBF2yXRO7aeXIn/xTlq2pX7JLVbkBnVLowHLg4/OL4+jBmv9hVrVGS+w==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/node-config-provider": "^4.1.3",
+ "@smithy/types": "^4.3.1",
+ "@smithy/util-config-provider": "^4.0.0",
+ "@smithy/util-middleware": "^4.0.4",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/core": {
+ "version": "3.7.1",
+ "resolved": "https://registry.npmjs.org/@smithy/core/-/core-3.7.1.tgz",
+ "integrity": "sha512-ExRCsHnXFtBPnM7MkfKBPcBBdHw1h/QS/cbNw4ho95qnyNHvnpmGbR39MIAv9KggTr5qSPxRSEL+hRXlyGyGQw==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/middleware-serde": "^4.0.8",
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/types": "^4.3.1",
+ "@smithy/util-base64": "^4.0.0",
+ "@smithy/util-body-length-browser": "^4.0.0",
+ "@smithy/util-middleware": "^4.0.4",
+ "@smithy/util-stream": "^4.2.3",
+ "@smithy/util-utf8": "^4.0.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/credential-provider-imds": {
+ "version": "4.0.6",
+ "resolved": "https://registry.npmjs.org/@smithy/credential-provider-imds/-/credential-provider-imds-4.0.6.tgz",
+ "integrity": "sha512-hKMWcANhUiNbCJouYkZ9V3+/Qf9pteR1dnwgdyzR09R4ODEYx8BbUysHwRSyex4rZ9zapddZhLFTnT4ZijR4pw==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/node-config-provider": "^4.1.3",
+ "@smithy/property-provider": "^4.0.4",
+ "@smithy/types": "^4.3.1",
+ "@smithy/url-parser": "^4.0.4",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/eventstream-codec": {
+ "version": "4.0.4",
+ "resolved": "https://registry.npmjs.org/@smithy/eventstream-codec/-/eventstream-codec-4.0.4.tgz",
+ "integrity": "sha512-7XoWfZqWb/QoR/rAU4VSi0mWnO2vu9/ltS6JZ5ZSZv0eovLVfDfu0/AX4ub33RsJTOth3TiFWSHS5YdztvFnig==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@aws-crypto/crc32": "5.2.0",
+ "@smithy/types": "^4.3.1",
+ "@smithy/util-hex-encoding": "^4.0.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/eventstream-serde-browser": {
+ "version": "4.0.4",
+ "resolved": "https://registry.npmjs.org/@smithy/eventstream-serde-browser/-/eventstream-serde-browser-4.0.4.tgz",
+ "integrity": "sha512-3fb/9SYaYqbpy/z/H3yIi0bYKyAa89y6xPmIqwr2vQiUT2St+avRt8UKwsWt9fEdEasc5d/V+QjrviRaX1JRFA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/eventstream-serde-universal": "^4.0.4",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/eventstream-serde-config-resolver": {
+ "version": "4.1.2",
+ "resolved": "https://registry.npmjs.org/@smithy/eventstream-serde-config-resolver/-/eventstream-serde-config-resolver-4.1.2.tgz",
+ "integrity": "sha512-JGtambizrWP50xHgbzZI04IWU7LdI0nh/wGbqH3sJesYToMi2j/DcoElqyOcqEIG/D4tNyxgRuaqBXWE3zOFhQ==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/eventstream-serde-node": {
+ "version": "4.0.4",
+ "resolved": "https://registry.npmjs.org/@smithy/eventstream-serde-node/-/eventstream-serde-node-4.0.4.tgz",
+ "integrity": "sha512-RD6UwNZ5zISpOWPuhVgRz60GkSIp0dy1fuZmj4RYmqLVRtejFqQ16WmfYDdoSoAjlp1LX+FnZo+/hkdmyyGZ1w==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/eventstream-serde-universal": "^4.0.4",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/eventstream-serde-universal": {
+ "version": "4.0.4",
+ "resolved": "https://registry.npmjs.org/@smithy/eventstream-serde-universal/-/eventstream-serde-universal-4.0.4.tgz",
+ "integrity": "sha512-UeJpOmLGhq1SLox79QWw/0n2PFX+oPRE1ZyRMxPIaFEfCqWaqpB7BU9C8kpPOGEhLF7AwEqfFbtwNxGy4ReENA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/eventstream-codec": "^4.0.4",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/fetch-http-handler": {
+ "version": "5.1.0",
+ "resolved": "https://registry.npmjs.org/@smithy/fetch-http-handler/-/fetch-http-handler-5.1.0.tgz",
+ "integrity": "sha512-mADw7MS0bYe2OGKkHYMaqarOXuDwRbO6ArD91XhHcl2ynjGCFF+hvqf0LyQcYxkA1zaWjefSkU7Ne9mqgApSgQ==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/querystring-builder": "^4.0.4",
+ "@smithy/types": "^4.3.1",
+ "@smithy/util-base64": "^4.0.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/hash-blob-browser": {
+ "version": "4.0.4",
+ "resolved": "https://registry.npmjs.org/@smithy/hash-blob-browser/-/hash-blob-browser-4.0.4.tgz",
+ "integrity": "sha512-WszRiACJiQV3QG6XMV44i5YWlkrlsM5Yxgz4jvsksuu7LDXA6wAtypfPajtNTadzpJy3KyJPoWehYpmZGKUFIQ==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/chunked-blob-reader": "^5.0.0",
+ "@smithy/chunked-blob-reader-native": "^4.0.0",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/hash-node": {
+ "version": "4.0.4",
+ "resolved": "https://registry.npmjs.org/@smithy/hash-node/-/hash-node-4.0.4.tgz",
+ "integrity": "sha512-qnbTPUhCVnCgBp4z4BUJUhOEkVwxiEi1cyFM+Zj6o+aY8OFGxUQleKWq8ltgp3dujuhXojIvJWdoqpm6dVO3lQ==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/types": "^4.3.1",
+ "@smithy/util-buffer-from": "^4.0.0",
+ "@smithy/util-utf8": "^4.0.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/hash-stream-node": {
+ "version": "4.0.4",
+ "resolved": "https://registry.npmjs.org/@smithy/hash-stream-node/-/hash-stream-node-4.0.4.tgz",
+ "integrity": "sha512-wHo0d8GXyVmpmMh/qOR0R7Y46/G1y6OR8U+bSTB4ppEzRxd1xVAQ9xOE9hOc0bSjhz0ujCPAbfNLkLrpa6cevg==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/types": "^4.3.1",
+ "@smithy/util-utf8": "^4.0.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/invalid-dependency": {
+ "version": "4.0.4",
+ "resolved": "https://registry.npmjs.org/@smithy/invalid-dependency/-/invalid-dependency-4.0.4.tgz",
+ "integrity": "sha512-bNYMi7WKTJHu0gn26wg8OscncTt1t2b8KcsZxvOv56XA6cyXtOAAAaNP7+m45xfppXfOatXF3Sb1MNsLUgVLTw==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/is-array-buffer": {
+ "version": "4.0.0",
+ "resolved": "https://registry.npmjs.org/@smithy/is-array-buffer/-/is-array-buffer-4.0.0.tgz",
+ "integrity": "sha512-saYhF8ZZNoJDTvJBEWgeBccCg+yvp1CX+ed12yORU3NilJScfc6gfch2oVb4QgxZrGUx3/ZJlb+c/dJbyupxlw==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/md5-js": {
+ "version": "4.0.4",
+ "resolved": "https://registry.npmjs.org/@smithy/md5-js/-/md5-js-4.0.4.tgz",
+ "integrity": "sha512-uGLBVqcOwrLvGh/v/jw423yWHq/ofUGK1W31M2TNspLQbUV1Va0F5kTxtirkoHawODAZcjXTSGi7JwbnPcDPJg==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/types": "^4.3.1",
+ "@smithy/util-utf8": "^4.0.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/middleware-content-length": {
+ "version": "4.0.4",
+ "resolved": "https://registry.npmjs.org/@smithy/middleware-content-length/-/middleware-content-length-4.0.4.tgz",
+ "integrity": "sha512-F7gDyfI2BB1Kc+4M6rpuOLne5LOcEknH1n6UQB69qv+HucXBR1rkzXBnQTB2q46sFy1PM/zuSJOB532yc8bg3w==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/middleware-endpoint": {
+ "version": "4.1.16",
+ "resolved": "https://registry.npmjs.org/@smithy/middleware-endpoint/-/middleware-endpoint-4.1.16.tgz",
+ "integrity": "sha512-plpa50PIGLqzMR2ANKAw2yOW5YKS626KYKqae3atwucbz4Ve4uQ9K9BEZxDLIFmCu7hKLcrq2zmj4a+PfmUV5w==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/core": "^3.7.1",
+ "@smithy/middleware-serde": "^4.0.8",
+ "@smithy/node-config-provider": "^4.1.3",
+ "@smithy/shared-ini-file-loader": "^4.0.4",
+ "@smithy/types": "^4.3.1",
+ "@smithy/url-parser": "^4.0.4",
+ "@smithy/util-middleware": "^4.0.4",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/middleware-retry": {
+ "version": "4.1.17",
+ "resolved": "https://registry.npmjs.org/@smithy/middleware-retry/-/middleware-retry-4.1.17.tgz",
+ "integrity": "sha512-gsCimeG6BApj0SBecwa1Be+Z+JOJe46iy3B3m3A8jKJHf7eIihP76Is4LwLrbJ1ygoS7Vg73lfqzejmLOrazUA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/node-config-provider": "^4.1.3",
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/service-error-classification": "^4.0.6",
+ "@smithy/smithy-client": "^4.4.8",
+ "@smithy/types": "^4.3.1",
+ "@smithy/util-middleware": "^4.0.4",
+ "@smithy/util-retry": "^4.0.6",
+ "tslib": "^2.6.2",
+ "uuid": "^9.0.1"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/middleware-retry/node_modules/uuid": {
+ "version": "9.0.1",
+ "resolved": "https://registry.npmjs.org/uuid/-/uuid-9.0.1.tgz",
+ "integrity": "sha512-b+1eJOlsR9K8HJpow9Ok3fiWOWSIcIzXodvv0rQjVoOVNpWMpxf1wZNpt4y9h10odCNrqnYp1OBzRktckBe3sA==",
+ "funding": [
+ "https://github.com/sponsors/broofa",
+ "https://github.com/sponsors/ctavan"
+ ],
+ "license": "MIT",
+ "bin": {
+ "uuid": "dist/bin/uuid"
+ }
+ },
+ "node_modules/@smithy/middleware-serde": {
+ "version": "4.0.8",
+ "resolved": "https://registry.npmjs.org/@smithy/middleware-serde/-/middleware-serde-4.0.8.tgz",
+ "integrity": "sha512-iSSl7HJoJaGyMIoNn2B7czghOVwJ9nD7TMvLhMWeSB5vt0TnEYyRRqPJu/TqW76WScaNvYYB8nRoiBHR9S1Ddw==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/middleware-stack": {
+ "version": "4.0.4",
+ "resolved": "https://registry.npmjs.org/@smithy/middleware-stack/-/middleware-stack-4.0.4.tgz",
+ "integrity": "sha512-kagK5ggDrBUCCzI93ft6DjteNSfY8Ulr83UtySog/h09lTIOAJ/xUSObutanlPT0nhoHAkpmW9V5K8oPyLh+QA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/node-config-provider": {
+ "version": "4.1.3",
+ "resolved": "https://registry.npmjs.org/@smithy/node-config-provider/-/node-config-provider-4.1.3.tgz",
+ "integrity": "sha512-HGHQr2s59qaU1lrVH6MbLlmOBxadtzTsoO4c+bF5asdgVik3I8o7JIOzoeqWc5MjVa+vD36/LWE0iXKpNqooRw==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/property-provider": "^4.0.4",
+ "@smithy/shared-ini-file-loader": "^4.0.4",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/node-http-handler": {
+ "version": "4.1.0",
+ "resolved": "https://registry.npmjs.org/@smithy/node-http-handler/-/node-http-handler-4.1.0.tgz",
+ "integrity": "sha512-vqfSiHz2v8b3TTTrdXi03vNz1KLYYS3bhHCDv36FYDqxT7jvTll1mMnCrkD+gOvgwybuunh/2VmvOMqwBegxEg==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/abort-controller": "^4.0.4",
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/querystring-builder": "^4.0.4",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/property-provider": {
+ "version": "4.0.4",
+ "resolved": "https://registry.npmjs.org/@smithy/property-provider/-/property-provider-4.0.4.tgz",
+ "integrity": "sha512-qHJ2sSgu4FqF4U/5UUp4DhXNmdTrgmoAai6oQiM+c5RZ/sbDwJ12qxB1M6FnP+Tn/ggkPZf9ccn4jqKSINaquw==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/protocol-http": {
+ "version": "5.1.2",
+ "resolved": "https://registry.npmjs.org/@smithy/protocol-http/-/protocol-http-5.1.2.tgz",
+ "integrity": "sha512-rOG5cNLBXovxIrICSBm95dLqzfvxjEmuZx4KK3hWwPFHGdW3lxY0fZNXfv2zebfRO7sJZ5pKJYHScsqopeIWtQ==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/querystring-builder": {
+ "version": "4.0.4",
+ "resolved": "https://registry.npmjs.org/@smithy/querystring-builder/-/querystring-builder-4.0.4.tgz",
+ "integrity": "sha512-SwREZcDnEYoh9tLNgMbpop+UTGq44Hl9tdj3rf+yeLcfH7+J8OXEBaMc2kDxtyRHu8BhSg9ADEx0gFHvpJgU8w==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/types": "^4.3.1",
+ "@smithy/util-uri-escape": "^4.0.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/querystring-parser": {
+ "version": "4.0.4",
+ "resolved": "https://registry.npmjs.org/@smithy/querystring-parser/-/querystring-parser-4.0.4.tgz",
+ "integrity": "sha512-6yZf53i/qB8gRHH/l2ZwUG5xgkPgQF15/KxH0DdXMDHjesA9MeZje/853ifkSY0x4m5S+dfDZ+c4x439PF0M2w==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/service-error-classification": {
+ "version": "4.0.6",
+ "resolved": "https://registry.npmjs.org/@smithy/service-error-classification/-/service-error-classification-4.0.6.tgz",
+ "integrity": "sha512-RRoTDL//7xi4tn5FrN2NzH17jbgmnKidUqd4KvquT0954/i6CXXkh1884jBiunq24g9cGtPBEXlU40W6EpNOOg==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/types": "^4.3.1"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/shared-ini-file-loader": {
+ "version": "4.0.4",
+ "resolved": "https://registry.npmjs.org/@smithy/shared-ini-file-loader/-/shared-ini-file-loader-4.0.4.tgz",
+ "integrity": "sha512-63X0260LoFBjrHifPDs+nM9tV0VMkOTl4JRMYNuKh/f5PauSjowTfvF3LogfkWdcPoxsA9UjqEOgjeYIbhb7Nw==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/signature-v4": {
+ "version": "5.1.2",
+ "resolved": "https://registry.npmjs.org/@smithy/signature-v4/-/signature-v4-5.1.2.tgz",
+ "integrity": "sha512-d3+U/VpX7a60seHziWnVZOHuEgJlclufjkS6zhXvxcJgkJq4UWdH5eOBLzHRMx6gXjsdT9h6lfpmLzbrdupHgQ==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/is-array-buffer": "^4.0.0",
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/types": "^4.3.1",
+ "@smithy/util-hex-encoding": "^4.0.0",
+ "@smithy/util-middleware": "^4.0.4",
+ "@smithy/util-uri-escape": "^4.0.0",
+ "@smithy/util-utf8": "^4.0.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/smithy-client": {
+ "version": "4.4.8",
+ "resolved": "https://registry.npmjs.org/@smithy/smithy-client/-/smithy-client-4.4.8.tgz",
+ "integrity": "sha512-pcW691/lx7V54gE+dDGC26nxz8nrvnvRSCJaIYD6XLPpOInEZeKdV/SpSux+wqeQ4Ine7LJQu8uxMvobTIBK0w==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/core": "^3.7.1",
+ "@smithy/middleware-endpoint": "^4.1.16",
+ "@smithy/middleware-stack": "^4.0.4",
+ "@smithy/protocol-http": "^5.1.2",
+ "@smithy/types": "^4.3.1",
+ "@smithy/util-stream": "^4.2.3",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/types": {
+ "version": "4.3.1",
+ "resolved": "https://registry.npmjs.org/@smithy/types/-/types-4.3.1.tgz",
+ "integrity": "sha512-UqKOQBL2x6+HWl3P+3QqFD4ncKq0I8Nuz9QItGv5WuKuMHuuwlhvqcZCoXGfc+P1QmfJE7VieykoYYmrOoFJxA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/url-parser": {
+ "version": "4.0.4",
+ "resolved": "https://registry.npmjs.org/@smithy/url-parser/-/url-parser-4.0.4.tgz",
+ "integrity": "sha512-eMkc144MuN7B0TDA4U2fKs+BqczVbk3W+qIvcoCY6D1JY3hnAdCuhCZODC+GAeaxj0p6Jroz4+XMUn3PCxQQeQ==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/querystring-parser": "^4.0.4",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/util-base64": {
+ "version": "4.0.0",
+ "resolved": "https://registry.npmjs.org/@smithy/util-base64/-/util-base64-4.0.0.tgz",
+ "integrity": "sha512-CvHfCmO2mchox9kjrtzoHkWHxjHZzaFojLc8quxXY7WAAMAg43nuxwv95tATVgQFNDwd4M9S1qFzj40Ul41Kmg==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/util-buffer-from": "^4.0.0",
+ "@smithy/util-utf8": "^4.0.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/util-body-length-browser": {
+ "version": "4.0.0",
+ "resolved": "https://registry.npmjs.org/@smithy/util-body-length-browser/-/util-body-length-browser-4.0.0.tgz",
+ "integrity": "sha512-sNi3DL0/k64/LO3A256M+m3CDdG6V7WKWHdAiBBMUN8S3hK3aMPhwnPik2A/a2ONN+9doY9UxaLfgqsIRg69QA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/util-body-length-node": {
+ "version": "4.0.0",
+ "resolved": "https://registry.npmjs.org/@smithy/util-body-length-node/-/util-body-length-node-4.0.0.tgz",
+ "integrity": "sha512-q0iDP3VsZzqJyje8xJWEJCNIu3lktUGVoSy1KB0UWym2CL1siV3artm+u1DFYTLejpsrdGyCSWBdGNjJzfDPjg==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/util-buffer-from": {
+ "version": "4.0.0",
+ "resolved": "https://registry.npmjs.org/@smithy/util-buffer-from/-/util-buffer-from-4.0.0.tgz",
+ "integrity": "sha512-9TOQ7781sZvddgO8nxueKi3+yGvkY35kotA0Y6BWRajAv8jjmigQ1sBwz0UX47pQMYXJPahSKEKYFgt+rXdcug==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/is-array-buffer": "^4.0.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/util-config-provider": {
+ "version": "4.0.0",
+ "resolved": "https://registry.npmjs.org/@smithy/util-config-provider/-/util-config-provider-4.0.0.tgz",
+ "integrity": "sha512-L1RBVzLyfE8OXH+1hsJ8p+acNUSirQnWQ6/EgpchV88G6zGBTDPdXiiExei6Z1wR2RxYvxY/XLw6AMNCCt8H3w==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/util-defaults-mode-browser": {
+ "version": "4.0.24",
+ "resolved": "https://registry.npmjs.org/@smithy/util-defaults-mode-browser/-/util-defaults-mode-browser-4.0.24.tgz",
+ "integrity": "sha512-UkQNgaQ+bidw1MgdgPO1z1k95W/v8Ej/5o/T/Is8PiVUYPspl/ZxV6WO/8DrzZQu5ULnmpB9CDdMSRwgRc21AA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/property-provider": "^4.0.4",
+ "@smithy/smithy-client": "^4.4.8",
+ "@smithy/types": "^4.3.1",
+ "bowser": "^2.11.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/util-defaults-mode-node": {
+ "version": "4.0.24",
+ "resolved": "https://registry.npmjs.org/@smithy/util-defaults-mode-node/-/util-defaults-mode-node-4.0.24.tgz",
+ "integrity": "sha512-phvGi/15Z4MpuQibTLOYIumvLdXb+XIJu8TA55voGgboln85jytA3wiD7CkUE8SNcWqkkb+uptZKPiuFouX/7g==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/config-resolver": "^4.1.4",
+ "@smithy/credential-provider-imds": "^4.0.6",
+ "@smithy/node-config-provider": "^4.1.3",
+ "@smithy/property-provider": "^4.0.4",
+ "@smithy/smithy-client": "^4.4.8",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/util-endpoints": {
+ "version": "3.0.6",
+ "resolved": "https://registry.npmjs.org/@smithy/util-endpoints/-/util-endpoints-3.0.6.tgz",
+ "integrity": "sha512-YARl3tFL3WgPuLzljRUnrS2ngLiUtkwhQtj8PAL13XZSyUiNLQxwG3fBBq3QXFqGFUXepIN73pINp3y8c2nBmA==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/node-config-provider": "^4.1.3",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/util-hex-encoding": {
+ "version": "4.0.0",
+ "resolved": "https://registry.npmjs.org/@smithy/util-hex-encoding/-/util-hex-encoding-4.0.0.tgz",
+ "integrity": "sha512-Yk5mLhHtfIgW2W2WQZWSg5kuMZCVbvhFmC7rV4IO2QqnZdbEFPmQnCcGMAX2z/8Qj3B9hYYNjZOhWym+RwhePw==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/util-middleware": {
+ "version": "4.0.4",
+ "resolved": "https://registry.npmjs.org/@smithy/util-middleware/-/util-middleware-4.0.4.tgz",
+ "integrity": "sha512-9MLKmkBmf4PRb0ONJikCbCwORACcil6gUWojwARCClT7RmLzF04hUR4WdRprIXal7XVyrddadYNfp2eF3nrvtQ==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/util-retry": {
+ "version": "4.0.6",
+ "resolved": "https://registry.npmjs.org/@smithy/util-retry/-/util-retry-4.0.6.tgz",
+ "integrity": "sha512-+YekoF2CaSMv6zKrA6iI/N9yva3Gzn4L6n35Luydweu5MMPYpiGZlWqehPHDHyNbnyaYlz/WJyYAZnC+loBDZg==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/service-error-classification": "^4.0.6",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/util-stream": {
+ "version": "4.2.3",
+ "resolved": "https://registry.npmjs.org/@smithy/util-stream/-/util-stream-4.2.3.tgz",
+ "integrity": "sha512-cQn412DWHHFNKrQfbHY8vSFI3nTROY1aIKji9N0tpp8gUABRilr7wdf8fqBbSlXresobM+tQFNk6I+0LXK/YZg==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/fetch-http-handler": "^5.1.0",
+ "@smithy/node-http-handler": "^4.1.0",
+ "@smithy/types": "^4.3.1",
+ "@smithy/util-base64": "^4.0.0",
+ "@smithy/util-buffer-from": "^4.0.0",
+ "@smithy/util-hex-encoding": "^4.0.0",
+ "@smithy/util-utf8": "^4.0.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/util-uri-escape": {
+ "version": "4.0.0",
+ "resolved": "https://registry.npmjs.org/@smithy/util-uri-escape/-/util-uri-escape-4.0.0.tgz",
+ "integrity": "sha512-77yfbCbQMtgtTylO9itEAdpPXSog3ZxMe09AEhm0dU0NLTalV70ghDZFR+Nfi1C60jnJoh/Re4090/DuZh2Omg==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/util-utf8": {
+ "version": "4.0.0",
+ "resolved": "https://registry.npmjs.org/@smithy/util-utf8/-/util-utf8-4.0.0.tgz",
+ "integrity": "sha512-b+zebfKCfRdgNJDknHCob3O7FpeYQN6ZG6YLExMcasDHsCXlsXCEuiPZeLnJLpwa5dvPetGlnGCiMHuLwGvFow==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/util-buffer-from": "^4.0.0",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
+ "node_modules/@smithy/util-waiter": {
+ "version": "4.0.6",
+ "resolved": "https://registry.npmjs.org/@smithy/util-waiter/-/util-waiter-4.0.6.tgz",
+ "integrity": "sha512-slcr1wdRbX7NFphXZOxtxRNA7hXAAtJAXJDE/wdoMAos27SIquVCKiSqfB6/28YzQ8FCsB5NKkhdM5gMADbqxg==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "@smithy/abort-controller": "^4.0.4",
+ "@smithy/types": "^4.3.1",
+ "tslib": "^2.6.2"
+ },
+ "engines": {
+ "node": ">=18.0.0"
+ }
+ },
"node_modules/@tootallnate/once": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/@tootallnate/once/-/once-1.1.2.tgz",
@@ -1465,6 +3138,12 @@
"integrity": "sha512-6WaYesThRMCl19iryMYP7/x2OVgCtbIVflDGFpWnb9irXI3UjYE4AzmYuiUKY1AJstGijoY+MgUszMgRxIYTYw==",
"license": "MIT"
},
+ "node_modules/@types/uuid": {
+ "version": "9.0.8",
+ "resolved": "https://registry.npmjs.org/@types/uuid/-/uuid-9.0.8.tgz",
+ "integrity": "sha512-jg+97EGIcY9AGHJJRaaPVgetKDsrTgbRjQ5Msgjh/DQKEFl0DtyRr/VCOyD1T2R1MNeWPK/u7JoGhlDZnKBAfA==",
+ "license": "MIT"
+ },
"node_modules/@types/yargs": {
"version": "17.0.33",
"resolved": "https://registry.npmjs.org/@types/yargs/-/yargs-17.0.33.tgz",
@@ -1508,6 +3187,27 @@
"node": ">= 0.6"
}
},
+ "node_modules/accepts/node_modules/mime-db": {
+ "version": "1.52.0",
+ "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz",
+ "integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==",
+ "license": "MIT",
+ "engines": {
+ "node": ">= 0.6"
+ }
+ },
+ "node_modules/accepts/node_modules/mime-types": {
+ "version": "2.1.35",
+ "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz",
+ "integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==",
+ "license": "MIT",
+ "dependencies": {
+ "mime-db": "1.52.0"
+ },
+ "engines": {
+ "node": ">= 0.6"
+ }
+ },
"node_modules/acorn": {
"version": "8.15.0",
"resolved": "https://registry.npmjs.org/acorn/-/acorn-8.15.0.tgz",
@@ -1762,7 +3462,6 @@
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/argparse/-/argparse-2.0.1.tgz",
"integrity": "sha512-8+9WqebbFzpX9OR+Wa6O29asIogeRMzcGtAINdpMHHyAg10f05aSFVBbcEqGf/PXw1EjAZ+q2/bEBg3DvurK3Q==",
- "dev": true,
"license": "Python-2.0"
},
"node_modules/array-flatten": {
@@ -2116,6 +3815,12 @@
"integrity": "sha512-Tpp60P6IUJDTuOq/5Z8cdskzJujfwqfOTkrwIwj7IRISpnkJnT6SyJ4PCPnGMoFjC9ddhal5KVIYtAt97ix05A==",
"license": "MIT"
},
+ "node_modules/bowser": {
+ "version": "2.11.0",
+ "resolved": "https://registry.npmjs.org/bowser/-/bowser-2.11.0.tgz",
+ "integrity": "sha512-AlcaJBi/pqqJBIQ8U9Mcpc9i8Aqxn88Skv5d+xBX006BY5u8N3mGLHa5Lgppa7L/HfwgwLgZ6NYs+Ag6uUmJRA==",
+ "license": "MIT"
+ },
"node_modules/brace-expansion": {
"version": "1.1.12",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.12.tgz",
@@ -3379,6 +5084,15 @@
"node": ">= 0.6"
}
},
+ "node_modules/events": {
+ "version": "3.3.0",
+ "resolved": "https://registry.npmjs.org/events/-/events-3.3.0.tgz",
+ "integrity": "sha512-mQw+2fkQbALzQ7V0MY0IqdnXNOeTtP4r0lN9z7AAawCXgqea7bDii20AYrIBrFd/Hx0M2Ocz6S111CaFkUcb0Q==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=0.8.x"
+ }
+ },
"node_modules/execa": {
"version": "5.1.1",
"resolved": "https://registry.npmjs.org/execa/-/execa-5.1.1.tgz",
@@ -3558,6 +5272,24 @@
"dev": true,
"license": "MIT"
},
+ "node_modules/fast-xml-parser": {
+ "version": "5.2.5",
+ "resolved": "https://registry.npmjs.org/fast-xml-parser/-/fast-xml-parser-5.2.5.tgz",
+ "integrity": "sha512-pfX9uG9Ki0yekDHx2SiuRIyFdyAr1kMIMitPvb0YBo8SUfKvia7w7FIyd/l6av85pFYRhZscS75MwMnbvY+hcQ==",
+ "funding": [
+ {
+ "type": "github",
+ "url": "https://github.com/sponsors/NaturalIntelligence"
+ }
+ ],
+ "license": "MIT",
+ "dependencies": {
+ "strnum": "^2.1.0"
+ },
+ "bin": {
+ "fxparser": "src/cli/cli.js"
+ }
+ },
"node_modules/fastq": {
"version": "1.19.1",
"resolved": "https://registry.npmjs.org/fastq/-/fastq-1.19.1.tgz",
@@ -3729,6 +5461,27 @@
"node": ">= 6"
}
},
+ "node_modules/form-data/node_modules/mime-db": {
+ "version": "1.52.0",
+ "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz",
+ "integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==",
+ "license": "MIT",
+ "engines": {
+ "node": ">= 0.6"
+ }
+ },
+ "node_modules/form-data/node_modules/mime-types": {
+ "version": "2.1.35",
+ "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz",
+ "integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==",
+ "license": "MIT",
+ "dependencies": {
+ "mime-db": "1.52.0"
+ },
+ "engines": {
+ "node": ">= 0.6"
+ }
+ },
"node_modules/formidable": {
"version": "2.1.5",
"resolved": "https://registry.npmjs.org/formidable/-/formidable-2.1.5.tgz",
@@ -5204,7 +6957,6 @@
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.1.0.tgz",
"integrity": "sha512-wpxZs9NoxZaJESJGIZTyDEaYpl0FKSA+FB9aJiyemKhMwkxQg63h4T1KJgUGHpTqPDNRcmmYLugrRjJlBtWvRA==",
- "dev": true,
"license": "MIT",
"dependencies": {
"argparse": "^2.0.1"
@@ -5773,21 +7525,21 @@
}
},
"node_modules/mime-db": {
- "version": "1.52.0",
- "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz",
- "integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==",
+ "version": "1.54.0",
+ "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.54.0.tgz",
+ "integrity": "sha512-aU5EJuIN2WDemCcAp2vFBfp/m4EAhWJnUNSSw0ixs7/kXbd6Pg64EmwJkNdFhB8aWt1sH2CTXrLxo/iAGV3oPQ==",
"license": "MIT",
"engines": {
"node": ">= 0.6"
}
},
"node_modules/mime-types": {
- "version": "2.1.35",
- "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz",
- "integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==",
+ "version": "3.0.1",
+ "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-3.0.1.tgz",
+ "integrity": "sha512-xRc4oEhT6eaBpU1XF7AjpOFD+xQmXNB5OVKwp4tqCuBpHLS/ZbBDrc07mYTDqVMg6PfxUjjNp85O6Cd2Z/5HWA==",
"license": "MIT",
"dependencies": {
- "mime-db": "1.52.0"
+ "mime-db": "^1.54.0"
},
"engines": {
"node": ">= 0.6"
@@ -7750,6 +9502,16 @@
"node": ">= 0.8"
}
},
+ "node_modules/stream-browserify": {
+ "version": "3.0.0",
+ "resolved": "https://registry.npmjs.org/stream-browserify/-/stream-browserify-3.0.0.tgz",
+ "integrity": "sha512-H73RAHsVBapbim0tU2JwwOiXUj+fikfiaoYAKHF3VJfA0pe2BCzkhAHBlLG6REzE+2WNZcxOXjK7lkso+9euLA==",
+ "license": "MIT",
+ "dependencies": {
+ "inherits": "~2.0.4",
+ "readable-stream": "^3.5.0"
+ }
+ },
"node_modules/streamsearch": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/streamsearch/-/streamsearch-1.1.0.tgz",
@@ -7853,6 +9615,18 @@
"url": "https://github.com/sponsors/sindresorhus"
}
},
+ "node_modules/strnum": {
+ "version": "2.1.1",
+ "resolved": "https://registry.npmjs.org/strnum/-/strnum-2.1.1.tgz",
+ "integrity": "sha512-7ZvoFTiCnGxBtDqJ//Cu6fWtZtc7Y3x+QOirG15wztbdngGSkht27o2pyGWrVy0b4WAy3jbKmnoK6g5VlVNUUw==",
+ "funding": [
+ {
+ "type": "github",
+ "url": "https://github.com/sponsors/NaturalIntelligence"
+ }
+ ],
+ "license": "MIT"
+ },
"node_modules/superagent": {
"version": "8.1.2",
"resolved": "https://registry.npmjs.org/superagent/-/superagent-8.1.2.tgz",
@@ -8121,6 +9895,12 @@
"node": ">= 14.0.0"
}
},
+ "node_modules/tslib": {
+ "version": "2.8.1",
+ "resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz",
+ "integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==",
+ "license": "0BSD"
+ },
"node_modules/tunnel-agent": {
"version": "0.6.0",
"resolved": "https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz",
@@ -8182,6 +9962,27 @@
"node": ">= 0.6"
}
},
+ "node_modules/type-is/node_modules/mime-db": {
+ "version": "1.52.0",
+ "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz",
+ "integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==",
+ "license": "MIT",
+ "engines": {
+ "node": ">= 0.6"
+ }
+ },
+ "node_modules/type-is/node_modules/mime-types": {
+ "version": "2.1.35",
+ "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz",
+ "integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==",
+ "license": "MIT",
+ "dependencies": {
+ "mime-db": "1.52.0"
+ },
+ "engines": {
+ "node": ">= 0.6"
+ }
+ },
"node_modules/typedarray": {
"version": "0.0.6",
"resolved": "https://registry.npmjs.org/typedarray/-/typedarray-0.0.6.tgz",
diff --git a/backend/package.json b/backend/package.json
index 8d69f4f..ca86f03 100644
--- a/backend/package.json
+++ b/backend/package.json
@@ -10,9 +10,14 @@
"migrate:safe": "node migrations/run-migrations-safe.js",
"fix-temp-photos": "node scripts/fix-temp-photos.js",
"test": "jest",
- "lint": "eslint src/"
+ "lint": "eslint src/",
+ "test-backup": "node scripts/test-backup-service.js",
+ "test-restore": "node scripts/test-restore-service.js"
},
"dependencies": {
+ "@aws-sdk/client-s3": "^3.850.0",
+ "@aws-sdk/lib-storage": "^3.850.0",
+ "@aws-sdk/s3-request-presigner": "^3.850.0",
"adm-zip": "^0.5.16",
"archiver": "^5.3.1",
"axios": "^1.10.0",
@@ -30,8 +35,10 @@
"i18next-browser-languagedetector": "^8.2.0",
"i18next-http-backend": "^3.0.2",
"joi": "^17.9.1",
+ "js-yaml": "^4.1.0",
"jsonwebtoken": "^9.0.0",
"knex": "^2.4.2",
+ "mime-types": "^3.0.1",
"multer": "^2.0.1",
"node-cron": "^3.0.2",
"nodemailer": "^6.9.1",
diff --git a/backend/scripts/test-backup-integration.js b/backend/scripts/test-backup-integration.js
new file mode 100755
index 0000000..289b3fc
--- /dev/null
+++ b/backend/scripts/test-backup-integration.js
@@ -0,0 +1,577 @@
+#!/usr/bin/env node
+
+/**
+ * Manual Integration Test Script for Enhanced Backup System
+ *
+ * This script provides a comprehensive test of the backup system with real services.
+ * It can be used to test against MinIO, AWS S3, or other S3-compatible services.
+ *
+ * Usage:
+ * node scripts/test-backup-integration.js [options]
+ *
+ * Options:
+ * --endpoint S3 endpoint URL (default: http://localhost:9000)
+ * --access-key S3 access key (default: minioadmin)
+ * --secret-key S3 secret key (default: minioadmin)
+ * --bucket S3 bucket name (default: test-backup-)
+ * --type Backup type: s3, local, rsync (default: s3)
+ * --cleanup Clean up test data after completion
+ * --verbose Enable verbose logging
+ * --help Show this help message
+ *
+ * Examples:
+ * # Test with local MinIO
+ * node scripts/test-backup-integration.js
+ *
+ * # Test with AWS S3
+ * node scripts/test-backup-integration.js \
+ * --endpoint https://s3.amazonaws.com \
+ * --access-key AKIAIOSFODNN7EXAMPLE \
+ * --secret-key wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY \
+ * --bucket my-test-bucket
+ *
+ * # Test local backup
+ * node scripts/test-backup-integration.js --type local
+ */
+
+const path = require('path');
+const fs = require('fs').promises;
+const crypto = require('crypto');
+const { S3Client, CreateBucketCommand, HeadBucketCommand, ListObjectsV2Command, GetObjectCommand, DeleteObjectsCommand, DeleteBucketCommand } = require('@aws-sdk/client-s3');
+
+// Parse command line arguments
+const args = process.argv.slice(2);
+const options = {
+ endpoint: 'http://localhost:9000',
+ accessKey: 'minioadmin',
+ secretKey: 'minioadmin',
+ bucket: `test-backup-${Date.now()}`,
+ type: 's3',
+ cleanup: false,
+ verbose: false
+};
+
+for (let i = 0; i < args.length; i++) {
+ switch (args[i]) {
+ case '--endpoint':
+ options.endpoint = args[++i];
+ break;
+ case '--access-key':
+ options.accessKey = args[++i];
+ break;
+ case '--secret-key':
+ options.secretKey = args[++i];
+ break;
+ case '--bucket':
+ options.bucket = args[++i];
+ break;
+ case '--type':
+ options.type = args[++i];
+ break;
+ case '--cleanup':
+ options.cleanup = true;
+ break;
+ case '--verbose':
+ options.verbose = true;
+ break;
+ case '--help':
+ console.log(module.exports.description || 'Manual Integration Test Script');
+ process.exit(0);
+ }
+}
+
+// Load environment and services
+require('dotenv').config();
+const { db, initialize: initDb } = require('../src/database/db');
+const backupService = require('../src/services/backupService');
+const S3StorageAdapter = require('../src/services/storage/s3Storage');
+const logger = require('../src/utils/logger');
+
+// Configure logger based on verbose flag
+if (!options.verbose) {
+ logger.info = () => {};
+ logger.debug = () => {};
+}
+
+// Test results
+const results = {
+ passed: 0,
+ failed: 0,
+ skipped: 0,
+ tests: []
+};
+
+// Test utilities
+async function runTest(name, testFn) {
+ console.log(`\n📋 Running: ${name}`);
+ try {
+ const startTime = Date.now();
+ await testFn();
+ const duration = Date.now() - startTime;
+ console.log(`✅ PASSED: ${name} (${duration}ms)`);
+ results.passed++;
+ results.tests.push({ name, status: 'passed', duration });
+ } catch (error) {
+ console.error(`❌ FAILED: ${name}`);
+ console.error(` Error: ${error.message}`);
+ if (options.verbose) {
+ console.error(error.stack);
+ }
+ results.failed++;
+ results.tests.push({ name, status: 'failed', error: error.message });
+ }
+}
+
+async function skipTest(name, reason) {
+ console.log(`\n⏭️ Skipping: ${name}`);
+ console.log(` Reason: ${reason}`);
+ results.skipped++;
+ results.tests.push({ name, status: 'skipped', reason });
+}
+
+// Test functions
+async function testS3Connection() {
+ const s3Adapter = new S3StorageAdapter({
+ bucket: options.bucket,
+ endpoint: options.endpoint,
+ accessKeyId: options.accessKey,
+ secretAccessKey: options.secretKey,
+ region: 'us-east-1',
+ forcePathStyle: true,
+ sslEnabled: options.endpoint.startsWith('https')
+ });
+
+ await s3Adapter.testConnection();
+ console.log(` ✓ Connected to S3 endpoint: ${options.endpoint}`);
+ console.log(` ✓ Bucket accessible: ${options.bucket}`);
+}
+
+async function setupTestData() {
+ const storagePath = path.join(__dirname, '../test-storage');
+ process.env.STORAGE_PATH = storagePath;
+
+ // Create directory structure
+ const dirs = [
+ 'events/active/wedding-2024',
+ 'events/active/birthday-2024',
+ 'events/archived',
+ 'thumbnails',
+ 'uploads',
+ 'backups'
+ ];
+
+ for (const dir of dirs) {
+ await fs.mkdir(path.join(storagePath, dir), { recursive: true });
+ }
+
+ // Create test files with various sizes
+ const files = [
+ { path: 'events/active/wedding-2024/photo1.jpg', size: 1024 * 1024 }, // 1MB
+ { path: 'events/active/wedding-2024/photo2.jpg', size: 512 * 1024 }, // 512KB
+ { path: 'events/active/birthday-2024/photo1.jpg', size: 2 * 1024 * 1024 }, // 2MB
+ { path: 'events/archived/old-event.zip', size: 5 * 1024 * 1024 }, // 5MB
+ { path: 'thumbnails/thumb1.jpg', size: 50 * 1024 }, // 50KB
+ { path: 'uploads/logo.png', size: 100 * 1024 } // 100KB
+ ];
+
+ let totalSize = 0;
+ for (const file of files) {
+ const content = crypto.randomBytes(file.size);
+ await fs.writeFile(path.join(storagePath, file.path), content);
+ totalSize += file.size;
+ }
+
+ console.log(` ✓ Created ${files.length} test files`);
+ console.log(` ✓ Total size: ${(totalSize / 1024 / 1024).toFixed(2)} MB`);
+
+ return { storagePath, fileCount: files.length, totalSize };
+}
+
+async function configureBackup(type) {
+ const baseSettings = [
+ { setting_key: 'backup_enabled', setting_value: 'true' },
+ { setting_key: 'backup_destination_type', setting_value: `"${type}"` },
+ { setting_key: 'backup_include_archived', setting_value: 'true' },
+ { setting_key: 'backup_include_database', setting_value: 'true' },
+ { setting_key: 'backup_incremental', setting_value: 'true' },
+ { setting_key: 'backup_manifest_format', setting_value: '"json"' },
+ { setting_key: 'backup_max_file_size_mb', setting_value: '100' }
+ ];
+
+ const typeSpecificSettings = {
+ s3: [
+ { setting_key: 'backup_s3_bucket', setting_value: `"${options.bucket}"` },
+ { setting_key: 'backup_s3_endpoint', setting_value: `"${options.endpoint}"` },
+ { setting_key: 'backup_s3_access_key', setting_value: `"${options.accessKey}"` },
+ { setting_key: 'backup_s3_secret_key', setting_value: `"${options.secretKey}"` },
+ { setting_key: 'backup_s3_region', setting_value: '"us-east-1"' },
+ { setting_key: 'backup_s3_force_path_style', setting_value: 'true' },
+ { setting_key: 'backup_s3_ssl_enabled', setting_value: options.endpoint.startsWith('https') ? 'true' : 'false' }
+ ],
+ local: [
+ { setting_key: 'backup_destination_path', setting_value: `"${path.join(__dirname, '../test-backup')}"` }
+ ],
+ rsync: [
+ { setting_key: 'backup_rsync_host', setting_value: '"localhost"' },
+ { setting_key: 'backup_rsync_path', setting_value: `"${path.join(__dirname, '../test-backup-rsync')}"` }
+ ]
+ };
+
+ const settings = [...baseSettings, ...(typeSpecificSettings[type] || [])];
+
+ // Clear existing settings
+ await db('app_settings').where('setting_type', 'backup').del();
+
+ // Insert new settings
+ for (const setting of settings) {
+ await db('app_settings').insert({
+ setting_type: 'backup',
+ ...setting,
+ created_at: new Date(),
+ updated_at: new Date()
+ });
+ }
+
+ console.log(` ✓ Configured ${type} backup with ${settings.length} settings`);
+}
+
+async function performBackup() {
+ const startTime = Date.now();
+
+ // Run the backup
+ await backupService.runBackup();
+
+ // Get backup results
+ const backupRun = await db('backup_runs')
+ .orderBy('started_at', 'desc')
+ .first();
+
+ if (!backupRun) {
+ throw new Error('No backup run found');
+ }
+
+ if (backupRun.status !== 'completed') {
+ throw new Error(`Backup failed with status: ${backupRun.status}, error: ${backupRun.error_message}`);
+ }
+
+ const duration = Date.now() - startTime;
+
+ console.log(` ✓ Backup completed in ${duration}ms`);
+ console.log(` ✓ Files backed up: ${backupRun.files_backed_up}`);
+ console.log(` ✓ Total size: ${(backupRun.total_size_bytes / 1024 / 1024).toFixed(2)} MB`);
+ console.log(` ✓ Manifest: ${backupRun.manifest_path ? 'Generated' : 'Not generated'}`);
+
+ return backupRun;
+}
+
+async function verifyS3Backup(backupRun) {
+ const s3Client = new S3Client({
+ endpoint: options.endpoint,
+ region: 'us-east-1',
+ credentials: {
+ accessKeyId: options.accessKey,
+ secretAccessKey: options.secretKey
+ },
+ forcePathStyle: true
+ });
+
+ // List objects in bucket
+ const listResponse = await s3Client.send(new ListObjectsV2Command({
+ Bucket: options.bucket
+ }));
+
+ const objects = listResponse.Contents || [];
+ console.log(` ✓ Objects in S3: ${objects.length}`);
+
+ // Verify key components
+ const hasBackupFolder = objects.some(obj => obj.Key.includes('backup-'));
+ const hasManifest = objects.some(obj => obj.Key.includes('backup-manifest'));
+ const hasSummary = objects.some(obj => obj.Key.includes('backup-summary.json'));
+ const hasPhotos = objects.some(obj => obj.Key.includes('events/active'));
+
+ if (!hasBackupFolder) throw new Error('No backup folder found in S3');
+ if (!hasManifest) throw new Error('No manifest found in S3');
+ if (!hasSummary) throw new Error('No summary found in S3');
+ if (!hasPhotos) throw new Error('No photos found in S3');
+
+ console.log(` ✓ Backup structure verified`);
+
+ // Download and verify a file
+ const photoObject = objects.find(obj => obj.Key.includes('photo1.jpg'));
+ if (photoObject) {
+ const getResponse = await s3Client.send(new GetObjectCommand({
+ Bucket: options.bucket,
+ Key: photoObject.Key
+ }));
+
+ const chunks = [];
+ for await (const chunk of getResponse.Body) {
+ chunks.push(chunk);
+ }
+ const content = Buffer.concat(chunks);
+
+ console.log(` ✓ Downloaded test file: ${photoObject.Key} (${content.length} bytes)`);
+ }
+}
+
+async function testIncrementalBackup(testData) {
+ // Modify a file
+ const modifiedFile = path.join(testData.storagePath, 'events/active/wedding-2024/photo1.jpg');
+ const newContent = crypto.randomBytes(1024 * 1024 + 100); // Slightly larger
+ await fs.writeFile(modifiedFile, newContent);
+
+ console.log(` ✓ Modified test file`);
+
+ // Perform incremental backup
+ const backupRun = await performBackup();
+
+ if (backupRun.files_backed_up !== 1) {
+ throw new Error(`Expected 1 file in incremental backup, got ${backupRun.files_backed_up}`);
+ }
+
+ console.log(` ✓ Incremental backup correctly identified changed file`);
+
+ // Verify manifest indicates incremental
+ if (backupRun.manifest_path) {
+ const { manifest } = await backupService.getBackupManifest(backupRun.id);
+ if (!manifest.incremental) {
+ throw new Error('Manifest does not indicate incremental backup');
+ }
+ console.log(` ✓ Manifest correctly marked as incremental`);
+ }
+
+ return backupRun;
+}
+
+async function testManifestValidation(backupRun) {
+ if (!backupRun.manifest_path) {
+ throw new Error('No manifest path in backup run');
+ }
+
+ const result = await backupService.validateBackupManifest(backupRun.manifest_path);
+
+ if (!result.valid) {
+ throw new Error(`Manifest validation failed: ${result.error}`);
+ }
+
+ console.log(` ✓ Manifest validation passed`);
+ console.log(` ✓ Manifest version: ${result.manifest.manifest.version}`);
+ console.log(` ✓ Files in manifest: ${result.manifest.files.count}`);
+}
+
+async function testBackupStatus() {
+ const status = await backupService.getBackupStatus(5);
+
+ console.log(` ✓ Backup service running: ${status.isRunning}`);
+ console.log(` ✓ Backup service healthy: ${status.isHealthy}`);
+ console.log(` ✓ Recent runs: ${status.recentRuns.length}`);
+
+ if (status.lastRun) {
+ console.log(` ✓ Last run status: ${status.lastRun.status}`);
+ console.log(` ✓ Manifest valid: ${status.lastRun.manifestValid}`);
+ }
+}
+
+async function cleanupTestData() {
+ if (!options.cleanup) {
+ console.log('\n📌 Test data retained for inspection');
+ console.log(` Storage: ${process.env.STORAGE_PATH}`);
+ if (options.type === 's3') {
+ console.log(` S3 Bucket: ${options.bucket}`);
+ }
+ return;
+ }
+
+ console.log('\n🧹 Cleaning up test data...');
+
+ // Clean storage directory
+ if (process.env.STORAGE_PATH) {
+ await fs.rm(process.env.STORAGE_PATH, { recursive: true, force: true });
+ console.log(' ✓ Removed test storage directory');
+ }
+
+ // Clean S3 bucket if used
+ if (options.type === 's3') {
+ const s3Client = new S3Client({
+ endpoint: options.endpoint,
+ region: 'us-east-1',
+ credentials: {
+ accessKeyId: options.accessKey,
+ secretAccessKey: options.secretKey
+ },
+ forcePathStyle: true
+ });
+
+ try {
+ // List and delete all objects
+ const listResponse = await s3Client.send(new ListObjectsV2Command({
+ Bucket: options.bucket
+ }));
+
+ if (listResponse.Contents && listResponse.Contents.length > 0) {
+ await s3Client.send(new DeleteObjectsCommand({
+ Bucket: options.bucket,
+ Delete: {
+ Objects: listResponse.Contents.map(obj => ({ Key: obj.Key }))
+ }
+ }));
+ console.log(` ✓ Deleted ${listResponse.Contents.length} objects from S3`);
+ }
+
+ // Delete bucket
+ await s3Client.send(new DeleteBucketCommand({
+ Bucket: options.bucket
+ }));
+ console.log(` ✓ Deleted S3 bucket: ${options.bucket}`);
+ } catch (error) {
+ console.error(` ⚠️ Failed to cleanup S3: ${error.message}`);
+ }
+ }
+
+ // Clean backup directories
+ const backupDirs = [
+ path.join(__dirname, '../test-backup'),
+ path.join(__dirname, '../test-backup-rsync')
+ ];
+
+ for (const dir of backupDirs) {
+ await fs.rm(dir, { recursive: true, force: true }).catch(() => {});
+ }
+ console.log(' ✓ Removed backup directories');
+}
+
+// Main test runner
+async function main() {
+ console.log('🚀 Enhanced Backup System Integration Test');
+ console.log('==========================================');
+ console.log(`Type: ${options.type}`);
+ console.log(`Endpoint: ${options.endpoint}`);
+ console.log(`Bucket: ${options.bucket}`);
+ console.log('');
+
+ let s3Client;
+ let testData;
+
+ try {
+ // Initialize database
+ console.log('📦 Initializing database...');
+ await initDb();
+ await db.migrate.latest();
+ console.log(' ✓ Database initialized');
+
+ // S3-specific setup
+ if (options.type === 's3') {
+ // Test S3 connection
+ await runTest('S3 Connection Test', testS3Connection);
+
+ // Create S3 bucket if needed
+ s3Client = new S3Client({
+ endpoint: options.endpoint,
+ region: 'us-east-1',
+ credentials: {
+ accessKeyId: options.accessKey,
+ secretAccessKey: options.secretKey
+ },
+ forcePathStyle: true
+ });
+
+ try {
+ await s3Client.send(new HeadBucketCommand({ Bucket: options.bucket }));
+ console.log(`\n📦 Using existing bucket: ${options.bucket}`);
+ } catch (error) {
+ if (error.name === 'NotFound') {
+ await s3Client.send(new CreateBucketCommand({ Bucket: options.bucket }));
+ console.log(`\n📦 Created new bucket: ${options.bucket}`);
+ } else {
+ throw error;
+ }
+ }
+ }
+
+ // Setup test data
+ console.log('\n📁 Setting up test data...');
+ testData = await setupTestData();
+
+ // Configure backup
+ console.log(`\n⚙️ Configuring ${options.type} backup...`);
+ await configureBackup(options.type);
+
+ // Run tests based on backup type
+ await runTest('Initial Full Backup', performBackup);
+
+ if (options.type === 's3') {
+ await runTest('Verify S3 Backup Contents', async () => {
+ const lastRun = await db('backup_runs').orderBy('started_at', 'desc').first();
+ await verifyS3Backup(lastRun);
+ });
+ }
+
+ await runTest('Incremental Backup', () => testIncrementalBackup(testData));
+
+ await runTest('Manifest Validation', async () => {
+ const lastRun = await db('backup_runs').orderBy('started_at', 'desc').first();
+ await testManifestValidation(lastRun);
+ });
+
+ await runTest('Backup Status Check', testBackupStatus);
+
+ // Performance test with larger files
+ if (options.type === 's3') {
+ await runTest('Large File Backup (10MB)', async () => {
+ const largeFile = path.join(testData.storagePath, 'events/active/large.jpg');
+ await fs.writeFile(largeFile, crypto.randomBytes(10 * 1024 * 1024));
+ await performBackup();
+ });
+ }
+
+ // Test backup service lifecycle
+ await runTest('Backup Service Start/Stop', async () => {
+ await backupService.startBackupService();
+ console.log(' ✓ Service started');
+
+ backupService.stopBackupService();
+ console.log(' ✓ Service stopped');
+ });
+
+ // Print results summary
+ console.log('\n📊 Test Results Summary');
+ console.log('======================');
+ console.log(`✅ Passed: ${results.passed}`);
+ console.log(`❌ Failed: ${results.failed}`);
+ console.log(`⏭️ Skipped: ${results.skipped}`);
+ console.log(`📋 Total: ${results.tests.length}`);
+
+ if (results.failed > 0) {
+ console.log('\nFailed Tests:');
+ results.tests
+ .filter(t => t.status === 'failed')
+ .forEach(t => console.log(` - ${t.name}: ${t.error}`));
+ }
+
+ } catch (error) {
+ console.error('\n💥 Fatal error:', error.message);
+ if (options.verbose) {
+ console.error(error.stack);
+ }
+ results.failed++;
+ } finally {
+ // Cleanup
+ await cleanupTestData();
+
+ // Close database
+ await db.destroy();
+
+ // Exit with appropriate code
+ process.exit(results.failed > 0 ? 1 : 0);
+ }
+}
+
+// Run if called directly
+if (require.main === module) {
+ main().catch(error => {
+ console.error('Unhandled error:', error);
+ process.exit(1);
+ });
+}
+
+module.exports = { runTest, skipTest };
\ No newline at end of file
diff --git a/backend/scripts/test-backup-manifest.js b/backend/scripts/test-backup-manifest.js
new file mode 100644
index 0000000..ca9669e
--- /dev/null
+++ b/backend/scripts/test-backup-manifest.js
@@ -0,0 +1,192 @@
+#!/usr/bin/env node
+
+/**
+ * Test script for backup manifest generator
+ * Demonstrates all features of the manifest generator
+ */
+
+const path = require('path');
+const fs = require('fs').promises;
+const backupManifest = require('../src/services/backupManifest');
+const logger = require('../src/utils/logger');
+
+async function testManifestGeneration() {
+ console.log('=== Testing Backup Manifest Generator ===\n');
+
+ try {
+ // 1. Generate a full backup manifest
+ console.log('1. Generating full backup manifest...');
+
+ const fullManifestOptions = {
+ backupType: 'full',
+ backupPath: '/backup/full/2025-01-21',
+ files: [
+ {
+ path: '/storage/events/active/wedding-smith-2025/DSC_001.jpg',
+ relativePath: 'events/active/wedding-smith-2025/DSC_001.jpg',
+ size: 2456789,
+ modified: new Date('2025-01-20T10:30:00Z'),
+ checksum: 'a1b2c3d4e5f6789012345678901234567890123456789012345678901234567890',
+ permissions: '644'
+ },
+ {
+ path: '/storage/events/active/wedding-smith-2025/DSC_002.jpg',
+ relativePath: 'events/active/wedding-smith-2025/DSC_002.jpg',
+ size: 2156789,
+ modified: new Date('2025-01-20T10:31:00Z'),
+ checksum: 'b2c3d4e5f67890123456789012345678901234567890123456789012345678901',
+ permissions: '644'
+ },
+ {
+ path: '/storage/thumbnails/wedding-smith-2025/thumb_DSC_001.jpg',
+ relativePath: 'thumbnails/wedding-smith-2025/thumb_DSC_001.jpg',
+ size: 45678,
+ modified: new Date('2025-01-20T10:35:00Z'),
+ checksum: 'c3d4e5f678901234567890123456789012345678901234567890123456789012',
+ permissions: '644'
+ }
+ ],
+ databaseInfo: {
+ type: 'sqlite',
+ backupFile: 'database-backup-20250121-103000.sql.gz',
+ size: 1048576,
+ checksum: 'd4e5f6789012345678901234567890123456789012345678901234567890123',
+ tables: {
+ events: 156,
+ photos: 4523,
+ access_logs: 12456,
+ admin_users: 3
+ },
+ rowCounts: {
+ events: 156,
+ photos: 4523,
+ access_logs: 12456,
+ admin_users: 3
+ }
+ },
+ format: 'json',
+ customMetadata: {
+ operator: 'admin@example.com',
+ reason: 'Scheduled daily backup',
+ retentionDays: 30,
+ compressionType: 'gzip'
+ }
+ };
+
+ const fullManifest = await backupManifest.generateManifest(fullManifestOptions);
+
+ // Save in both formats
+ const jsonPath = path.join(__dirname, 'test-manifest-full.json');
+ const yamlPath = path.join(__dirname, 'test-manifest-full.yaml');
+
+ await backupManifest.saveManifest(fullManifest, jsonPath, 'json');
+ await backupManifest.saveManifest(fullManifest, yamlPath, 'yaml');
+
+ console.log('✓ Full backup manifest generated and saved\n');
+
+ // 2. Generate summary report
+ console.log('2. Generating summary report...');
+ const summaryReport = backupManifest.generateSummaryReport(fullManifest);
+ console.log(summaryReport);
+ console.log('\n');
+
+ // 3. Load and validate manifest
+ console.log('3. Loading and validating manifest...');
+ const loadedManifest = await backupManifest.loadManifest(jsonPath);
+ console.log('✓ Manifest loaded and validated successfully\n');
+
+ // 4. Generate incremental backup manifest
+ console.log('4. Generating incremental backup manifest...');
+
+ const incrementalOptions = {
+ backupType: 'incremental',
+ backupPath: '/backup/incremental/2025-01-22',
+ parentBackupId: fullManifest.backup.id,
+ files: [
+ // Original files with same checksums (unchanged)
+ fullManifestOptions.files[0],
+ fullManifestOptions.files[2],
+ // Modified file
+ {
+ ...fullManifestOptions.files[1],
+ size: 2256789,
+ modified: new Date('2025-01-21T14:00:00Z'),
+ checksum: 'e5f678901234567890123456789012345678901234567890123456789012345'
+ },
+ // New file
+ {
+ path: '/storage/events/active/wedding-smith-2025/DSC_003.jpg',
+ relativePath: 'events/active/wedding-smith-2025/DSC_003.jpg',
+ size: 2356789,
+ modified: new Date('2025-01-21T14:30:00Z'),
+ checksum: 'f6789012345678901234567890123456789012345678901234567890123456',
+ permissions: '644'
+ }
+ ],
+ databaseInfo: {
+ ...fullManifestOptions.databaseInfo,
+ size: 1148576,
+ checksum: 'g7890123456789012345678901234567890123456789012345678901234567',
+ rowCounts: {
+ events: 158,
+ photos: 4567,
+ access_logs: 12789,
+ admin_users: 3
+ }
+ }
+ };
+
+ const incrementalManifest = await backupManifest.generateIncrementalManifest(
+ incrementalOptions,
+ fullManifest
+ );
+
+ const incrementalJsonPath = path.join(__dirname, 'test-manifest-incremental.json');
+ await backupManifest.saveManifest(incrementalManifest, incrementalJsonPath, 'json');
+
+ console.log('✓ Incremental backup manifest generated');
+ console.log(` - Added files: ${incrementalManifest.incremental.changes.added_files_count}`);
+ console.log(` - Modified files: ${incrementalManifest.incremental.changes.modified_files_count}`);
+ console.log(` - Deleted files: ${incrementalManifest.incremental.changes.deleted_files_count}`);
+ console.log(` - Size difference: ${(incrementalManifest.incremental.changes.size_difference / 1024).toFixed(2)} KB\n`);
+
+ // 5. Compare manifests
+ console.log('5. Comparing manifests...');
+ const comparison = backupManifest.compareManifests(incrementalManifest, fullManifest);
+ console.log('Comparison results:');
+ console.log(` - Added: ${comparison.added_files.length} files`);
+ console.log(` - Modified: ${comparison.modified_files.length} files`);
+ console.log(` - Deleted: ${comparison.deleted_files.length} files`);
+ console.log(` - Unchanged: ${comparison.unchanged_files.length} files`);
+ console.log(` - Database changed: ${comparison.database_changes.checksum_changed ? 'Yes' : 'No'}\n`);
+
+ // 6. Test manifest integrity
+ console.log('6. Testing manifest integrity...');
+
+ // Corrupt the manifest
+ const corruptedManifest = JSON.parse(JSON.stringify(incrementalManifest));
+ corruptedManifest.files.manifest[0].size = 9999999; // Change a file size
+
+ try {
+ backupManifest.validateManifest(corruptedManifest);
+ console.log('✗ Validation should have failed for corrupted manifest');
+ } catch (error) {
+ console.log('✓ Correctly detected corrupted manifest:', error.message);
+ }
+
+ console.log('\n=== All tests completed successfully! ===');
+
+ // Clean up test files
+ await fs.unlink(jsonPath).catch(() => {});
+ await fs.unlink(yamlPath).catch(() => {});
+ await fs.unlink(incrementalJsonPath).catch(() => {});
+
+ } catch (error) {
+ console.error('Test failed:', error);
+ logger.error('Manifest test failed:', error);
+ process.exit(1);
+ }
+}
+
+// Run tests
+testManifestGeneration().catch(console.error);
\ No newline at end of file
diff --git a/backend/scripts/test-backup-service.js b/backend/scripts/test-backup-service.js
new file mode 100644
index 0000000..9dfc9fe
--- /dev/null
+++ b/backend/scripts/test-backup-service.js
@@ -0,0 +1,46 @@
+#!/usr/bin/env node
+
+require('dotenv').config();
+const { initializeDatabase } = require('../src/database/db');
+const { runBackup, getBackupStatus } = require('../src/services/backupService');
+const logger = require('../src/utils/logger');
+
+async function testBackupService() {
+ try {
+ console.log('Testing backup service...\n');
+
+ // Initialize database
+ await initializeDatabase();
+
+ // Get current backup status
+ console.log('Getting backup status...');
+ const statusBefore = await getBackupStatus();
+ console.log('Last run:', statusBefore.lastRun ? statusBefore.lastRun.started_at : 'Never');
+ console.log('Is healthy:', statusBefore.isHealthy);
+ console.log('');
+
+ // Run backup
+ console.log('Running backup...');
+ await runBackup();
+
+ // Get status after backup
+ console.log('\nGetting status after backup...');
+ const statusAfter = await getBackupStatus();
+ console.log('Last run:', statusAfter.lastRun ? statusAfter.lastRun.started_at : 'Never');
+ console.log('Status:', statusAfter.lastRun ? statusAfter.lastRun.status : 'Unknown');
+ console.log('Files backed up:', statusAfter.lastRun ? statusAfter.lastRun.files_backed_up : 0);
+ console.log('Total size:', statusAfter.lastRun ? `${(statusAfter.lastRun.total_size_bytes / 1024 / 1024).toFixed(2)} MB` : '0 MB');
+
+ if (statusAfter.lastRun && statusAfter.lastRun.error_message) {
+ console.log('Error:', statusAfter.lastRun.error_message);
+ }
+
+ console.log('\nBackup test completed!');
+ process.exit(0);
+ } catch (error) {
+ console.error('Test failed:', error);
+ process.exit(1);
+ }
+}
+
+testBackupService();
\ No newline at end of file
diff --git a/backend/scripts/test-restore-service.js b/backend/scripts/test-restore-service.js
new file mode 100644
index 0000000..e4c994a
--- /dev/null
+++ b/backend/scripts/test-restore-service.js
@@ -0,0 +1,325 @@
+/**
+ * Test script for the restore service
+ *
+ * This script demonstrates the restore service functionality with safety checks
+ *
+ * Usage:
+ * node scripts/test-restore-service.js [options]
+ *
+ * Options:
+ * --dry-run Perform validation only without actual restore
+ * --force Force restore even with warnings
+ * --type Restore type: full, database, files, selective (default: full)
+ * --source Backup source path or S3 URL
+ * --manifest Path to backup manifest
+ */
+
+require('dotenv').config();
+const { restoreService } = require('../src/services/restoreService');
+const { db } = require('../src/database/db');
+const logger = require('../src/utils/logger');
+const path = require('path');
+const fs = require('fs').promises;
+
+// Parse command line arguments
+const args = process.argv.slice(2);
+const options = {
+ dryRun: args.includes('--dry-run'),
+ force: args.includes('--force'),
+ restoreType: 'full',
+ source: null,
+ manifestPath: null
+};
+
+// Parse restore type
+const typeIndex = args.indexOf('--type');
+if (typeIndex !== -1 && args[typeIndex + 1]) {
+ options.restoreType = args[typeIndex + 1];
+}
+
+// Parse source
+const sourceIndex = args.indexOf('--source');
+if (sourceIndex !== -1 && args[sourceIndex + 1]) {
+ options.source = args[sourceIndex + 1];
+}
+
+// Parse manifest
+const manifestIndex = args.indexOf('--manifest');
+if (manifestIndex !== -1 && args[manifestIndex + 1]) {
+ options.manifestPath = args[manifestIndex + 1];
+}
+
+async function testRestore() {
+ console.log('=== PicPeak Restore Service Test ===\n');
+
+ try {
+ // If no source/manifest provided, try to find a recent backup
+ if (!options.source || !options.manifestPath) {
+ console.log('No backup source specified. Looking for recent backups...\n');
+
+ const recentBackup = await db('backup_runs')
+ .where('status', 'completed')
+ .whereNotNull('manifest_path')
+ .orderBy('completed_at', 'desc')
+ .first();
+
+ if (!recentBackup) {
+ console.error('❌ No completed backups found in the database');
+ console.log('\nPlease run a backup first or specify --source and --manifest');
+ process.exit(1);
+ }
+
+ console.log(`Found recent backup from ${recentBackup.completed_at}`);
+ console.log(`Backup ID: ${recentBackup.manifest_id}`);
+ console.log(`Files backed up: ${recentBackup.files_backed_up}`);
+ console.log(`Total size: ${(recentBackup.total_size_bytes / 1024 / 1024).toFixed(2)} MB`);
+ console.log(`Manifest: ${recentBackup.manifest_path}\n`);
+
+ // For this test, we'll create a mock scenario
+ console.log('⚠️ This is a TEST MODE - using mock data for safety\n');
+
+ // Create test backup directory
+ const testBackupDir = path.join(__dirname, '../temp/test-backup');
+ await fs.mkdir(testBackupDir, { recursive: true });
+
+ // Create test manifest
+ const testManifest = {
+ manifest: {
+ version: '2.0',
+ created: new Date().toISOString(),
+ generator: 'Test Script',
+ format: 'json'
+ },
+ backup: {
+ id: 'test-backup-' + Date.now(),
+ type: 'full',
+ timestamp: new Date().toISOString(),
+ path: testBackupDir,
+ parent_backup_id: null,
+ retention_days: 30
+ },
+ system: {
+ hostname: require('os').hostname(),
+ platform: process.platform,
+ os_release: require('os').release(),
+ architecture: require('os').arch()
+ },
+ application: {
+ name: 'PicPeak',
+ version: require('../package.json').version,
+ node_version: process.version,
+ environment: 'test'
+ },
+ files: {
+ count: 0,
+ total_size: 0,
+ checksums: {},
+ manifest: []
+ },
+ database: {
+ type: process.env.DB_TYPE === 'postgresql' ? 'postgresql' : 'sqlite',
+ backup_file: null,
+ size: 0,
+ checksum: null,
+ tables: {},
+ row_counts: {}
+ },
+ verification: {
+ total_checksum: null,
+ file_count_check: 0,
+ size_check: 0,
+ integrity_timestamp: new Date().toISOString()
+ },
+ metadata: {
+ test_mode: true
+ }
+ };
+
+ // Calculate checksum
+ const crypto = require('crypto');
+ const manifestCopy = JSON.parse(JSON.stringify(testManifest));
+ delete manifestCopy.verification.total_checksum;
+ testManifest.verification.total_checksum = crypto
+ .createHash('sha256')
+ .update(JSON.stringify(manifestCopy, Object.keys(manifestCopy).sort()))
+ .digest('hex');
+
+ // Save test manifest
+ const testManifestPath = path.join(testBackupDir, 'test-manifest.json');
+ await fs.writeFile(testManifestPath, JSON.stringify(testManifest, null, 2));
+
+ options.source = testBackupDir;
+ options.manifestPath = testManifestPath;
+ }
+
+ // Display restore options
+ console.log('Restore Options:');
+ console.log(`- Type: ${options.restoreType}`);
+ console.log(`- Source: ${options.source}`);
+ console.log(`- Manifest: ${options.manifestPath}`);
+ console.log(`- Dry Run: ${options.dryRun ? 'Yes' : 'No'}`);
+ console.log(`- Force: ${options.force ? 'Yes' : 'No'}`);
+ console.log('');
+
+ // Add S3 config if source is S3
+ if (options.source.startsWith('s3://')) {
+ options.s3Config = {
+ accessKeyId: process.env.BACKUP_S3_ACCESS_KEY,
+ secretAccessKey: process.env.BACKUP_S3_SECRET_KEY,
+ region: process.env.BACKUP_S3_REGION || 'us-east-1',
+ endpoint: process.env.BACKUP_S3_ENDPOINT
+ };
+
+ if (!options.s3Config.accessKeyId || !options.s3Config.secretAccessKey) {
+ console.error('❌ S3 credentials not configured in environment');
+ process.exit(1);
+ }
+ }
+
+ // Confirm before proceeding (unless dry run)
+ if (!options.dryRun) {
+ console.log('⚠️ WARNING: This will restore data from the backup!');
+ console.log('⚠️ Current data may be overwritten!');
+ console.log('');
+ console.log('Press Ctrl+C to cancel, or wait 5 seconds to continue...');
+ await new Promise(resolve => setTimeout(resolve, 5000));
+ }
+
+ console.log('\nStarting restore operation...\n');
+
+ // Perform restore
+ const result = await restoreService.restore(options);
+
+ if (options.dryRun) {
+ console.log('\n=== DRY RUN RESULTS ===\n');
+
+ console.log('Validation:');
+ console.log(`- Valid: ${result.validation.isValid ? '✅ Yes' : '❌ No'}`);
+
+ if (result.validation.errors.length > 0) {
+ console.log('- Errors:');
+ result.validation.errors.forEach(err => console.log(` ❌ ${err}`));
+ }
+
+ if (result.validation.warnings.length > 0) {
+ console.log('- Warnings:');
+ result.validation.warnings.forEach(warn => console.log(` ⚠️ ${warn}`));
+ }
+
+ console.log('\nDisk Space:');
+ console.log(`- Required: ${result.spaceCheck.requiredFormatted}`);
+ console.log(`- Available: ${result.spaceCheck.availableFormatted}`);
+ console.log(`- Sufficient: ${result.spaceCheck.hasEnoughSpace ? '✅ Yes' : '❌ No'}`);
+
+ } else {
+ console.log('\n=== RESTORE RESULTS ===\n');
+
+ console.log(`Status: ${result.success ? '✅ SUCCESS' : '❌ FAILED'}`);
+ console.log(`Duration: ${result.duration}s`);
+
+ if (result.result) {
+ console.log('\nItems Restored:');
+ if (result.result.databaseRestored !== undefined) {
+ console.log(`- Database: ${result.result.databaseRestored ? '✅' : '❌'}`);
+ }
+ if (result.result.filesRestored !== undefined) {
+ console.log(`- Files: ${result.result.filesRestored}`);
+ }
+ if (result.result.errors && result.result.errors.length > 0) {
+ console.log('- Errors:');
+ result.result.errors.forEach(err => console.log(` ❌ ${err}`));
+ }
+ }
+
+ if (result.verification) {
+ console.log('\nVerification:');
+ console.log(`- Valid: ${result.verification.isValid ? '✅ Yes' : '❌ No'}`);
+ if (result.verification.errors.length > 0) {
+ console.log('- Errors:');
+ result.verification.errors.forEach(err => console.log(` ❌ ${err}`));
+ }
+ }
+
+ if (result.preRestoreBackup) {
+ console.log('\nSafety Backup:');
+ console.log(`- Location: ${result.preRestoreBackup}`);
+ console.log('- This backup can be used to rollback if needed');
+ }
+ }
+
+ // Show recent log entries
+ console.log('\nRecent Log Entries:');
+ result.logs.slice(-10).forEach(log => {
+ const icon = log.level === 'error' ? '❌' : log.level === 'warn' ? '⚠️ ' : 'ℹ️ ';
+ console.log(`${icon} [${log.timestamp}] ${log.message}`);
+ });
+
+ // Clean up test files
+ if (options.source && options.source.includes('test-backup')) {
+ await fs.rmdir(path.dirname(options.source), { recursive: true }).catch(() => {});
+ }
+
+ } catch (error) {
+ console.error('\n❌ Restore operation failed:', error.message);
+
+ // Show logs if available
+ if (restoreService.restoreLog && restoreService.restoreLog.length > 0) {
+ console.log('\nError Log:');
+ restoreService.restoreLog.slice(-10).forEach(log => {
+ if (log.level === 'error' || log.level === 'warn') {
+ console.log(`[${log.timestamp}] ${log.level.toUpperCase()}: ${log.message}`);
+ }
+ });
+ }
+
+ process.exit(1);
+ }
+
+ // Cleanup
+ await db.destroy();
+ process.exit(0);
+}
+
+// Show help if requested
+if (args.includes('--help') || args.includes('-h')) {
+ console.log(`
+PicPeak Restore Service Test
+
+This script tests the restore service functionality with safety checks.
+
+Usage:
+ node scripts/test-restore-service.js [options]
+
+Options:
+ --dry-run Perform validation only without actual restore
+ --force Force restore even with warnings
+ --type Restore type: full, database, files, selective (default: full)
+ --source Backup source path or S3 URL
+ --manifest Path to backup manifest
+ --help Show this help message
+
+Examples:
+ # Dry run with automatic backup selection
+ node scripts/test-restore-service.js --dry-run
+
+ # Full restore from specific backup
+ node scripts/test-restore-service.js --source /backup/2024-01-20 --manifest /backup/2024-01-20/manifest.json
+
+ # Database-only restore with force
+ node scripts/test-restore-service.js --type database --force --source /backup/2024-01-20 --manifest /backup/2024-01-20/manifest.json
+
+ # Restore from S3
+ node scripts/test-restore-service.js --source s3://my-bucket/backups/2024-01-20 --manifest s3://my-bucket/backups/2024-01-20/manifest.json
+
+Safety Features:
+- Pre-restore validation checks compatibility and warns about potential issues
+- Automatic pre-restore backup is created (unless skipped)
+- Post-restore verification ensures data integrity
+- Rollback capability if restore fails
+- Detailed logging of all operations
+`);
+ process.exit(0);
+}
+
+// Run the test
+testRestore();
\ No newline at end of file
diff --git a/backend/server.js b/backend/server.js
index 92a3d93..fb7dce9 100644
--- a/backend/server.js
+++ b/backend/server.js
@@ -20,6 +20,8 @@ const { initializeDatabase, db } = require('./src/database/db');
const { startFileWatcher } = require('./src/services/fileWatcher');
const { startExpirationChecker } = require('./src/services/expirationChecker');
const { initializeTransporter, startEmailQueueProcessor } = require('./src/services/emailProcessor');
+const { startBackupService } = require('./src/services/backupService');
+const { startScheduledBackups } = require('./src/services/databaseBackup');
const { maintenanceMiddleware } = require('./src/middleware/maintenance');
const { sessionTimeoutMiddleware } = require('./src/middleware/sessionTimeout');
const { createRateLimiter, createAuthRateLimiter } = require('./src/services/rateLimitService');
@@ -201,6 +203,8 @@ app.use('/api/gallery', galleryRoutes);
app.use('/api/admin', adminRoutes);
app.use('/api/admin/auth', adminAuthRoutes);
app.use('/api/admin/system', require('./src/routes/adminSystem'));
+app.use('/api/admin/backup', require('./src/routes/adminBackup'));
+app.use('/api/admin/database-backup', require('./src/routes/adminDatabaseBackup'));
app.use('/api/public/settings', require('./src/routes/publicSettings'));
app.use('/api/public', require('./src/routes/publicCMS'));
app.use('/api/images', require('./src/routes/protectedImages'));
@@ -243,6 +247,12 @@ async function startServer() {
await initializeTransporter();
startEmailQueueProcessor();
+ // Start backup service
+ await startBackupService();
+
+ // Start database backup service
+ await startScheduledBackups();
+
app.listen(PORT, () => {
logger.info(`Server running on port ${PORT}`);
logger.info(`Admin interface: ${process.env.ADMIN_URL || 'http://localhost:3000'}`);
diff --git a/backend/src/routes/admin.js b/backend/src/routes/admin.js
index 7bfa3cb..aabba78 100644
--- a/backend/src/routes/admin.js
+++ b/backend/src/routes/admin.js
@@ -11,6 +11,8 @@ const photosRoutes = require('./adminPhotos');
const categoriesRoutes = require('./adminCategories');
const cmsRoutes = require('./adminCMS');
const notificationsRoutes = require('./adminNotifications');
+const backupRoutes = require('./adminBackup');
+const restoreRoutes = require('./adminRestore');
// Mount sub-routers
router.use('/dashboard', dashboardRoutes);
@@ -22,5 +24,7 @@ router.use('/events', photosRoutes);
router.use('/categories', categoriesRoutes);
router.use('/cms', cmsRoutes);
router.use('/notifications', notificationsRoutes);
+router.use('/backup', backupRoutes);
+router.use('/restore', restoreRoutes);
module.exports = router;
\ No newline at end of file
diff --git a/backend/src/routes/adminBackup.js b/backend/src/routes/adminBackup.js
new file mode 100644
index 0000000..bef0e69
--- /dev/null
+++ b/backend/src/routes/adminBackup.js
@@ -0,0 +1,956 @@
+const express = require('express');
+const { db } = require('../database/db');
+const { adminAuth } = require('../middleware/auth');
+const { triggerManualBackup, getBackupStatus, cleanupOldBackupRuns, getBackupManifest, validateBackupManifest } = require('../services/backupService');
+const logger = require('../utils/logger');
+const fs = require('fs').promises;
+const path = require('path');
+const crypto = require('crypto');
+const archiver = require('archiver');
+const S3StorageAdapter = require('../services/storage/s3Storage');
+
+const router = express.Router();
+
+// Get backup configuration
+router.get('/config', adminAuth, async (req, res) => {
+ try {
+ const settings = await db('app_settings')
+ .where('setting_type', 'backup')
+ .select('setting_key', 'setting_value');
+
+ const config = {};
+ settings.forEach(setting => {
+ try {
+ config[setting.setting_key] = JSON.parse(setting.setting_value);
+ } catch (e) {
+ config[setting.setting_key] = setting.setting_value;
+ }
+ });
+
+ res.json(config);
+ } catch (error) {
+ logger.error('Failed to get backup configuration:', error);
+ res.status(500).json({ error: 'Failed to get backup configuration' });
+ }
+});
+
+// Update backup configuration
+router.put('/config', adminAuth, async (req, res) => {
+ try {
+ const updates = req.body;
+
+ // Validate required fields based on destination type
+ if (updates.backup_destination_type) {
+ switch (updates.backup_destination_type) {
+ case 'local':
+ if (!updates.backup_destination_path) {
+ return res.status(400).json({ error: 'Local backup requires destination path' });
+ }
+ break;
+ case 'rsync':
+ if (!updates.backup_rsync_host || !updates.backup_rsync_path) {
+ return res.status(400).json({ error: 'Rsync backup requires host and path' });
+ }
+ break;
+ case 's3':
+ if (!updates.backup_s3_endpoint || !updates.backup_s3_bucket ||
+ !updates.backup_s3_access_key || !updates.backup_s3_secret_key) {
+ return res.status(400).json({ error: 'S3 backup requires endpoint, bucket, and credentials' });
+ }
+ break;
+ }
+ }
+
+ // Update settings
+ for (const [key, value] of Object.entries(updates)) {
+ if (key.startsWith('backup_')) {
+ await db('app_settings')
+ .insert({
+ setting_key: key,
+ setting_value: JSON.stringify(value),
+ setting_type: 'backup',
+ updated_at: new Date()
+ })
+ .onConflict('setting_key')
+ .merge({
+ setting_value: JSON.stringify(value),
+ updated_at: new Date()
+ });
+ }
+ }
+
+ // Restart backup service if enabled status changed
+ if ('backup_enabled' in updates) {
+ const { startBackupService, stopBackupService } = require('../services/backupService');
+ if (updates.backup_enabled) {
+ await startBackupService();
+ } else {
+ stopBackupService();
+ }
+ }
+
+ res.json({ success: true, message: 'Backup configuration updated' });
+ } catch (error) {
+ logger.error('Failed to update backup configuration:', error);
+ res.status(500).json({ error: 'Failed to update backup configuration' });
+ }
+});
+
+// Get backup status and history
+router.get('/status', adminAuth, async (req, res) => {
+ try {
+ const limit = parseInt(req.query.limit) || 10;
+ const status = await getBackupStatus(limit);
+
+ res.json(status);
+ } catch (error) {
+ logger.error('Failed to get backup status:', error);
+ res.status(500).json({ error: 'Failed to get backup status' });
+ }
+});
+
+// Trigger manual backup
+router.post('/run', adminAuth, async (req, res) => {
+ try {
+ // Check if backup is already running
+ const status = await getBackupStatus();
+ if (status.isRunning) {
+ return res.status(409).json({ error: 'Backup is already running' });
+ }
+
+ // Start backup in background
+ triggerManualBackup().catch(error => {
+ logger.error('Manual backup failed:', error);
+ });
+
+ res.json({ success: true, message: 'Backup started' });
+ } catch (error) {
+ logger.error('Failed to trigger manual backup:', error);
+ res.status(500).json({ error: 'Failed to trigger backup' });
+ }
+});
+
+// Get backup run details
+router.get('/runs/:id', adminAuth, async (req, res) => {
+ try {
+ const { id } = req.params;
+
+ const run = await db('backup_runs')
+ .where('id', id)
+ .first();
+
+ if (!run) {
+ return res.status(404).json({ error: 'Backup run not found' });
+ }
+
+ // Parse JSON fields
+ if (run.statistics) {
+ try {
+ run.statistics = JSON.parse(run.statistics);
+ } catch (e) {
+ // Keep as string if parsing fails
+ }
+ }
+
+ res.json(run);
+ } catch (error) {
+ logger.error('Failed to get backup run details:', error);
+ res.status(500).json({ error: 'Failed to get backup run details' });
+ }
+});
+
+// Get file states (for debugging/monitoring)
+router.get('/files', adminAuth, async (req, res) => {
+ try {
+ const { page = 1, limit = 50, search = '' } = req.query;
+ const offset = (page - 1) * limit;
+
+ let query = db('backup_file_states');
+
+ if (search) {
+ query = query.where('file_path', 'like', `%${search}%`);
+ }
+
+ const [files, totalCount] = await Promise.all([
+ query
+ .orderBy('last_backed_up', 'desc')
+ .limit(limit)
+ .offset(offset),
+ db('backup_file_states').count('* as count').first()
+ ]);
+
+ res.json({
+ files,
+ pagination: {
+ page: parseInt(page),
+ limit: parseInt(limit),
+ total: totalCount.count,
+ pages: Math.ceil(totalCount.count / limit)
+ }
+ });
+ } catch (error) {
+ logger.error('Failed to get backup file states:', error);
+ res.status(500).json({ error: 'Failed to get file states' });
+ }
+});
+
+// Clean up old backup runs
+router.delete('/cleanup', adminAuth, async (req, res) => {
+ try {
+ const { days = 30 } = req.body;
+
+ await cleanupOldBackupRuns(days);
+
+ res.json({ success: true, message: `Cleaned up backup runs older than ${days} days` });
+ } catch (error) {
+ logger.error('Failed to cleanup old backup runs:', error);
+ res.status(500).json({ error: 'Failed to cleanup backup runs' });
+ }
+});
+
+// Test backup destination connectivity
+router.post('/test-connection', adminAuth, async (req, res) => {
+ try {
+ const { destination_type, ...config } = req.body;
+
+ switch (destination_type) {
+ case 'local':
+ // Test local path access
+ const fs = require('fs').promises;
+ try {
+ await fs.access(config.path, fs.constants.W_OK);
+ res.json({ success: true, message: 'Local path is writable' });
+ } catch (error) {
+ res.json({ success: false, message: 'Cannot write to local path: ' + error.message });
+ }
+ break;
+
+ case 'rsync':
+ // Test rsync connection
+ const { exec } = require('child_process');
+ const { promisify } = require('util');
+ const execAsync = promisify(exec);
+
+ const sshCommand = config.ssh_key
+ ? `ssh -i ${config.ssh_key} -o StrictHostKeyChecking=no -o ConnectTimeout=10`
+ : 'ssh -o StrictHostKeyChecking=no -o ConnectTimeout=10';
+
+ const testCommand = config.user
+ ? `${sshCommand} ${config.user}@${config.host} "echo 'Connection successful'"`
+ : `${sshCommand} ${config.host} "echo 'Connection successful'"`;
+
+ try {
+ const { stdout } = await execAsync(testCommand);
+ res.json({ success: true, message: 'Rsync connection successful' });
+ } catch (error) {
+ res.json({ success: false, message: 'Rsync connection failed: ' + error.message });
+ }
+ break;
+
+ case 's3':
+ // Test S3 connection (would need AWS SDK)
+ res.json({ success: false, message: 'S3 testing not implemented yet' });
+ break;
+
+ default:
+ res.status(400).json({ error: 'Invalid destination type' });
+ }
+ } catch (error) {
+ logger.error('Failed to test backup connection:', error);
+ res.status(500).json({ error: 'Failed to test connection' });
+ }
+});
+
+// Get backup manifest for a specific backup run
+router.get('/manifest/:backupRunId', adminAuth, async (req, res) => {
+ try {
+ const { backupRunId } = req.params;
+ const result = await getBackupManifest(backupRunId);
+
+ res.json({
+ backupRunId,
+ manifest: result.manifest,
+ summary: result.summary
+ });
+ } catch (error) {
+ logger.error('Failed to get backup manifest:', error);
+ res.status(404).json({ error: error.message || 'Backup manifest not found' });
+ }
+});
+
+// Validate a backup manifest
+router.post('/manifest/validate', adminAuth, async (req, res) => {
+ try {
+ const { manifestPath } = req.body;
+
+ if (!manifestPath) {
+ return res.status(400).json({ error: 'manifestPath is required' });
+ }
+
+ const result = await validateBackupManifest(manifestPath);
+
+ res.json({
+ valid: result.valid,
+ error: result.error,
+ manifestPath
+ });
+ } catch (error) {
+ logger.error('Failed to validate manifest:', error);
+ res.status(500).json({ error: 'Failed to validate manifest' });
+ }
+});
+
+// Download backup manifest
+router.get('/manifest/:backupRunId/download', adminAuth, async (req, res) => {
+ try {
+ const { backupRunId } = req.params;
+ const { format = 'json' } = req.query;
+
+ const result = await getBackupManifest(backupRunId);
+
+ // Set appropriate headers
+ const filename = `backup-manifest-${backupRunId}.${format}`;
+ res.setHeader('Content-Type', format === 'yaml' ? 'text/yaml' : 'application/json');
+ res.setHeader('Content-Disposition', `attachment; filename="${filename}"`);
+
+ // Send the manifest in requested format
+ if (format === 'yaml') {
+ const yaml = require('js-yaml');
+ res.send(yaml.dump(result.manifest, {
+ indent: 2,
+ lineWidth: -1,
+ noRefs: true,
+ sortKeys: true
+ }));
+ } else {
+ res.json(result.manifest);
+ }
+ } catch (error) {
+ logger.error('Failed to download backup manifest:', error);
+ res.status(404).json({ error: error.message || 'Backup manifest not found' });
+ }
+});
+
+// Get manifest for specific backup
+router.get('/manifests/:backupId', adminAuth, async (req, res) => {
+ try {
+ const { backupId } = req.params;
+ const result = await getBackupManifest(backupId);
+
+ res.json({
+ backupId,
+ manifest: result.manifest,
+ summary: result.summary
+ });
+ } catch (error) {
+ logger.error('Failed to get backup manifest:', error);
+ res.status(404).json({ error: error.message || 'Backup manifest not found' });
+ }
+});
+
+// Download manifest file
+router.get('/manifests/:backupId/download', adminAuth, async (req, res) => {
+ try {
+ const { backupId } = req.params;
+ const { format = 'json' } = req.query;
+
+ const result = await getBackupManifest(backupId);
+
+ // Set appropriate headers
+ const filename = `backup-manifest-${backupId}.${format}`;
+ res.setHeader('Content-Type', format === 'yaml' ? 'text/yaml' : 'application/json');
+ res.setHeader('Content-Disposition', `attachment; filename="${filename}"`);
+
+ // Send the manifest in requested format
+ if (format === 'yaml') {
+ const yaml = require('js-yaml');
+ res.send(yaml.dump(result.manifest, {
+ indent: 2,
+ lineWidth: -1,
+ noRefs: true,
+ sortKeys: true
+ }));
+ } else {
+ res.json(result.manifest);
+ }
+ } catch (error) {
+ logger.error('Failed to download backup manifest:', error);
+ res.status(404).json({ error: error.message || 'Backup manifest not found' });
+ }
+});
+
+// Validate a manifest
+router.post('/manifests/validate', adminAuth, async (req, res) => {
+ try {
+ const { manifestPath, manifestData } = req.body;
+
+ if (!manifestPath && !manifestData) {
+ return res.status(400).json({ error: 'Either manifestPath or manifestData is required' });
+ }
+
+ if (manifestData) {
+ // Validate provided manifest data directly
+ const validationResult = await validateManifestData(manifestData);
+ return res.json(validationResult);
+ }
+
+ // Use existing validation function for path
+ const result = await validateBackupManifest(manifestPath);
+
+ res.json({
+ valid: result.valid,
+ error: result.error,
+ manifestPath
+ });
+ } catch (error) {
+ logger.error('Failed to validate manifest:', error);
+ res.status(500).json({ error: 'Failed to validate manifest' });
+ }
+});
+
+// List S3 buckets
+router.get('/s3/buckets', adminAuth, async (req, res) => {
+ try {
+ const config = await getBackupConfig();
+
+ if (config.backup_destination_type !== 's3') {
+ return res.status(400).json({ error: 'S3 backup not configured' });
+ }
+
+ const s3Adapter = new S3StorageAdapter({
+ endpoint: config.backup_s3_endpoint,
+ bucket: config.backup_s3_bucket,
+ accessKeyId: config.backup_s3_access_key,
+ secretAccessKey: config.backup_s3_secret_key,
+ region: config.backup_s3_region || 'us-east-1',
+ forcePathStyle: config.backup_s3_force_path_style || false
+ });
+
+ // List buckets using the S3 client
+ const { ListBucketsCommand } = require('@aws-sdk/client-s3');
+ const result = await s3Adapter.s3Client.send(new ListBucketsCommand({}));
+
+ res.json({
+ buckets: result.Buckets || [],
+ owner: result.Owner || null
+ });
+ } catch (error) {
+ logger.error('Failed to list S3 buckets:', error);
+ res.status(500).json({ error: 'Failed to list S3 buckets: ' + error.message });
+ }
+});
+
+// List files in S3 backup location
+router.get('/s3/files', adminAuth, async (req, res) => {
+ try {
+ const { prefix = '', maxKeys = 100, continuationToken } = req.query;
+ const config = await getBackupConfig();
+
+ if (config.backup_destination_type !== 's3') {
+ return res.status(400).json({ error: 'S3 backup not configured' });
+ }
+
+ const s3Adapter = new S3StorageAdapter({
+ endpoint: config.backup_s3_endpoint,
+ bucket: config.backup_s3_bucket,
+ accessKeyId: config.backup_s3_access_key,
+ secretAccessKey: config.backup_s3_secret_key,
+ region: config.backup_s3_region || 'us-east-1',
+ forcePathStyle: config.backup_s3_force_path_style || false
+ });
+
+ const result = await s3Adapter.list(prefix, {
+ maxKeys: parseInt(maxKeys),
+ continuationToken
+ });
+
+ res.json({
+ files: result.objects || [],
+ directories: result.directories || [],
+ isTruncated: result.isTruncated,
+ nextContinuationToken: result.nextContinuationToken,
+ prefix: prefix
+ });
+ } catch (error) {
+ logger.error('Failed to list S3 files:', error);
+ res.status(500).json({ error: 'Failed to list S3 files: ' + error.message });
+ }
+});
+
+// Clean up old S3 backups
+router.delete('/s3/cleanup', adminAuth, async (req, res) => {
+ try {
+ const { retentionDays = 30, dryRun = false } = req.body;
+ const config = await getBackupConfig();
+
+ if (config.backup_destination_type !== 's3') {
+ return res.status(400).json({ error: 'S3 backup not configured' });
+ }
+
+ const s3Adapter = new S3StorageAdapter({
+ endpoint: config.backup_s3_endpoint,
+ bucket: config.backup_s3_bucket,
+ accessKeyId: config.backup_s3_access_key,
+ secretAccessKey: config.backup_s3_secret_key,
+ region: config.backup_s3_region || 'us-east-1',
+ forcePathStyle: config.backup_s3_force_path_style || false
+ });
+
+ const cutoffDate = new Date();
+ cutoffDate.setDate(cutoffDate.getDate() - retentionDays);
+
+ // List all backup files
+ const backupFiles = await s3Adapter.list('backups/', { maxKeys: 1000 });
+ const filesToDelete = [];
+ let totalSize = 0;
+
+ for (const file of backupFiles.objects || []) {
+ if (file.lastModified && new Date(file.lastModified) < cutoffDate) {
+ filesToDelete.push(file.key);
+ totalSize += file.size || 0;
+ }
+ }
+
+ if (dryRun) {
+ return res.json({
+ wouldDelete: filesToDelete.length,
+ totalSize: totalSize,
+ files: filesToDelete.slice(0, 100), // Limit preview
+ message: 'Dry run completed - no files were deleted'
+ });
+ }
+
+ // Delete files in batches
+ const deleteResult = await s3Adapter.deleteMany(filesToDelete);
+ const deletedCount = deleteResult.Deleted ? deleteResult.Deleted.length : 0;
+
+ // Also clean up database records
+ await cleanupOldBackupRuns(retentionDays);
+
+ res.json({
+ success: true,
+ deletedCount: deletedCount,
+ totalSize: totalSize,
+ message: `Cleaned up ${deletedCount} S3 backup files older than ${retentionDays} days`
+ });
+ } catch (error) {
+ logger.error('Failed to cleanup S3 backups:', error);
+ res.status(500).json({ error: 'Failed to cleanup S3 backups: ' + error.message });
+ }
+});
+
+// Test S3 upload functionality
+router.post('/s3/test-upload', adminAuth, async (req, res) => {
+ try {
+ const config = await getBackupConfig();
+
+ if (config.backup_destination_type !== 's3') {
+ return res.status(400).json({ error: 'S3 backup not configured' });
+ }
+
+ const s3Adapter = new S3StorageAdapter({
+ endpoint: config.backup_s3_endpoint,
+ bucket: config.backup_s3_bucket,
+ accessKeyId: config.backup_s3_access_key,
+ secretAccessKey: config.backup_s3_secret_key,
+ region: config.backup_s3_region || 'us-east-1',
+ forcePathStyle: config.backup_s3_force_path_style || false
+ });
+
+ // Create test content
+ const testKey = `test/backup-test-${Date.now()}.txt`;
+ const testContent = `PicPeak S3 backup test\nTimestamp: ${new Date().toISOString()}\nEndpoint: ${config.backup_s3_endpoint || 'AWS'}\nBucket: ${config.backup_s3_bucket}`;
+
+ // Test upload
+ const uploadStart = Date.now();
+ await s3Adapter.upload(testKey, Buffer.from(testContent));
+ const uploadTime = Date.now() - uploadStart;
+
+ // Test download
+ const downloadStart = Date.now();
+ const downloadedContent = await s3Adapter.download(testKey);
+ const downloadTime = Date.now() - downloadStart;
+
+ // Verify content
+ const contentMatch = downloadedContent.toString() === testContent;
+
+ // Test deletion
+ await s3Adapter.delete(testKey);
+
+ res.json({
+ success: true,
+ testKey: testKey,
+ uploadTime: uploadTime,
+ downloadTime: downloadTime,
+ contentMatch: contentMatch,
+ message: 'S3 upload test completed successfully'
+ });
+ } catch (error) {
+ logger.error('S3 upload test failed:', error);
+ res.status(500).json({ error: 'S3 upload test failed: ' + error.message });
+ }
+});
+
+// Download entire backup
+router.get('/download/:backupId', adminAuth, async (req, res) => {
+ try {
+ const { backupId } = req.params;
+
+ // Get backup run details
+ const backupRun = await db('backup_runs')
+ .where('id', backupId)
+ .first();
+
+ if (!backupRun) {
+ return res.status(404).json({ error: 'Backup not found' });
+ }
+
+ if (backupRun.status !== 'completed') {
+ return res.status(400).json({ error: 'Backup is not completed' });
+ }
+
+ const config = await getBackupConfig();
+
+ // Handle different backup types
+ switch (config.backup_destination_type) {
+ case 'local':
+ // Stream local backup as zip
+ const backupPath = path.join(config.backup_destination_path, `backup-${backupRun.id}`);
+ const archive = archiver('zip', { zlib: { level: 9 } });
+
+ res.attachment(`picpeak-backup-${backupRun.id}.zip`);
+ archive.pipe(res);
+
+ // Add backup directory contents
+ archive.directory(backupPath, false);
+
+ // Add manifest if exists
+ if (backupRun.manifest_path && await fs.access(backupRun.manifest_path).then(() => true).catch(() => false)) {
+ archive.file(backupRun.manifest_path, { name: 'manifest.json' });
+ }
+
+ await archive.finalize();
+ break;
+
+ case 's3':
+ // For S3, provide pre-signed URLs or stream files
+ const s3Adapter = new S3StorageAdapter({
+ endpoint: config.backup_s3_endpoint,
+ bucket: config.backup_s3_bucket,
+ accessKeyId: config.backup_s3_access_key,
+ secretAccessKey: config.backup_s3_secret_key,
+ region: config.backup_s3_region || 'us-east-1',
+ forcePathStyle: config.backup_s3_force_path_style || false
+ });
+
+ // List all files for this backup
+ const prefix = `backups/${backupRun.id}/`;
+ const files = await s3Adapter.list(prefix, { maxKeys: 1000 });
+
+ // Generate pre-signed URLs
+ const urls = [];
+ for (const file of files.objects || []) {
+ const url = await s3Adapter.getSignedUrl('getObject', file.key, { expiresIn: 3600 }); // 1 hour
+ urls.push({
+ key: file.key,
+ size: file.size,
+ url: url
+ });
+ }
+
+ res.json({
+ backupId: backupRun.id,
+ type: 's3',
+ files: urls,
+ expiresIn: 3600,
+ message: 'Use the provided URLs to download individual files'
+ });
+ break;
+
+ case 'rsync':
+ return res.status(400).json({ error: 'Direct download not available for rsync backups' });
+
+ default:
+ return res.status(400).json({ error: 'Unknown backup type' });
+ }
+ } catch (error) {
+ logger.error('Failed to download backup:', error);
+ res.status(500).json({ error: 'Failed to download backup: ' + error.message });
+ }
+});
+
+// Get current file checksums
+router.get('/checksums', adminAuth, async (req, res) => {
+ try {
+ const { path: targetPath = '', recursive = true } = req.query;
+ const checksums = {};
+
+ // Get storage path
+ const storagePath = process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
+ const basePath = targetPath ? path.join(storagePath, targetPath) : storagePath;
+
+ // Calculate checksums for files
+ async function calculateDirChecksums(dirPath, relative = '') {
+ try {
+ const entries = await fs.readdir(dirPath, { withFileTypes: true });
+
+ for (const entry of entries) {
+ const fullPath = path.join(dirPath, entry.name);
+ const relativePath = path.join(relative, entry.name);
+
+ if (entry.isDirectory() && recursive) {
+ await calculateDirChecksums(fullPath, relativePath);
+ } else if (entry.isFile()) {
+ const hash = crypto.createHash('sha256');
+ const stream = require('fs').createReadStream(fullPath);
+
+ await new Promise((resolve, reject) => {
+ stream.on('data', data => hash.update(data));
+ stream.on('end', () => {
+ checksums[relativePath] = {
+ checksum: hash.digest('hex'),
+ size: entry.size,
+ modified: entry.mtime
+ };
+ resolve();
+ });
+ stream.on('error', reject);
+ });
+ }
+ }
+ } catch (error) {
+ logger.error(`Failed to calculate checksums for ${dirPath}:`, error);
+ }
+ }
+
+ await calculateDirChecksums(basePath);
+
+ // Also get database checksums from backup_file_states
+ const dbChecksums = await db('backup_file_states')
+ .select('file_path', 'checksum', 'size_bytes', 'last_modified');
+
+ res.json({
+ currentChecksums: checksums,
+ totalFiles: Object.keys(checksums).length,
+ databaseChecksums: dbChecksums.reduce((acc, row) => {
+ acc[row.file_path] = {
+ checksum: row.checksum,
+ size: row.size_bytes,
+ modified: row.last_modified
+ };
+ return acc;
+ }, {}),
+ path: targetPath || '/'
+ });
+ } catch (error) {
+ logger.error('Failed to get file checksums:', error);
+ res.status(500).json({ error: 'Failed to get file checksums: ' + error.message });
+ }
+});
+
+// Estimate backup size before running
+router.post('/estimate', adminAuth, async (req, res) => {
+ try {
+ const { includeArchived = true } = req.body;
+
+ // Get storage path
+ const storagePath = process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
+ let totalSize = 0;
+ let fileCount = 0;
+ const breakdown = {};
+
+ // Estimate size for each directory
+ async function estimateDir(dirPath, category) {
+ let dirSize = 0;
+ let dirCount = 0;
+
+ try {
+ const entries = await fs.readdir(dirPath, { withFileTypes: true });
+
+ for (const entry of entries) {
+ const fullPath = path.join(dirPath, entry.name);
+
+ if (entry.isDirectory()) {
+ const subResult = await estimateDir(fullPath, category);
+ dirSize += subResult.size;
+ dirCount += subResult.count;
+ } else if (entry.isFile()) {
+ const stats = await fs.stat(fullPath);
+ dirSize += stats.size;
+ dirCount++;
+ }
+ }
+ } catch (error) {
+ if (error.code !== 'ENOENT') {
+ logger.error(`Failed to estimate ${dirPath}:`, error);
+ }
+ }
+
+ return { size: dirSize, count: dirCount };
+ }
+
+ // Estimate each category
+ const categories = [
+ { path: 'events/active', name: 'Active Events' },
+ { path: 'thumbnails', name: 'Thumbnails' },
+ { path: 'uploads', name: 'Uploads' }
+ ];
+
+ if (includeArchived) {
+ categories.push({ path: 'events/archived', name: 'Archived Events' });
+ }
+
+ for (const category of categories) {
+ const result = await estimateDir(path.join(storagePath, category.path), category.name);
+ breakdown[category.name] = {
+ size: result.size,
+ sizeFormatted: formatBytes(result.size),
+ fileCount: result.count
+ };
+ totalSize += result.size;
+ fileCount += result.count;
+ }
+
+ // Estimate database size
+ const dbPath = process.env.DB_TYPE === 'postgresql'
+ ? null
+ : path.join(__dirname, '../../database.sqlite');
+
+ if (dbPath) {
+ try {
+ const dbStats = await fs.stat(dbPath);
+ breakdown['Database'] = {
+ size: dbStats.size,
+ sizeFormatted: formatBytes(dbStats.size),
+ fileCount: 1
+ };
+ totalSize += dbStats.size;
+ fileCount += 1;
+ } catch (error) {
+ logger.error('Failed to get database size:', error);
+ }
+ }
+
+ // Estimate compression ratio (typically 20-40% for mixed media)
+ const estimatedCompressedSize = Math.round(totalSize * 0.7);
+
+ res.json({
+ totalSize: totalSize,
+ totalSizeFormatted: formatBytes(totalSize),
+ estimatedCompressedSize: estimatedCompressedSize,
+ estimatedCompressedSizeFormatted: formatBytes(estimatedCompressedSize),
+ fileCount: fileCount,
+ breakdown: breakdown,
+ includeArchived: includeArchived,
+ estimatedDuration: Math.max(60, Math.round(totalSize / (50 * 1024 * 1024))), // Estimate 50MB/s
+ warnings: totalSize > 10 * 1024 * 1024 * 1024 ? ['Backup size exceeds 10GB, may take significant time'] : []
+ });
+ } catch (error) {
+ logger.error('Failed to estimate backup size:', error);
+ res.status(500).json({ error: 'Failed to estimate backup size: ' + error.message });
+ }
+});
+
+// Helper function to format bytes
+function formatBytes(bytes, decimals = 2) {
+ if (bytes === 0) return '0 Bytes';
+ const k = 1024;
+ const dm = decimals < 0 ? 0 : decimals;
+ const sizes = ['Bytes', 'KB', 'MB', 'GB', 'TB'];
+ const i = Math.floor(Math.log(bytes) / Math.log(k));
+ return parseFloat((bytes / Math.pow(k, i)).toFixed(dm)) + ' ' + sizes[i];
+}
+
+// Helper function to get backup configuration
+async function getBackupConfig() {
+ try {
+ const settings = await db('app_settings')
+ .where('setting_type', 'backup')
+ .select('setting_key', 'setting_value');
+
+ const config = {};
+ settings.forEach(setting => {
+ try {
+ config[setting.setting_key] = JSON.parse(setting.setting_value);
+ } catch (e) {
+ config[setting.setting_key] = setting.setting_value;
+ }
+ });
+
+ return config;
+ } catch (error) {
+ logger.error('Failed to get backup configuration:', error);
+ return {};
+ }
+}
+
+// Helper function to validate manifest data
+async function validateManifestData(manifestData) {
+ try {
+ // Check required fields
+ const requiredFields = ['version', 'backupId', 'timestamp', 'files'];
+ const missingFields = requiredFields.filter(field => !manifestData[field]);
+
+ if (missingFields.length > 0) {
+ return {
+ valid: false,
+ error: `Missing required fields: ${missingFields.join(', ')}`,
+ details: { missingFields }
+ };
+ }
+
+ // Validate version
+ if (manifestData.version !== '1.0') {
+ return {
+ valid: false,
+ error: `Unsupported manifest version: ${manifestData.version}`,
+ details: { version: manifestData.version }
+ };
+ }
+
+ // Validate files array
+ if (!Array.isArray(manifestData.files)) {
+ return {
+ valid: false,
+ error: 'Files must be an array',
+ details: { filesType: typeof manifestData.files }
+ };
+ }
+
+ // Validate each file entry
+ const invalidFiles = [];
+ for (let i = 0; i < manifestData.files.length; i++) {
+ const file = manifestData.files[i];
+ if (!file.path || !file.checksum || typeof file.size !== 'number') {
+ invalidFiles.push({ index: i, file });
+ }
+ }
+
+ if (invalidFiles.length > 0) {
+ return {
+ valid: false,
+ error: `Invalid file entries: ${invalidFiles.length}`,
+ details: { invalidFiles: invalidFiles.slice(0, 10) } // Limit to first 10
+ };
+ }
+
+ return {
+ valid: true,
+ details: {
+ version: manifestData.version,
+ backupId: manifestData.backupId,
+ timestamp: manifestData.timestamp,
+ fileCount: manifestData.files.length,
+ totalSize: manifestData.files.reduce((sum, f) => sum + (f.size || 0), 0)
+ }
+ };
+ } catch (error) {
+ return {
+ valid: false,
+ error: `Validation error: ${error.message}`,
+ details: { error: error.message }
+ };
+ }
+}
+
+module.exports = router;
\ No newline at end of file
diff --git a/backend/src/routes/adminDatabaseBackup.js b/backend/src/routes/adminDatabaseBackup.js
new file mode 100644
index 0000000..5197005
--- /dev/null
+++ b/backend/src/routes/adminDatabaseBackup.js
@@ -0,0 +1,273 @@
+const express = require('express');
+const router = express.Router();
+const { adminAuth } = require('../middleware/auth');
+const { databaseBackupService } = require('../services/databaseBackup');
+const { db } = require('../database/db');
+const logger = require('../utils/logger');
+
+// All routes require admin authentication
+router.use(adminAuth);
+
+/**
+ * Get database backup status and configuration
+ */
+router.get('/status', async (req, res) => {
+ try {
+ // Get configuration
+ const config = await databaseBackupService.getBackupConfig();
+
+ // Get recent backup history
+ const history = await databaseBackupService.getBackupHistory(10);
+
+ // Get current progress if running
+ const progress = databaseBackupService.getProgress();
+
+ // Calculate health status
+ const lastBackup = history[0];
+ const isHealthy = lastBackup && lastBackup.status === 'completed' &&
+ new Date(lastBackup.completed_at) > new Date(Date.now() - 48 * 60 * 60 * 1000); // Within 48 hours
+
+ res.json({
+ config,
+ isRunning: databaseBackupService.isRunning,
+ isHealthy,
+ currentProgress: progress,
+ lastBackup,
+ recentBackups: history,
+ dbType: databaseBackupService.dbType
+ });
+ } catch (error) {
+ logger.error('Failed to get database backup status:', error);
+ res.status(500).json({ error: 'Failed to get backup status' });
+ }
+});
+
+/**
+ * Update database backup configuration
+ */
+router.put('/config', async (req, res) => {
+ try {
+ const allowedSettings = [
+ 'database_backup_enabled',
+ 'database_backup_schedule',
+ 'database_backup_destination_path',
+ 'database_backup_compress',
+ 'database_backup_validate_integrity',
+ 'database_backup_include_checksums',
+ 'database_backup_retention_days',
+ 'database_backup_email_on_failure',
+ 'database_backup_email_on_success'
+ ];
+
+ const updates = [];
+
+ for (const [key, value] of Object.entries(req.body)) {
+ if (allowedSettings.includes(key)) {
+ // Check if setting exists
+ const existing = await db('app_settings')
+ .where('setting_key', key)
+ .first();
+
+ if (existing) {
+ await db('app_settings')
+ .where('setting_key', key)
+ .update({
+ setting_value: JSON.stringify(value),
+ updated_at: new Date()
+ });
+ } else {
+ await db('app_settings').insert({
+ setting_key: key,
+ setting_value: JSON.stringify(value),
+ setting_type: 'database_backup'
+ });
+ }
+
+ updates.push(key);
+ }
+ }
+
+ // Restart scheduled backups if enabled state changed
+ if (updates.includes('database_backup_enabled') || updates.includes('database_backup_schedule')) {
+ const { startScheduledBackups, stopScheduledBackups } = require('../services/databaseBackup');
+ stopScheduledBackups();
+ await startScheduledBackups();
+ }
+
+ res.json({
+ success: true,
+ updatedSettings: updates,
+ message: 'Database backup configuration updated successfully'
+ });
+ } catch (error) {
+ logger.error('Failed to update database backup config:', error);
+ res.status(500).json({ error: 'Failed to update configuration' });
+ }
+});
+
+/**
+ * Trigger manual database backup
+ */
+router.post('/backup', async (req, res) => {
+ try {
+ if (databaseBackupService.isRunning) {
+ return res.status(409).json({ error: 'Backup already in progress' });
+ }
+
+ // Start backup asynchronously
+ res.json({
+ success: true,
+ message: 'Database backup started',
+ trackingUrl: '/api/admin/database-backup/progress'
+ });
+
+ // Run backup in background
+ databaseBackupService.backup(req.body).catch(error => {
+ logger.error('Manual database backup failed:', error);
+ });
+ } catch (error) {
+ logger.error('Failed to start database backup:', error);
+ res.status(500).json({ error: 'Failed to start backup' });
+ }
+});
+
+/**
+ * Get current backup progress
+ */
+router.get('/progress', async (req, res) => {
+ try {
+ const progress = databaseBackupService.getProgress();
+
+ res.json({
+ isRunning: databaseBackupService.isRunning,
+ progress
+ });
+ } catch (error) {
+ logger.error('Failed to get backup progress:', error);
+ res.status(500).json({ error: 'Failed to get progress' });
+ }
+});
+
+/**
+ * Get backup history with pagination
+ */
+router.get('/history', async (req, res) => {
+ try {
+ const page = parseInt(req.query.page) || 1;
+ const limit = parseInt(req.query.limit) || 20;
+ const offset = (page - 1) * limit;
+
+ const [backups, totalCount] = await Promise.all([
+ db('database_backup_runs')
+ .orderBy('started_at', 'desc')
+ .limit(limit)
+ .offset(offset),
+ db('database_backup_runs').count('* as count').first()
+ ]);
+
+ res.json({
+ backups,
+ pagination: {
+ page,
+ limit,
+ total: totalCount.count,
+ pages: Math.ceil(totalCount.count / limit)
+ }
+ });
+ } catch (error) {
+ logger.error('Failed to get backup history:', error);
+ res.status(500).json({ error: 'Failed to get history' });
+ }
+});
+
+/**
+ * Delete old backup files
+ */
+router.delete('/cleanup', async (req, res) => {
+ try {
+ const { retentionDays = 30 } = req.body;
+
+ await databaseBackupService.cleanupOldBackups(retentionDays);
+
+ res.json({
+ success: true,
+ message: `Cleaned up backups older than ${retentionDays} days`
+ });
+ } catch (error) {
+ logger.error('Failed to cleanup old backups:', error);
+ res.status(500).json({ error: 'Failed to cleanup backups' });
+ }
+});
+
+/**
+ * Test database backup configuration
+ */
+router.post('/test', async (req, res) => {
+ try {
+ const config = await databaseBackupService.getBackupConfig();
+
+ // Test database connection
+ const testResults = {
+ databaseConnection: false,
+ destinationWritable: false,
+ compressionAvailable: true,
+ estimatedSize: null
+ };
+
+ // Test database connection
+ try {
+ await db.raw('SELECT 1');
+ testResults.databaseConnection = true;
+ } catch (error) {
+ testResults.databaseConnectionError = error.message;
+ }
+
+ // Test destination path
+ if (config.destinationPath) {
+ try {
+ const fs = require('fs').promises;
+ const testFile = `${config.destinationPath}/.test-${Date.now()}`;
+ await fs.writeFile(testFile, 'test');
+ await fs.unlink(testFile);
+ testResults.destinationWritable = true;
+ } catch (error) {
+ testResults.destinationError = error.message;
+ }
+ }
+
+ // Estimate database size
+ try {
+ testResults.estimatedSize = await databaseBackupService.getDatabaseSize();
+ } catch (error) {
+ testResults.sizeError = error.message;
+ }
+
+ res.json({
+ success: testResults.databaseConnection && testResults.destinationWritable,
+ results: testResults
+ });
+ } catch (error) {
+ logger.error('Failed to test backup configuration:', error);
+ res.status(500).json({ error: 'Failed to test configuration' });
+ }
+});
+
+/**
+ * Get table checksums
+ */
+router.get('/checksums', async (req, res) => {
+ try {
+ const checksums = await databaseBackupService.getTableChecksums();
+
+ res.json({
+ checksums,
+ tableCount: Object.keys(checksums).length,
+ totalRows: Object.values(checksums).reduce((sum, table) => sum + table.rowCount, 0)
+ });
+ } catch (error) {
+ logger.error('Failed to get table checksums:', error);
+ res.status(500).json({ error: 'Failed to get checksums' });
+ }
+});
+
+module.exports = router;
\ No newline at end of file
diff --git a/backend/src/routes/adminRestore.js b/backend/src/routes/adminRestore.js
new file mode 100644
index 0000000..063ff58
--- /dev/null
+++ b/backend/src/routes/adminRestore.js
@@ -0,0 +1,461 @@
+const express = require('express');
+const router = express.Router();
+const { restoreService } = require('../services/restoreService');
+const { adminAuth } = require('../middleware/auth');
+const { body, query, validationResult } = require('express-validator');
+const logger = require('../utils/logger');
+const { db } = require('../database/db');
+const path = require('path');
+const fs = require('fs').promises;
+
+/**
+ * Admin routes for restore operations
+ * All routes require admin authentication
+ */
+
+// Apply admin authentication to all routes
+router.use(adminAuth);
+
+/**
+ * Get restore service status and history
+ */
+router.get('/status', async (req, res) => {
+ try {
+ const limit = parseInt(req.query.limit) || 10;
+ const history = await restoreService.getRestoreHistory(limit);
+
+ const status = {
+ isRunning: restoreService.isRunning,
+ currentProgress: restoreService.getProgress(),
+ history: history,
+ settings: await getRestoreSettings()
+ };
+
+ res.json({
+ success: true,
+ data: status
+ });
+ } catch (error) {
+ logger.error('Failed to get restore status:', error);
+ res.status(500).json({
+ success: false,
+ error: 'Failed to get restore status'
+ });
+ }
+});
+
+/**
+ * Validate restore request
+ */
+router.post('/validate', [
+ body('source').notEmpty().withMessage('Backup source is required'),
+ body('manifestPath').notEmpty().withMessage('Manifest path is required'),
+ body('restoreType').isIn(['full', 'database', 'files', 'selective']).withMessage('Invalid restore type'),
+ body('selectedItems').optional().isArray(),
+ body('s3Config').optional().isObject()
+], async (req, res) => {
+ const errors = validationResult(req);
+ if (!errors.isEmpty()) {
+ return res.status(400).json({
+ success: false,
+ errors: errors.array()
+ });
+ }
+
+ try {
+ // Perform dry run validation
+ const result = await restoreService.restore({
+ ...req.body,
+ dryRun: true,
+ force: false
+ });
+
+ res.json({
+ success: true,
+ data: {
+ validation: result.validation,
+ spaceCheck: result.spaceCheck,
+ logs: result.logs
+ }
+ });
+ } catch (error) {
+ logger.error('Restore validation failed:', error);
+ res.status(400).json({
+ success: false,
+ error: error.message,
+ logs: restoreService.restoreLog
+ });
+ }
+});
+
+/**
+ * Start restore operation
+ */
+router.post('/start', [
+ body('source').notEmpty().withMessage('Backup source is required'),
+ body('manifestPath').notEmpty().withMessage('Manifest path is required'),
+ body('restoreType').isIn(['full', 'database', 'files', 'selective']).withMessage('Invalid restore type'),
+ body('selectedItems').optional().isArray(),
+ body('skipPreBackup').optional().isBoolean(),
+ body('force').optional().isBoolean(),
+ body('s3Config').optional().isObject()
+], async (req, res) => {
+ const errors = validationResult(req);
+ if (!errors.isEmpty()) {
+ return res.status(400).json({
+ success: false,
+ errors: errors.array()
+ });
+ }
+
+ try {
+ // Check if restore is already running
+ if (restoreService.isRunning) {
+ return res.status(409).json({
+ success: false,
+ error: 'Restore operation already in progress'
+ });
+ }
+
+ // Check permissions for dangerous options
+ const settings = await getRestoreSettings();
+ if (req.body.force && !settings.restore_allow_force) {
+ return res.status(403).json({
+ success: false,
+ error: 'Force restore is not allowed by system settings'
+ });
+ }
+
+ if (req.body.skipPreBackup && settings.restore_require_pre_backup) {
+ return res.status(403).json({
+ success: false,
+ error: 'Skipping pre-restore backup is not allowed by system settings'
+ });
+ }
+
+ // Log restore attempt
+ logger.warn('Restore operation started', {
+ user: req.user.email,
+ ip: req.ip,
+ restoreType: req.body.restoreType,
+ source: req.body.source
+ });
+
+ // Start restore in background
+ restoreService.restore({
+ ...req.body,
+ dryRun: false,
+ operator: {
+ type: 'manual',
+ userId: req.user.id,
+ ip: req.ip
+ }
+ }).catch(error => {
+ logger.error('Background restore failed:', error);
+ });
+
+ res.json({
+ success: true,
+ message: 'Restore operation started'
+ });
+ } catch (error) {
+ logger.error('Failed to start restore:', error);
+ res.status(500).json({
+ success: false,
+ error: error.message
+ });
+ }
+});
+
+/**
+ * Get current restore progress
+ */
+router.get('/progress', async (req, res) => {
+ try {
+ const progress = restoreService.getProgress();
+ const logs = restoreService.restoreLog.slice(-50); // Last 50 log entries
+
+ res.json({
+ success: true,
+ data: {
+ isRunning: restoreService.isRunning,
+ progress: progress,
+ logs: logs
+ }
+ });
+ } catch (error) {
+ logger.error('Failed to get restore progress:', error);
+ res.status(500).json({
+ success: false,
+ error: 'Failed to get restore progress'
+ });
+ }
+});
+
+/**
+ * Get restore run details
+ */
+router.get('/run/:id', async (req, res) => {
+ try {
+ const run = await db('restore_runs')
+ .where('id', req.params.id)
+ .first();
+
+ if (!run) {
+ return res.status(404).json({
+ success: false,
+ error: 'Restore run not found'
+ });
+ }
+
+ // Parse JSON fields
+ if (run.statistics) run.statistics = JSON.parse(run.statistics);
+ if (run.restore_log) run.restore_log = JSON.parse(run.restore_log);
+ if (run.metadata) run.metadata = JSON.parse(run.metadata);
+
+ // Get validation results
+ const validations = await db('restore_validation_results')
+ .where('restore_run_id', run.id)
+ .select('*');
+
+ validations.forEach(v => {
+ if (v.errors) v.errors = JSON.parse(v.errors);
+ if (v.warnings) v.warnings = JSON.parse(v.warnings);
+ if (v.checksums) v.checksums = JSON.parse(v.checksums);
+ });
+
+ // Get file operations summary
+ const fileOps = await db('restore_file_operations')
+ .where('restore_run_id', run.id)
+ .select('status', db.raw('COUNT(*) as count'))
+ .groupBy('status');
+
+ res.json({
+ success: true,
+ data: {
+ run: run,
+ validations: validations,
+ fileOperations: fileOps
+ }
+ });
+ } catch (error) {
+ logger.error('Failed to get restore run details:', error);
+ res.status(500).json({
+ success: false,
+ error: 'Failed to get restore run details'
+ });
+ }
+});
+
+/**
+ * Get restore run report
+ */
+router.get('/run/:id/report', async (req, res) => {
+ try {
+ const run = await db('restore_runs')
+ .where('id', req.params.id)
+ .first();
+
+ if (!run) {
+ return res.status(404).json({
+ success: false,
+ error: 'Restore run not found'
+ });
+ }
+
+ // Parse JSON fields
+ if (run.statistics) run.statistics = JSON.parse(run.statistics);
+ if (run.restore_log) run.restore_log = JSON.parse(run.restore_log);
+
+ // Generate report
+ const report = restoreService.generateRestoreReport({
+ success: run.status === 'completed',
+ duration: run.duration_seconds,
+ dryRun: run.is_dry_run,
+ result: run.statistics,
+ logs: run.restore_log || []
+ });
+
+ res.type('text/plain').send(report);
+ } catch (error) {
+ logger.error('Failed to generate restore report:', error);
+ res.status(500).json({
+ success: false,
+ error: 'Failed to generate restore report'
+ });
+ }
+});
+
+/**
+ * List available backups for restore
+ */
+router.get('/available-backups', async (req, res) => {
+ try {
+ const backups = [];
+
+ // Get local file backups
+ const backupConfig = await getBackupConfig();
+ if (backupConfig.backup_destination_type === 'local' && backupConfig.backup_destination_path) {
+ try {
+ const files = await fs.readdir(backupConfig.backup_destination_path);
+ for (const file of files) {
+ if (file.endsWith('.json') || file.endsWith('.yaml')) {
+ const filePath = path.join(backupConfig.backup_destination_path, file);
+ const stats = await fs.stat(filePath);
+ backups.push({
+ type: 'local',
+ name: file,
+ path: filePath,
+ size: stats.size,
+ modified: stats.mtime
+ });
+ }
+ }
+ } catch (error) {
+ logger.warn('Failed to list local backups:', error);
+ }
+ }
+
+ // Get database backups from backup_runs table
+ const backupRuns = await db('backup_runs')
+ .where('status', 'completed')
+ .whereNotNull('manifest_path')
+ .orderBy('completed_at', 'desc')
+ .limit(20);
+
+ for (const run of backupRuns) {
+ backups.push({
+ type: run.manifest_path.startsWith('s3://') ? 's3' : 'local',
+ name: `Backup ${run.completed_at}`,
+ path: run.manifest_path,
+ manifestId: run.manifest_id,
+ size: run.total_size_bytes,
+ filesCount: run.files_backed_up,
+ duration: run.duration_seconds,
+ completed: run.completed_at
+ });
+ }
+
+ res.json({
+ success: true,
+ data: backups
+ });
+ } catch (error) {
+ logger.error('Failed to list available backups:', error);
+ res.status(500).json({
+ success: false,
+ error: 'Failed to list available backups'
+ });
+ }
+});
+
+/**
+ * Get restore settings
+ */
+router.get('/settings', async (req, res) => {
+ try {
+ const settings = await getRestoreSettings();
+ res.json({
+ success: true,
+ data: settings
+ });
+ } catch (error) {
+ logger.error('Failed to get restore settings:', error);
+ res.status(500).json({
+ success: false,
+ error: 'Failed to get restore settings'
+ });
+ }
+});
+
+/**
+ * Update restore settings
+ */
+router.put('/settings', [
+ body('restore_allow_force').optional().isBoolean(),
+ body('restore_require_pre_backup').optional().isBoolean(),
+ body('restore_max_file_size_mb').optional().isInt({ min: 1 }),
+ body('restore_verify_checksums').optional().isBoolean(),
+ body('restore_email_on_completion').optional().isBoolean(),
+ body('restore_retention_days').optional().isInt({ min: 1 })
+], async (req, res) => {
+ const errors = validationResult(req);
+ if (!errors.isEmpty()) {
+ return res.status(400).json({
+ success: false,
+ errors: errors.array()
+ });
+ }
+
+ try {
+ // Update settings
+ for (const [key, value] of Object.entries(req.body)) {
+ await db('app_settings')
+ .where('setting_key', key)
+ .where('setting_type', 'restore')
+ .update({
+ setting_value: typeof value === 'boolean' ? (value ? '1' : '0') : value.toString(),
+ updated_at: db.fn.now()
+ });
+ }
+
+ logger.info('Restore settings updated', {
+ user: req.user.email,
+ settings: req.body
+ });
+
+ res.json({
+ success: true,
+ message: 'Settings updated successfully'
+ });
+ } catch (error) {
+ logger.error('Failed to update restore settings:', error);
+ res.status(500).json({
+ success: false,
+ error: 'Failed to update settings'
+ });
+ }
+});
+
+/**
+ * Helper function to get restore settings
+ */
+async function getRestoreSettings() {
+ const settings = await db('app_settings')
+ .where('setting_type', 'restore')
+ .select('setting_key', 'setting_value');
+
+ const result = {};
+ settings.forEach(setting => {
+ // Convert boolean strings to actual booleans
+ if (setting.setting_value === '1' || setting.setting_value === '0') {
+ result[setting.setting_key] = setting.setting_value === '1';
+ } else {
+ result[setting.setting_key] = setting.setting_value;
+ }
+ });
+
+ return result;
+}
+
+/**
+ * Helper function to get backup configuration
+ */
+async function getBackupConfig() {
+ const settings = await db('app_settings')
+ .where('setting_type', 'backup')
+ .select('setting_key', 'setting_value');
+
+ const config = {};
+ settings.forEach(setting => {
+ try {
+ config[setting.setting_key] = JSON.parse(setting.setting_value);
+ } catch (e) {
+ config[setting.setting_key] = setting.setting_value;
+ }
+ });
+
+ return config;
+}
+
+module.exports = router;
\ No newline at end of file
diff --git a/backend/src/services/__tests__/databaseBackup.test.js b/backend/src/services/__tests__/databaseBackup.test.js
new file mode 100644
index 0000000..be91ab2
--- /dev/null
+++ b/backend/src/services/__tests__/databaseBackup.test.js
@@ -0,0 +1,251 @@
+const { DatabaseBackupService } = require('../databaseBackup');
+const { db } = require('../../database/db');
+const fs = require('fs').promises;
+const path = require('path');
+const crypto = require('crypto');
+
+// Mock dependencies
+jest.mock('../../database/db');
+jest.mock('../../utils/logger');
+jest.mock('../emailProcessor');
+jest.mock('child_process');
+
+describe('DatabaseBackupService', () => {
+ let service;
+ let mockExecAsync;
+
+ beforeEach(() => {
+ service = new DatabaseBackupService();
+ mockExecAsync = jest.fn();
+
+ // Reset mocks
+ jest.clearAllMocks();
+
+ // Mock execAsync
+ const childProcess = require('child_process');
+ childProcess.exec = jest.fn((cmd, opts, callback) => {
+ if (callback) {
+ callback(null, { stdout: 'ok', stderr: '' });
+ }
+ });
+ });
+
+ afterEach(async () => {
+ // Cleanup test files
+ try {
+ await fs.rmdir('/tmp/test-backup', { recursive: true });
+ } catch (e) {
+ // Ignore
+ }
+ });
+
+ describe('calculateChecksum', () => {
+ it('should calculate SHA256 checksum of a file', async () => {
+ const testFile = '/tmp/test-checksum.txt';
+ const testContent = 'Hello, World!';
+ await fs.writeFile(testFile, testContent);
+
+ const checksum = await service.calculateChecksum(testFile);
+
+ // Expected checksum for "Hello, World!"
+ const expectedChecksum = crypto
+ .createHash('sha256')
+ .update(testContent)
+ .digest('hex');
+
+ expect(checksum).toBe(expectedChecksum);
+
+ await fs.unlink(testFile);
+ });
+ });
+
+ describe('getTableChecksums', () => {
+ it('should get checksums for all tables', async () => {
+ // Mock getTables
+ service.getTables = jest.fn().mockResolvedValue(['events', 'photos']);
+
+ // Mock SQLite response
+ db.raw = jest.fn()
+ .mockResolvedValueOnce([{ row_count: 10, data_sum: 1000 }])
+ .mockResolvedValueOnce([{ row_count: 20, data_sum: 2000 }]);
+
+ const checksums = await service.getTableChecksums();
+
+ expect(checksums).toHaveProperty('events');
+ expect(checksums).toHaveProperty('photos');
+ expect(checksums.events.rowCount).toBe(10);
+ expect(checksums.photos.rowCount).toBe(20);
+ expect(checksums.events.checksum).toBeDefined();
+ expect(checksums.photos.checksum).toBeDefined();
+ });
+ });
+
+ describe('getTables', () => {
+ it('should get list of tables for SQLite', async () => {
+ service.dbType = 'sqlite';
+
+ db.raw = jest.fn().mockResolvedValue([
+ { name: 'events' },
+ { name: 'photos' },
+ { name: 'admin_users' }
+ ]);
+
+ const tables = await service.getTables();
+
+ expect(tables).toEqual(['events', 'photos', 'admin_users']);
+ expect(db.raw).toHaveBeenCalledWith(expect.stringContaining('sqlite_master'));
+ });
+
+ it('should get list of tables for PostgreSQL', async () => {
+ service.dbType = 'postgresql';
+
+ db.raw = jest.fn().mockResolvedValue({
+ rows: [
+ { table_name: 'events' },
+ { table_name: 'photos' },
+ { table_name: 'admin_users' }
+ ]
+ });
+
+ const tables = await service.getTables();
+
+ expect(tables).toEqual(['events', 'photos', 'admin_users']);
+ expect(db.raw).toHaveBeenCalledWith(expect.stringContaining('information_schema.tables'));
+ });
+ });
+
+ describe('getDatabaseSize', () => {
+ it('should get database size for SQLite', async () => {
+ service.dbType = 'sqlite';
+ const mockSize = 1024 * 1024 * 10; // 10MB
+
+ // Mock fs.stat
+ const originalStat = fs.stat;
+ fs.stat = jest.fn().mockResolvedValue({ size: mockSize });
+
+ const size = await service.getDatabaseSize();
+
+ expect(size).toBe(mockSize);
+
+ fs.stat = originalStat;
+ });
+
+ it('should get database size for PostgreSQL', async () => {
+ service.dbType = 'postgresql';
+ const mockSize = 1024 * 1024 * 100; // 100MB
+
+ db.raw = jest.fn().mockResolvedValue({
+ rows: [{ size: mockSize.toString() }]
+ });
+
+ const size = await service.getDatabaseSize();
+
+ expect(size).toBe(mockSize);
+ expect(db.raw).toHaveBeenCalledWith(expect.stringContaining('pg_database_size'));
+ });
+ });
+
+ describe('compressFile', () => {
+ it('should compress file and return stats', async () => {
+ const testFile = '/tmp/test-compress.txt';
+ const compressedFile = '/tmp/test-compress.txt.gz';
+
+ // Create test file with repetitive content (compresses well)
+ const testContent = 'Hello, World! '.repeat(1000);
+ await fs.writeFile(testFile, testContent);
+
+ const stats = await service.compressFile(testFile, compressedFile);
+
+ expect(stats.originalSize).toBeGreaterThan(0);
+ expect(stats.compressedSize).toBeGreaterThan(0);
+ expect(stats.compressedSize).toBeLessThan(stats.originalSize);
+ expect(parseFloat(stats.compressionRatio)).toBeGreaterThan(0);
+
+ // Cleanup
+ await fs.unlink(testFile);
+ await fs.unlink(compressedFile);
+ });
+ });
+
+ describe('backup configuration', () => {
+ it('should get backup configuration from database', async () => {
+ const mockConfig = [
+ { setting_key: 'database_backup_enabled', setting_value: 'true' },
+ { setting_key: 'database_backup_compress', setting_value: 'true' },
+ { setting_key: 'database_backup_retention_days', setting_value: '30' }
+ ];
+
+ db.mockReturnValue({
+ where: jest.fn().mockReturnThis(),
+ select: jest.fn().mockResolvedValue(mockConfig)
+ });
+
+ const config = await service.getBackupConfig();
+
+ expect(config.database_backup_enabled).toBe(true);
+ expect(config.database_backup_compress).toBe(true);
+ expect(config.database_backup_retention_days).toBe(30);
+ });
+ });
+
+ describe('cleanupOldBackups', () => {
+ it('should delete old backup files and records', async () => {
+ const oldBackups = [
+ { id: 1, file_path: '/backup/old1.sql.gz' },
+ { id: 2, file_path: '/backup/old2.sql.gz' }
+ ];
+
+ db.mockReturnValue({
+ where: jest.fn().mockReturnThis(),
+ select: jest.fn().mockResolvedValue(oldBackups),
+ delete: jest.fn().mockResolvedValue(1)
+ });
+
+ // Mock fs.unlink
+ fs.unlink = jest.fn().mockResolvedValue(undefined);
+
+ await service.cleanupOldBackups(30);
+
+ expect(fs.unlink).toHaveBeenCalledTimes(2);
+ expect(fs.unlink).toHaveBeenCalledWith('/backup/old1.sql.gz');
+ expect(fs.unlink).toHaveBeenCalledWith('/backup/old2.sql.gz');
+ });
+ });
+
+ describe('progress tracking', () => {
+ it('should update and retrieve progress', () => {
+ expect(service.getProgress()).toBeNull();
+
+ service.updateProgress('Testing...', { step: 1 });
+
+ const progress = service.getProgress();
+ expect(progress.message).toBe('Testing...');
+ expect(progress.details.step).toBe(1);
+ expect(progress.timestamp).toBeDefined();
+ });
+ });
+
+ describe('backup history', () => {
+ it('should retrieve backup history', async () => {
+ const mockHistory = [
+ {
+ id: 1,
+ started_at: new Date(),
+ completed_at: new Date(),
+ status: 'completed',
+ file_size_bytes: 1024000
+ }
+ ];
+
+ db.mockReturnValue({
+ orderBy: jest.fn().mockReturnThis(),
+ limit: jest.fn().mockResolvedValue(mockHistory)
+ });
+
+ const history = await service.getBackupHistory(10);
+
+ expect(history).toEqual(mockHistory);
+ expect(history.length).toBe(1);
+ });
+ });
+});
\ No newline at end of file
diff --git a/backend/src/services/backupManifest.js b/backend/src/services/backupManifest.js
new file mode 100644
index 0000000..0a082f3
--- /dev/null
+++ b/backend/src/services/backupManifest.js
@@ -0,0 +1,501 @@
+const fs = require('fs').promises;
+const path = require('path');
+const crypto = require('crypto');
+const yaml = require('js-yaml');
+const os = require('os');
+const { db } = require('../database/db');
+const logger = require('../utils/logger');
+
+/**
+ * Backup Manifest Generator
+ *
+ * Generates comprehensive manifests for backups including:
+ * - Version information (app, node, OS)
+ * - File listings with metadata and checksums
+ * - Database information
+ * - System state at backup time
+ * - Support for both JSON and YAML formats
+ * - Incremental backup support with parent references
+ */
+
+class BackupManifestGenerator {
+ constructor() {
+ this.appVersion = require('../../package.json').version;
+ this.nodeVersion = process.version;
+ this.platform = process.platform;
+ this.osRelease = os.release();
+ this.hostname = os.hostname();
+ }
+
+ /**
+ * Generate a comprehensive backup manifest
+ * @param {Object} options - Manifest generation options
+ * @param {string} options.backupType - 'full' or 'incremental'
+ * @param {string} options.backupPath - Path to the backup directory
+ * @param {Array} options.files - Array of backed up files with metadata
+ * @param {Object} options.databaseInfo - Database backup information
+ * @param {string} options.parentBackupId - For incremental backups, reference to parent
+ * @param {string} options.format - 'json' or 'yaml' (default: 'json')
+ * @param {Object} options.customMetadata - Additional metadata to include
+ * @returns {Object} Generated manifest object
+ */
+ async generateManifest(options) {
+ const {
+ backupType = 'full',
+ backupPath,
+ files = [],
+ databaseInfo = {},
+ parentBackupId = null,
+ format = 'json',
+ customMetadata = {}
+ } = options;
+
+ const manifest = {
+ // Manifest metadata
+ manifest: {
+ version: '2.0',
+ created: new Date().toISOString(),
+ generator: 'PicPeak Backup Manifest Generator',
+ format: format
+ },
+
+ // Backup information
+ backup: {
+ id: this.generateBackupId(),
+ type: backupType,
+ timestamp: new Date().toISOString(),
+ path: backupPath,
+ parent_backup_id: parentBackupId,
+ retention_days: customMetadata.retentionDays || 30
+ },
+
+ // System information
+ system: {
+ hostname: this.hostname,
+ platform: this.platform,
+ os_release: this.osRelease,
+ architecture: os.arch(),
+ cpu_count: os.cpus().length,
+ total_memory: os.totalmem(),
+ free_memory: os.freemem(),
+ uptime: os.uptime()
+ },
+
+ // Application information
+ application: {
+ name: 'PicPeak',
+ version: this.appVersion,
+ node_version: this.nodeVersion,
+ environment: process.env.NODE_ENV || 'production',
+ storage_path: process.env.STORAGE_PATH || path.join(__dirname, '../../../storage')
+ },
+
+ // Files information
+ files: {
+ count: files.length,
+ total_size: files.reduce((sum, file) => sum + (file.size || 0), 0),
+ checksums: await this.generateFileChecksums(files),
+ manifest: files.map(file => ({
+ path: file.relativePath || file.path,
+ size: file.size,
+ modified: file.modified,
+ checksum: file.checksum,
+ type: this.getFileType(file.path),
+ permissions: file.permissions
+ }))
+ },
+
+ // Database information
+ database: {
+ type: databaseInfo.type || this.getDatabaseType(),
+ backup_file: databaseInfo.backupFile,
+ size: databaseInfo.size,
+ checksum: databaseInfo.checksum,
+ tables: databaseInfo.tables || {},
+ row_counts: databaseInfo.rowCounts || {},
+ schema_version: await this.getSchemaVersion()
+ },
+
+ // Verification information
+ verification: {
+ total_checksum: null, // Will be calculated after manifest is complete
+ file_count_check: files.length,
+ size_check: files.reduce((sum, file) => sum + (file.size || 0), 0),
+ integrity_timestamp: new Date().toISOString()
+ },
+
+ // Custom metadata
+ metadata: {
+ ...customMetadata,
+ backup_settings: await this.getBackupSettings(),
+ active_events_count: await this.getActiveEventsCount(),
+ archived_events_count: await this.getArchivedEventsCount(),
+ total_photos_count: await this.getTotalPhotosCount()
+ }
+ };
+
+ // Calculate total checksum of the manifest
+ manifest.verification.total_checksum = this.calculateManifestChecksum(manifest);
+
+ return manifest;
+ }
+
+ /**
+ * Save manifest to file
+ * @param {Object} manifest - Manifest object to save
+ * @param {string} filePath - Path to save the manifest
+ * @param {string} format - 'json' or 'yaml'
+ */
+ async saveManifest(manifest, filePath, format = 'json') {
+ try {
+ let content;
+
+ if (format === 'yaml') {
+ content = yaml.dump(manifest, {
+ indent: 2,
+ lineWidth: -1,
+ noRefs: true,
+ sortKeys: true
+ });
+ } else {
+ content = JSON.stringify(manifest, null, 2);
+ }
+
+ await fs.writeFile(filePath, content, 'utf8');
+ logger.info(`Manifest saved to ${filePath} (format: ${format})`);
+
+ return filePath;
+ } catch (error) {
+ logger.error('Failed to save manifest:', error);
+ throw error;
+ }
+ }
+
+ /**
+ * Load and validate an existing manifest
+ * @param {string} filePath - Path to the manifest file
+ * @returns {Object} Loaded and validated manifest
+ */
+ async loadManifest(filePath) {
+ try {
+ const content = await fs.readFile(filePath, 'utf8');
+ let manifest;
+
+ // Detect format and parse
+ if (filePath.endsWith('.yaml') || filePath.endsWith('.yml')) {
+ manifest = yaml.load(content);
+ } else {
+ manifest = JSON.parse(content);
+ }
+
+ // Validate manifest structure
+ this.validateManifest(manifest);
+
+ return manifest;
+ } catch (error) {
+ logger.error('Failed to load manifest:', error);
+ throw error;
+ }
+ }
+
+ /**
+ * Validate manifest structure and integrity
+ * @param {Object} manifest - Manifest to validate
+ * @throws {Error} If validation fails
+ */
+ validateManifest(manifest) {
+ // Check required sections
+ const requiredSections = ['manifest', 'backup', 'system', 'application', 'files', 'database', 'verification'];
+ for (const section of requiredSections) {
+ if (!manifest[section]) {
+ throw new Error(`Missing required section: ${section}`);
+ }
+ }
+
+ // Validate manifest version
+ if (!manifest.manifest.version) {
+ throw new Error('Missing manifest version');
+ }
+
+ // Validate file checksums
+ if (manifest.files.count !== manifest.files.manifest.length) {
+ throw new Error('File count mismatch');
+ }
+
+ // Validate total checksum
+ const calculatedChecksum = this.calculateManifestChecksum(manifest);
+ if (manifest.verification.total_checksum !== calculatedChecksum) {
+ throw new Error('Manifest checksum verification failed');
+ }
+
+ logger.info('Manifest validation passed');
+ return true;
+ }
+
+ /**
+ * Compare two manifests for incremental backup
+ * @param {Object} currentManifest - Current backup manifest
+ * @param {Object} parentManifest - Parent backup manifest
+ * @returns {Object} Comparison results
+ */
+ compareManifests(currentManifest, parentManifest) {
+ const comparison = {
+ added_files: [],
+ modified_files: [],
+ deleted_files: [],
+ unchanged_files: [],
+ size_difference: 0,
+ database_changes: {}
+ };
+
+ // Create file maps for easy comparison
+ const currentFiles = new Map(
+ currentManifest.files.manifest.map(f => [f.path, f])
+ );
+ const parentFiles = new Map(
+ parentManifest.files.manifest.map(f => [f.path, f])
+ );
+
+ // Find added and modified files
+ for (const [path, file] of currentFiles) {
+ const parentFile = parentFiles.get(path);
+ if (!parentFile) {
+ comparison.added_files.push(file);
+ comparison.size_difference += file.size;
+ } else if (file.checksum !== parentFile.checksum) {
+ comparison.modified_files.push(file);
+ comparison.size_difference += file.size - parentFile.size;
+ } else {
+ comparison.unchanged_files.push(file);
+ }
+ }
+
+ // Find deleted files
+ for (const [path, file] of parentFiles) {
+ if (!currentFiles.has(path)) {
+ comparison.deleted_files.push(file);
+ comparison.size_difference -= file.size;
+ }
+ }
+
+ // Compare database info
+ comparison.database_changes = {
+ size_difference: currentManifest.database.size - parentManifest.database.size,
+ checksum_changed: currentManifest.database.checksum !== parentManifest.database.checksum,
+ schema_version_changed: currentManifest.database.schema_version !== parentManifest.database.schema_version
+ };
+
+ return comparison;
+ }
+
+ /**
+ * Generate incremental manifest based on parent
+ * @param {Object} options - Manifest generation options
+ * @param {Object} parentManifest - Parent backup manifest
+ * @returns {Object} Incremental manifest
+ */
+ async generateIncrementalManifest(options, parentManifest) {
+ const fullManifest = await this.generateManifest({
+ ...options,
+ backupType: 'incremental'
+ });
+
+ const comparison = this.compareManifests(fullManifest, parentManifest);
+
+ // Add incremental-specific information
+ fullManifest.incremental = {
+ parent_backup_id: parentManifest.backup.id,
+ parent_timestamp: parentManifest.backup.timestamp,
+ changes: {
+ added_files_count: comparison.added_files.length,
+ modified_files_count: comparison.modified_files.length,
+ deleted_files_count: comparison.deleted_files.length,
+ unchanged_files_count: comparison.unchanged_files.length,
+ size_difference: comparison.size_difference
+ },
+ added_files: comparison.added_files.map(f => f.path),
+ modified_files: comparison.modified_files.map(f => f.path),
+ deleted_files: comparison.deleted_files.map(f => f.path)
+ };
+
+ return fullManifest;
+ }
+
+ // Helper methods
+
+ generateBackupId() {
+ const timestamp = new Date().toISOString().replace(/[:-]/g, '').replace('T', '-').split('.')[0];
+ const random = crypto.randomBytes(4).toString('hex');
+ return `backup-${timestamp}-${random}`;
+ }
+
+ async generateFileChecksums(files) {
+ const checksums = {};
+ for (const file of files) {
+ if (file.checksum) {
+ checksums[file.relativePath || file.path] = file.checksum;
+ }
+ }
+ return checksums;
+ }
+
+ getFileType(filePath) {
+ const ext = path.extname(filePath).toLowerCase();
+ const typeMap = {
+ '.jpg': 'image',
+ '.jpeg': 'image',
+ '.png': 'image',
+ '.gif': 'image',
+ '.webp': 'image',
+ '.zip': 'archive',
+ '.sql': 'database',
+ '.db': 'database',
+ '.json': 'config',
+ '.yaml': 'config',
+ '.yml': 'config'
+ };
+ return typeMap[ext] || 'other';
+ }
+
+ getDatabaseType() {
+ return process.env.DB_TYPE === 'postgresql' ? 'postgresql' : 'sqlite';
+ }
+
+ async getSchemaVersion() {
+ try {
+ const result = await db('migrations')
+ .orderBy('run_at', 'desc')
+ .first();
+ return result ? result.migration_name : 'unknown';
+ } catch (error) {
+ return 'unknown';
+ }
+ }
+
+ async getBackupSettings() {
+ try {
+ const settings = await db('app_settings')
+ .where('setting_type', 'backup')
+ .select('setting_key', 'setting_value');
+
+ const config = {};
+ settings.forEach(setting => {
+ try {
+ config[setting.setting_key] = JSON.parse(setting.setting_value);
+ } catch (e) {
+ config[setting.setting_key] = setting.setting_value;
+ }
+ });
+
+ return config;
+ } catch (error) {
+ return {};
+ }
+ }
+
+ async getActiveEventsCount() {
+ try {
+ const result = await db('events')
+ .where('status', 'active')
+ .count('* as count')
+ .first();
+ return result ? parseInt(result.count) : 0;
+ } catch (error) {
+ return 0;
+ }
+ }
+
+ async getArchivedEventsCount() {
+ try {
+ const result = await db('events')
+ .where('status', 'archived')
+ .count('* as count')
+ .first();
+ return result ? parseInt(result.count) : 0;
+ } catch (error) {
+ return 0;
+ }
+ }
+
+ async getTotalPhotosCount() {
+ try {
+ const result = await db('photos')
+ .count('* as count')
+ .first();
+ return result ? parseInt(result.count) : 0;
+ } catch (error) {
+ return 0;
+ }
+ }
+
+ calculateManifestChecksum(manifest) {
+ // Create a copy without the checksum field
+ const manifestCopy = JSON.parse(JSON.stringify(manifest));
+ if (manifestCopy.verification) {
+ delete manifestCopy.verification.total_checksum;
+ }
+
+ // Calculate SHA256 of the sorted JSON
+ const content = JSON.stringify(manifestCopy, Object.keys(manifestCopy).sort());
+ return crypto.createHash('sha256').update(content).digest('hex');
+ }
+
+ /**
+ * Generate a summary report from a manifest
+ * @param {Object} manifest - Manifest to summarize
+ * @returns {string} Human-readable summary
+ */
+ generateSummaryReport(manifest) {
+ const report = [];
+
+ report.push('=== BACKUP MANIFEST SUMMARY ===');
+ report.push(`Backup ID: ${manifest.backup.id}`);
+ report.push(`Type: ${manifest.backup.type}`);
+ report.push(`Created: ${manifest.backup.timestamp}`);
+
+ if (manifest.backup.parent_backup_id) {
+ report.push(`Parent Backup: ${manifest.backup.parent_backup_id}`);
+ }
+
+ report.push('\n--- System Information ---');
+ report.push(`Host: ${manifest.system.hostname}`);
+ report.push(`Platform: ${manifest.system.platform} ${manifest.system.os_release}`);
+ report.push(`Architecture: ${manifest.system.architecture}`);
+
+ report.push('\n--- Application Information ---');
+ report.push(`App Version: ${manifest.application.version}`);
+ report.push(`Node Version: ${manifest.application.node_version}`);
+ report.push(`Environment: ${manifest.application.environment}`);
+
+ report.push('\n--- Files Summary ---');
+ report.push(`Total Files: ${manifest.files.count}`);
+ report.push(`Total Size: ${(manifest.files.total_size / 1024 / 1024).toFixed(2)} MB`);
+
+ if (manifest.incremental) {
+ report.push('\n--- Incremental Changes ---');
+ report.push(`Added Files: ${manifest.incremental.changes.added_files_count}`);
+ report.push(`Modified Files: ${manifest.incremental.changes.modified_files_count}`);
+ report.push(`Deleted Files: ${manifest.incremental.changes.deleted_files_count}`);
+ report.push(`Size Difference: ${(manifest.incremental.changes.size_difference / 1024 / 1024).toFixed(2)} MB`);
+ }
+
+ report.push('\n--- Database Information ---');
+ report.push(`Type: ${manifest.database.type}`);
+ report.push(`Size: ${manifest.database.size ? (manifest.database.size / 1024 / 1024).toFixed(2) + ' MB' : 'N/A'}`);
+ report.push(`Schema Version: ${manifest.database.schema_version}`);
+
+ report.push('\n--- Content Statistics ---');
+ report.push(`Active Events: ${manifest.metadata.active_events_count}`);
+ report.push(`Archived Events: ${manifest.metadata.archived_events_count}`);
+ report.push(`Total Photos: ${manifest.metadata.total_photos_count}`);
+
+ report.push('\n--- Verification ---');
+ report.push(`Manifest Checksum: ${manifest.verification.total_checksum}`);
+ report.push(`Integrity Timestamp: ${manifest.verification.integrity_timestamp}`);
+
+ return report.join('\n');
+ }
+}
+
+// Export singleton instance
+module.exports = new BackupManifestGenerator();
\ No newline at end of file
diff --git a/backend/src/services/backupService.js b/backend/src/services/backupService.js
new file mode 100644
index 0000000..1691efd
--- /dev/null
+++ b/backend/src/services/backupService.js
@@ -0,0 +1,1141 @@
+const cron = require('node-cron');
+const path = require('path');
+const fs = require('fs').promises;
+const crypto = require('crypto');
+const { exec } = require('child_process');
+const { promisify } = require('util');
+const execAsync = promisify(exec);
+const { db } = require('../database/db');
+const { queueEmail } = require('./emailProcessor');
+const logger = require('../utils/logger');
+const { formatBoolean } = require('../utils/dbCompat');
+const backupManifest = require('./backupManifest');
+const S3StorageAdapter = require('./storage/s3Storage');
+
+// Backup job reference
+let backupJob = null;
+let backupConfig = null;
+let isRunning = false;
+
+// Storage paths
+const getStoragePath = () => process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
+
+/**
+ * Calculate file checksum using SHA256
+ */
+async function calculateChecksum(filePath) {
+ const hash = crypto.createHash('sha256');
+ const stream = require('fs').createReadStream(filePath);
+
+ return new Promise((resolve, reject) => {
+ stream.on('data', data => hash.update(data));
+ stream.on('end', () => resolve(hash.digest('hex')));
+ stream.on('error', reject);
+ });
+}
+
+/**
+ * Get database backup information
+ */
+async function getDatabaseBackupInfo() {
+ try {
+ // Check for recent database backup
+ const recentDbBackup = await db('database_backup_runs')
+ .where('status', 'completed')
+ .orderBy('completed_at', 'desc')
+ .first();
+
+ if (recentDbBackup && recentDbBackup.file_path) {
+ // Check if database has changed since backup
+ const hasChanged = await hasDatabaseChanged(recentDbBackup.completed_at);
+
+ return {
+ type: recentDbBackup.backup_type,
+ backupFile: recentDbBackup.file_path,
+ size: recentDbBackup.file_size_bytes,
+ checksum: recentDbBackup.checksum,
+ tables: recentDbBackup.statistics ? JSON.parse(recentDbBackup.statistics).tables : {},
+ rowCounts: recentDbBackup.table_checksums ? JSON.parse(recentDbBackup.table_checksums) : {},
+ hasChanged: hasChanged,
+ backupTime: recentDbBackup.completed_at
+ };
+ }
+
+ return {
+ type: process.env.DB_TYPE === 'postgresql' ? 'postgresql' : 'sqlite',
+ backupFile: null,
+ size: 0,
+ checksum: null,
+ tables: {},
+ rowCounts: {},
+ hasChanged: true,
+ backupTime: null
+ };
+ } catch (error) {
+ logger.error('Failed to get database backup info:', error);
+ return {
+ type: 'unknown',
+ backupFile: null,
+ size: 0,
+ checksum: null,
+ tables: {},
+ rowCounts: {},
+ hasChanged: true,
+ backupTime: null
+ };
+ }
+}
+
+/**
+ * Check if database has changed since a given time
+ */
+async function hasDatabaseChanged(sinceTime) {
+ try {
+ // List of tables that track modifications
+ const tablesToCheck = [
+ 'events',
+ 'photos',
+ 'admin_users',
+ 'app_settings',
+ 'email_queue',
+ 'access_logs'
+ ];
+
+ for (const table of tablesToCheck) {
+ try {
+ // Check for updated_at timestamps
+ const hasUpdates = await db(table)
+ .where('updated_at', '>', sinceTime)
+ .limit(1)
+ .first();
+
+ if (hasUpdates) {
+ logger.debug(`Database table ${table} has changes since ${sinceTime}`);
+ return true;
+ }
+
+ // Also check created_at for new records
+ const hasNewRecords = await db(table)
+ .where('created_at', '>', sinceTime)
+ .limit(1)
+ .first();
+
+ if (hasNewRecords) {
+ logger.debug(`Database table ${table} has new records since ${sinceTime}`);
+ return true;
+ }
+ } catch (error) {
+ // Table might not exist or not have timestamp columns
+ logger.debug(`Could not check table ${table} for changes:`, error.message);
+ }
+ }
+
+ return false;
+ } catch (error) {
+ logger.error('Failed to check database changes:', error);
+ // Assume changed if we can't check
+ return true;
+ }
+}
+
+/**
+ * Get backup configuration from database
+ */
+async function getBackupConfig() {
+ try {
+ const settings = await db('app_settings')
+ .where('setting_type', 'backup')
+ .select('setting_key', 'setting_value');
+
+ const config = {};
+ settings.forEach(setting => {
+ try {
+ config[setting.setting_key] = JSON.parse(setting.setting_value);
+ } catch (e) {
+ config[setting.setting_key] = setting.setting_value;
+ }
+ });
+
+ return config;
+ } catch (error) {
+ logger.error('Failed to get backup configuration:', error);
+ return null;
+ }
+}
+
+/**
+ * Get list of files to backup
+ */
+async function getFilesToBackup(includeArchived = true) {
+ const files = [];
+ const storagePath = getStoragePath();
+
+ try {
+ // Active events
+ const activePath = path.join(storagePath, 'events/active');
+ await scanDirectory(activePath, files, storagePath);
+
+ // Archived events (if enabled)
+ if (includeArchived) {
+ const archivePath = path.join(storagePath, 'events/archived');
+ await scanDirectory(archivePath, files, storagePath);
+ }
+
+ // Thumbnails
+ const thumbsPath = path.join(storagePath, 'thumbnails');
+ await scanDirectory(thumbsPath, files, storagePath);
+
+ // Uploads (logos, favicons, etc.)
+ const uploadsPath = path.join(storagePath, 'uploads');
+ await scanDirectory(uploadsPath, files, storagePath);
+
+ return files;
+ } catch (error) {
+ logger.error('Failed to get files to backup:', error);
+ throw error;
+ }
+}
+
+/**
+ * Recursively scan directory for files
+ */
+async function scanDirectory(dirPath, fileList, basePath, excludePatterns = []) {
+ try {
+ const entries = await fs.readdir(dirPath, { withFileTypes: true });
+
+ for (const entry of entries) {
+ const fullPath = path.join(dirPath, entry.name);
+ const relativePath = path.relative(basePath, fullPath);
+
+ // Check exclude patterns
+ if (excludePatterns.some(pattern => {
+ if (pattern.includes('*')) {
+ return new RegExp(pattern.replace(/\*/g, '.*')).test(entry.name);
+ }
+ return entry.name === pattern;
+ })) {
+ continue;
+ }
+
+ if (entry.isDirectory()) {
+ await scanDirectory(fullPath, fileList, basePath, excludePatterns);
+ } else if (entry.isFile()) {
+ const stats = await fs.stat(fullPath);
+ fileList.push({
+ path: fullPath,
+ relativePath: relativePath,
+ size: stats.size,
+ modified: stats.mtime
+ });
+ }
+ }
+ } catch (error) {
+ if (error.code !== 'ENOENT') {
+ logger.error(`Failed to scan directory ${dirPath}:`, error);
+ }
+ }
+}
+
+/**
+ * Check if file has changed since last backup
+ */
+async function hasFileChanged(filePath, checksum) {
+ try {
+ const fileState = await db('backup_file_states')
+ .where('file_path', filePath)
+ .first();
+
+ return !fileState || fileState.checksum !== checksum;
+ } catch (error) {
+ logger.error('Failed to check file state:', error);
+ return true; // Assume changed if we can't check
+ }
+}
+
+/**
+ * Update file state in database
+ */
+async function updateFileState(filePath, checksum, size, modified) {
+ try {
+ const existing = await db('backup_file_states')
+ .where('file_path', filePath)
+ .first();
+
+ const data = {
+ file_path: filePath,
+ checksum: checksum,
+ size_bytes: size,
+ last_modified: modified,
+ last_backed_up: new Date()
+ };
+
+ if (existing) {
+ await db('backup_file_states')
+ .where('id', existing.id)
+ .update(data);
+ } else {
+ await db('backup_file_states').insert(data);
+ }
+ } catch (error) {
+ logger.error('Failed to update file state:', error);
+ }
+}
+
+/**
+ * Perform local directory backup
+ */
+async function performLocalBackup(config, files) {
+ const destPath = config.backup_destination_path;
+ const storagePath = getStoragePath();
+ let backedUpCount = 0;
+ let backedUpSize = 0;
+ const backedUpFiles = [];
+
+ // Ensure destination exists
+ await fs.mkdir(destPath, { recursive: true });
+
+ for (const file of files) {
+ try {
+ // Skip large files if configured
+ const maxSizeMB = config.backup_max_file_size_mb || 5000;
+ if (file.size > maxSizeMB * 1024 * 1024) {
+ logger.warn(`Skipping large file: ${file.relativePath} (${(file.size / 1024 / 1024).toFixed(2)} MB)`);
+ continue;
+ }
+
+ // Calculate checksum
+ const checksum = await calculateChecksum(file.path);
+ file.checksum = checksum; // Add checksum to file object
+
+ // Check if file has changed
+ const changed = await hasFileChanged(file.relativePath, checksum);
+ if (!changed) {
+ continue;
+ }
+
+ // Copy file
+ const destFilePath = path.join(destPath, file.relativePath);
+ const destDir = path.dirname(destFilePath);
+ await fs.mkdir(destDir, { recursive: true });
+ await fs.copyFile(file.path, destFilePath);
+
+ // Update state
+ await updateFileState(file.relativePath, checksum, file.size, file.modified);
+
+ backedUpCount++;
+ backedUpSize += file.size;
+ backedUpFiles.push(file.relativePath);
+ } catch (error) {
+ logger.error(`Failed to backup file ${file.relativePath}:`, error);
+ }
+ }
+
+ return { backedUpCount, backedUpSize, backedUpFiles };
+}
+
+/**
+ * Perform rsync backup
+ */
+async function performRsyncBackup(config, files) {
+ const storagePath = getStoragePath();
+ const host = config.backup_rsync_host;
+ const user = config.backup_rsync_user;
+ const remotePath = config.backup_rsync_path;
+ const sshKey = config.backup_rsync_ssh_key;
+
+ if (!host || !remotePath) {
+ throw new Error('Rsync configuration incomplete');
+ }
+
+ // Build rsync command
+ const rsyncOptions = [
+ '-avz', // archive, verbose, compress
+ '--delete', // remove deleted files
+ '--stats' // show statistics
+ ];
+
+ if (sshKey) {
+ rsyncOptions.push(`-e "ssh -i ${sshKey} -o StrictHostKeyChecking=no"`);
+ }
+
+ // Add exclude patterns
+ const excludePatterns = config.backup_exclude_patterns || [];
+ excludePatterns.forEach(pattern => {
+ rsyncOptions.push(`--exclude="${pattern}"`);
+ });
+
+ const source = `${storagePath}/`;
+ const destination = user ? `${user}@${host}:${remotePath}` : `${host}:${remotePath}`;
+
+ const rsyncCommand = `rsync ${rsyncOptions.join(' ')} "${source}" "${destination}"`;
+
+ try {
+ const { stdout, stderr } = await execAsync(rsyncCommand);
+
+ // Parse rsync stats
+ const stats = parseRsyncStats(stdout);
+
+ // Update file states for successfully synced files
+ for (const file of files) {
+ try {
+ const checksum = await calculateChecksum(file.path);
+ await updateFileState(file.relativePath, checksum, file.size, file.modified);
+ } catch (error) {
+ logger.error(`Failed to update state for ${file.relativePath}:`, error);
+ }
+ }
+
+ return {
+ backedUpCount: stats.filesTransferred || files.length,
+ backedUpSize: stats.totalSize || files.reduce((sum, f) => sum + f.size, 0),
+ backedUpFiles: files.map(f => f.relativePath)
+ };
+ } catch (error) {
+ logger.error('Rsync backup failed:', error);
+ throw new Error(`Rsync backup failed: ${error.message}`);
+ }
+}
+
+/**
+ * Parse rsync statistics from output
+ */
+function parseRsyncStats(output) {
+ const stats = {};
+
+ // Extract files transferred
+ const filesMatch = output.match(/Number of files transferred: (\d+)/);
+ if (filesMatch) {
+ stats.filesTransferred = parseInt(filesMatch[1]);
+ }
+
+ // Extract total size
+ const sizeMatch = output.match(/Total file size: ([\d,]+) bytes/);
+ if (sizeMatch) {
+ stats.totalSize = parseInt(sizeMatch[1].replace(/,/g, ''));
+ }
+
+ return stats;
+}
+
+/**
+ * Perform S3-compatible backup
+ */
+async function performS3Backup(config, files) {
+ let s3Client = null;
+ let backedUpCount = 0;
+ let backedUpSize = 0;
+ const backedUpFiles = [];
+ const storagePath = getStoragePath();
+
+ try {
+ // Initialize S3 client with configuration
+ const s3Config = {
+ bucket: config.backup_s3_bucket,
+ region: config.backup_s3_region || 'us-east-1',
+ endpoint: config.backup_s3_endpoint,
+ accessKeyId: config.backup_s3_access_key,
+ secretAccessKey: config.backup_s3_secret_key,
+ forcePathStyle: config.backup_s3_force_path_style || false,
+ sslEnabled: config.backup_s3_ssl_enabled !== false, // Default true
+ maxRetries: 3,
+ retryDelay: 1000
+ };
+
+ // Validate required S3 configuration
+ if (!s3Config.bucket || !s3Config.accessKeyId || !s3Config.secretAccessKey) {
+ throw new Error('S3 backup configuration incomplete: bucket, access key, and secret key are required');
+ }
+
+ // Create S3 client
+ s3Client = new S3StorageAdapter(s3Config);
+
+ // Test connection
+ logger.info('Testing S3 connection...');
+ await s3Client.testConnection();
+
+ // Determine backup prefix based on date and configuration
+ const now = new Date();
+ const datePrefix = `${now.getFullYear()}/${String(now.getMonth() + 1).padStart(2, '0')}/${String(now.getDate()).padStart(2, '0')}`;
+ const backupId = `backup-${now.getTime()}`;
+ const s3Prefix = config.backup_s3_prefix ?
+ path.posix.join(config.backup_s3_prefix, datePrefix, backupId) :
+ path.posix.join('backups', datePrefix, backupId);
+
+ logger.info(`Starting S3 backup to prefix: ${s3Prefix}`);
+
+ // Process each file
+ for (const file of files) {
+ try {
+ // Skip large files if configured
+ const maxSizeMB = config.backup_max_file_size_mb || 5000;
+ if (file.size > maxSizeMB * 1024 * 1024) {
+ logger.warn(`Skipping large file: ${file.relativePath} (${(file.size / 1024 / 1024).toFixed(2)} MB)`);
+ continue;
+ }
+
+ // Calculate checksum
+ const checksum = await calculateChecksum(file.path);
+ file.checksum = checksum; // Add checksum to file object
+
+ // Check if file has changed
+ const changed = await hasFileChanged(file.relativePath, checksum);
+ if (!changed && config.backup_incremental !== false) {
+ continue;
+ }
+
+ // Determine S3 key for the file
+ const s3Key = path.posix.join(s3Prefix, file.relativePath);
+
+ // Upload file to S3
+ logger.debug(`Uploading ${file.relativePath} to S3 key: ${s3Key}`);
+
+ let uploadStartTime = Date.now();
+ await s3Client.upload(file.path, s3Key, {
+ metadata: {
+ 'original-path': file.relativePath,
+ 'checksum': checksum,
+ 'backup-id': backupId,
+ 'backup-time': now.toISOString()
+ },
+ onProgress: (loaded, total) => {
+ const percentComplete = Math.round((loaded / total) * 100);
+ if (percentComplete % 25 === 0) { // Log at 25%, 50%, 75%, 100%
+ logger.debug(`Upload progress for ${file.relativePath}: ${percentComplete}%`);
+ }
+ }
+ });
+
+ const uploadDuration = Date.now() - uploadStartTime;
+ logger.debug(`Uploaded ${file.relativePath} in ${uploadDuration}ms`);
+
+ // Update state
+ await updateFileState(file.relativePath, checksum, file.size, file.modified);
+
+ backedUpCount++;
+ backedUpSize += file.size;
+ backedUpFiles.push(file.relativePath);
+
+ } catch (error) {
+ logger.error(`Failed to backup file ${file.relativePath} to S3:`, error);
+ // Continue with other files even if one fails
+ }
+ }
+
+ // Check if database backup should be included
+ if (config.backup_include_database !== false) {
+ try {
+ logger.info('Including database backup in S3 backup...');
+ const dbInfo = await getDatabaseBackupInfo();
+
+ if (dbInfo.backupFile && await fs.stat(dbInfo.backupFile).catch(() => null)) {
+ // Upload database backup file
+ const dbFileName = path.basename(dbInfo.backupFile);
+ const dbS3Key = path.posix.join(s3Prefix, 'database', dbFileName);
+
+ await s3Client.upload(dbInfo.backupFile, dbS3Key, {
+ metadata: {
+ 'backup-type': 'database',
+ 'database-type': dbInfo.type,
+ 'checksum': dbInfo.checksum,
+ 'backup-id': backupId,
+ 'backup-time': now.toISOString()
+ }
+ });
+
+ backedUpCount++;
+ backedUpSize += dbInfo.size;
+ logger.info(`Database backup uploaded to S3: ${dbS3Key}`);
+ } else {
+ logger.warn('No recent database backup found to include in S3 backup');
+ }
+ } catch (error) {
+ logger.error('Failed to include database backup in S3:', error);
+ }
+ }
+
+ // Create and upload a backup summary file
+ try {
+ const summary = {
+ backupId: backupId,
+ timestamp: now.toISOString(),
+ s3Bucket: config.backup_s3_bucket,
+ s3Prefix: s3Prefix,
+ filesBackedUp: backedUpCount,
+ totalSize: backedUpSize,
+ totalSizeFormatted: formatBytes(backedUpSize),
+ configuration: {
+ incremental: config.backup_incremental !== false,
+ includeArchived: config.backup_include_archived,
+ includeDatabase: config.backup_include_database !== false,
+ maxFileSizeMB: config.backup_max_file_size_mb || 5000
+ }
+ };
+
+ const summaryJson = JSON.stringify(summary, null, 2);
+ const summaryS3Key = path.posix.join(s3Prefix, 'backup-summary.json');
+
+ // Create a temporary file for the summary
+ const tempSummaryPath = path.join(storagePath, `temp-summary-${backupId}.json`);
+ await fs.writeFile(tempSummaryPath, summaryJson);
+
+ await s3Client.upload(tempSummaryPath, summaryS3Key, {
+ contentType: 'application/json',
+ metadata: {
+ 'backup-id': backupId,
+ 'backup-type': 'summary'
+ }
+ });
+
+ // Clean up temp file
+ await fs.unlink(tempSummaryPath).catch(() => {});
+
+ logger.info(`Backup summary uploaded to S3: ${summaryS3Key}`);
+ } catch (error) {
+ logger.error('Failed to upload backup summary:', error);
+ }
+
+ logger.info(`S3 backup completed: ${backedUpCount} files, ${formatBytes(backedUpSize)} uploaded to ${s3Prefix}`);
+
+ return {
+ backedUpCount,
+ backedUpSize,
+ backedUpFiles,
+ s3Prefix,
+ s3Bucket: config.backup_s3_bucket
+ };
+
+ } catch (error) {
+ logger.error('S3 backup failed:', error);
+ throw new Error(`S3 backup failed: ${error.message}`);
+ }
+}
+
+/**
+ * Format bytes to human readable string
+ */
+function formatBytes(bytes, decimals = 2) {
+ if (bytes === 0) return '0 Bytes';
+
+ const k = 1024;
+ const dm = decimals < 0 ? 0 : decimals;
+ const sizes = ['Bytes', 'KB', 'MB', 'GB', 'TB'];
+
+ const i = Math.floor(Math.log(bytes) / Math.log(k));
+
+ return parseFloat((bytes / Math.pow(k, i)).toFixed(dm)) + ' ' + sizes[i];
+}
+
+/**
+ * Run backup process
+ */
+async function runBackup() {
+ if (isRunning) {
+ logger.warn('Backup already running, skipping');
+ return;
+ }
+
+ isRunning = true;
+ const startTime = new Date();
+ let backupRun = null;
+
+ try {
+ // Get current configuration
+ const config = await getBackupConfig();
+ if (!config.backup_enabled) {
+ logger.info('Backup is disabled, skipping');
+ return;
+ }
+
+ // Create backup run record
+ const [runId] = await db('backup_runs').insert({
+ started_at: startTime,
+ status: 'running',
+ backup_type: 'scheduled'
+ });
+
+ backupRun = { id: runId };
+
+ // Get files to backup
+ const files = await getFilesToBackup(config.backup_include_archived);
+ logger.info(`Found ${files.length} files to check for backup`);
+
+ // Perform backup based on destination type
+ let result;
+ switch (config.backup_destination_type) {
+ case 'local':
+ result = await performLocalBackup(config, files);
+ break;
+ case 'rsync':
+ result = await performRsyncBackup(config, files);
+ break;
+ case 's3':
+ result = await performS3Backup(config, files);
+ break;
+ default:
+ throw new Error(`Unknown backup destination type: ${config.backup_destination_type}`);
+ }
+
+ // Calculate duration
+ const endTime = new Date();
+ const durationSeconds = Math.round((endTime - startTime) / 1000);
+
+ // Generate backup manifest
+ let manifestPath = null;
+ try {
+ logger.info('Generating backup manifest...');
+
+ // Get database backup info if available
+ const databaseInfo = await getDatabaseBackupInfo();
+
+ // Determine if this is an incremental backup
+ const lastSuccessfulBackup = await db('backup_runs')
+ .where('status', 'completed')
+ .whereNot('id', runId)
+ .orderBy('completed_at', 'desc')
+ .first();
+
+ // Prepare backup path based on destination type
+ let backupPath;
+ if (config.backup_destination_type === 's3') {
+ backupPath = `s3://${result.s3Bucket}/${result.s3Prefix}`;
+ } else {
+ backupPath = config.backup_destination_path || config.backup_destination_type;
+ }
+
+ let manifest;
+ const manifestOptions = {
+ backupType: lastSuccessfulBackup ? 'incremental' : 'full',
+ backupPath: backupPath,
+ files: files.filter(f => result.backedUpFiles && result.backedUpFiles.includes(f.relativePath)),
+ databaseInfo: databaseInfo,
+ parentBackupId: lastSuccessfulBackup ? lastSuccessfulBackup.manifest_id : null,
+ format: config.backup_manifest_format || 'json',
+ customMetadata: {
+ backup_run_id: runId,
+ destination_type: config.backup_destination_type,
+ operator: 'system',
+ reason: 'scheduled',
+ retentionDays: config.backup_retention_days || 30,
+ // Add S3-specific metadata if applicable
+ ...(config.backup_destination_type === 's3' ? {
+ s3_bucket: result.s3Bucket,
+ s3_prefix: result.s3Prefix,
+ s3_region: config.backup_s3_region || 'us-east-1',
+ s3_endpoint: config.backup_s3_endpoint
+ } : {})
+ }
+ };
+
+ if (lastSuccessfulBackup && lastSuccessfulBackup.manifest_path) {
+ try {
+ const parentManifest = await backupManifest.loadManifest(lastSuccessfulBackup.manifest_path);
+ manifest = await backupManifest.generateIncrementalManifest(manifestOptions, parentManifest);
+ } catch (error) {
+ logger.warn('Failed to load parent manifest, generating full manifest:', error);
+ manifest = await backupManifest.generateManifest(manifestOptions);
+ }
+ } else {
+ manifest = await backupManifest.generateManifest(manifestOptions);
+ }
+
+ // Save manifest
+ const manifestFileName = `backup-manifest-${manifest.backup.id}.${config.backup_manifest_format || 'json'}`;
+
+ if (config.backup_destination_type === 's3') {
+ // For S3 backups, save manifest locally first then upload to S3
+ const tempManifestDir = path.join(getStoragePath(), 'temp');
+ await fs.mkdir(tempManifestDir, { recursive: true });
+
+ const tempManifestPath = path.join(tempManifestDir, manifestFileName);
+ await backupManifest.saveManifest(manifest, tempManifestPath, config.backup_manifest_format || 'json');
+
+ // Upload manifest to S3
+ try {
+ const s3Config = {
+ bucket: config.backup_s3_bucket,
+ region: config.backup_s3_region || 'us-east-1',
+ endpoint: config.backup_s3_endpoint,
+ accessKeyId: config.backup_s3_access_key,
+ secretAccessKey: config.backup_s3_secret_key,
+ forcePathStyle: config.backup_s3_force_path_style || false,
+ sslEnabled: config.backup_s3_ssl_enabled !== false
+ };
+
+ const s3Client = new S3StorageAdapter(s3Config);
+ const manifestS3Key = path.posix.join(result.s3Prefix, 'manifests', manifestFileName);
+
+ await s3Client.upload(tempManifestPath, manifestS3Key, {
+ contentType: config.backup_manifest_format === 'xml' ? 'application/xml' : 'application/json',
+ metadata: {
+ 'backup-id': manifest.backup.id,
+ 'backup-type': 'manifest',
+ 'manifest-version': manifest.version
+ }
+ });
+
+ // Clean up temp file
+ await fs.unlink(tempManifestPath).catch(() => {});
+
+ // Store S3 path as manifest path
+ manifestPath = `s3://${config.backup_s3_bucket}/${manifestS3Key}`;
+ logger.info(`Backup manifest uploaded to S3: ${manifestPath}`);
+
+ } catch (error) {
+ logger.error('Failed to upload manifest to S3:', error);
+ // Keep local path as fallback
+ manifestPath = tempManifestPath;
+ }
+ } else {
+ // For local/rsync backups, save to configured directory
+ const manifestDir = config.backup_manifest_path || path.join(config.backup_destination_path || '/backup', 'manifests');
+ await fs.mkdir(manifestDir, { recursive: true });
+
+ manifestPath = path.join(manifestDir, manifestFileName);
+ await backupManifest.saveManifest(manifest, manifestPath, config.backup_manifest_format || 'json');
+
+ logger.info(`Backup manifest saved to ${manifestPath}`);
+ }
+
+ } catch (error) {
+ logger.error('Failed to generate backup manifest:', error);
+ // Don't fail the entire backup for manifest generation failure
+ }
+
+ // Update backup run record
+ await db('backup_runs')
+ .where('id', runId)
+ .update({
+ completed_at: endTime,
+ status: 'completed',
+ files_backed_up: result.backedUpCount,
+ total_size_bytes: result.backedUpSize,
+ duration_seconds: durationSeconds,
+ manifest_path: manifestPath,
+ manifest_id: manifestPath ? path.basename(manifestPath, path.extname(manifestPath)) : null,
+ statistics: JSON.stringify({
+ totalFilesChecked: files.length,
+ filesBackedUp: result.backedUpCount,
+ totalSize: result.backedUpSize,
+ averageFileSize: result.backedUpCount > 0 ? Math.round(result.backedUpSize / result.backedUpCount) : 0,
+ manifestGenerated: !!manifestPath
+ })
+ });
+
+ logger.info(`Backup completed: ${result.backedUpCount} files, ${(result.backedUpSize / 1024 / 1024).toFixed(2)} MB in ${durationSeconds}s`);
+
+ // Send success email if configured
+ if (config.backup_email_on_success) {
+ // Get admin emails
+ const admins = await db('admin_users').where('is_active', formatBoolean(true));
+ for (const admin of admins) {
+ await queueEmail(null, admin.email, 'backup_completed', {
+ start_time: startTime.toISOString(),
+ duration: `${durationSeconds} seconds`,
+ files_count: result.backedUpCount.toString(),
+ total_size: `${(result.backedUpSize / 1024 / 1024).toFixed(2)} MB`,
+ backup_type: config.backup_destination_type
+ });
+ }
+ }
+
+ } catch (error) {
+ logger.error('Backup failed:', error);
+
+ // Update backup run record
+ if (backupRun) {
+ await db('backup_runs')
+ .where('id', backupRun.id)
+ .update({
+ completed_at: new Date(),
+ status: 'failed',
+ error_message: error.message
+ });
+ }
+
+ // Send failure email
+ const config = await getBackupConfig();
+ if (config && config.backup_email_on_failure) {
+ const admins = await db('admin_users').where('is_active', formatBoolean(true));
+ for (const admin of admins) {
+ await queueEmail(null, admin.email, 'backup_failed', {
+ start_time: startTime.toISOString(),
+ backup_type: config.backup_destination_type || 'unknown',
+ error_message: error.message
+ });
+ }
+ }
+ } finally {
+ isRunning = false;
+ }
+}
+
+/**
+ * Start backup service
+ */
+async function startBackupService() {
+ try {
+ // Get configuration
+ backupConfig = await getBackupConfig();
+
+ if (!backupConfig || !backupConfig.backup_enabled) {
+ logger.info('Backup service is disabled');
+ return;
+ }
+
+ // Cancel existing job if any
+ if (backupJob) {
+ backupJob.stop();
+ }
+
+ // Schedule backup job
+ const schedule = backupConfig.backup_schedule || '0 2 * * *'; // Default: 2 AM daily
+ backupJob = cron.schedule(schedule, async () => {
+ logger.info('Starting scheduled backup');
+ await runBackup();
+ });
+
+ logger.info(`Backup service started with schedule: ${schedule}`);
+ } catch (error) {
+ logger.error('Failed to start backup service:', error);
+ }
+}
+
+/**
+ * Stop backup service
+ */
+function stopBackupService() {
+ if (backupJob) {
+ backupJob.stop();
+ backupJob = null;
+ logger.info('Backup service stopped');
+ }
+}
+
+/**
+ * Trigger manual backup
+ */
+async function triggerManualBackup() {
+ logger.info('Starting manual backup');
+ await runBackup();
+}
+
+/**
+ * Get backup status and history
+ */
+async function getBackupStatus(limit = 10) {
+ try {
+ const runs = await db('backup_runs')
+ .orderBy('started_at', 'desc')
+ .limit(limit);
+
+ const lastRun = runs[0];
+ const isHealthy = lastRun && lastRun.status === 'completed';
+
+ // Validate manifest if exists
+ let manifestValid = false;
+ if (lastRun && lastRun.manifest_path) {
+ try {
+ const manifest = await backupManifest.loadManifest(lastRun.manifest_path);
+ backupManifest.validateManifest(manifest);
+ manifestValid = true;
+ } catch (error) {
+ logger.warn('Manifest validation failed:', error);
+ }
+ }
+
+ return {
+ isRunning,
+ isHealthy,
+ lastRun: lastRun ? {
+ ...lastRun,
+ manifestValid
+ } : null,
+ recentRuns: runs,
+ nextScheduledRun: backupJob ? getNextScheduledRun() : null
+ };
+ } catch (error) {
+ logger.error('Failed to get backup status:', error);
+ return {
+ isRunning,
+ isHealthy: false,
+ error: error.message
+ };
+ }
+}
+
+/**
+ * Get next scheduled run time
+ */
+function getNextScheduledRun() {
+ // This is a simplified version - would need proper cron parsing
+ const now = new Date();
+ const tomorrow = new Date(now);
+ tomorrow.setDate(tomorrow.getDate() + 1);
+ tomorrow.setHours(2, 0, 0, 0); // Assuming default 2 AM schedule
+ return tomorrow.toISOString();
+}
+
+/**
+ * Clean up old backup runs
+ */
+async function cleanupOldBackupRuns(retentionDays = 30) {
+ try {
+ const cutoffDate = new Date();
+ cutoffDate.setDate(cutoffDate.getDate() - retentionDays);
+
+ const deleted = await db('backup_runs')
+ .where('started_at', '<', cutoffDate)
+ .delete();
+
+ if (deleted > 0) {
+ logger.info(`Cleaned up ${deleted} old backup runs`);
+ }
+ } catch (error) {
+ logger.error('Failed to cleanup old backup runs:', error);
+ }
+}
+
+/**
+ * Get backup manifest for a specific backup run
+ */
+async function getBackupManifest(backupRunId) {
+ try {
+ const run = await db('backup_runs')
+ .where('id', backupRunId)
+ .first();
+
+ if (!run || !run.manifest_path) {
+ throw new Error('Backup manifest not found');
+ }
+
+ let manifest;
+
+ // Check if manifest is stored in S3
+ if (run.manifest_path.startsWith('s3://')) {
+ // Parse S3 path
+ const s3PathMatch = run.manifest_path.match(/^s3:\/\/([^\/]+)\/(.+)$/);
+ if (!s3PathMatch) {
+ throw new Error('Invalid S3 manifest path');
+ }
+
+ const [, bucket, key] = s3PathMatch;
+
+ // Get S3 configuration from backup settings
+ const config = await getBackupConfig();
+ if (!config.backup_s3_access_key || !config.backup_s3_secret_key) {
+ throw new Error('S3 credentials not configured for manifest retrieval');
+ }
+
+ // Initialize S3 client
+ const s3Config = {
+ bucket: bucket,
+ region: config.backup_s3_region || 'us-east-1',
+ endpoint: config.backup_s3_endpoint,
+ accessKeyId: config.backup_s3_access_key,
+ secretAccessKey: config.backup_s3_secret_key,
+ forcePathStyle: config.backup_s3_force_path_style || false,
+ sslEnabled: config.backup_s3_ssl_enabled !== false
+ };
+
+ const s3Client = new S3StorageAdapter(s3Config);
+
+ // Download manifest to temporary location
+ const tempDir = path.join(getStoragePath(), 'temp');
+ await fs.mkdir(tempDir, { recursive: true });
+
+ const tempManifestPath = path.join(tempDir, `manifest-${backupRunId}.json`);
+ await s3Client.download(key, tempManifestPath);
+
+ // Load manifest
+ manifest = await backupManifest.loadManifest(tempManifestPath);
+
+ // Clean up temp file
+ await fs.unlink(tempManifestPath).catch(() => {});
+
+ } else {
+ // Load manifest from local filesystem
+ manifest = await backupManifest.loadManifest(run.manifest_path);
+ }
+
+ return {
+ manifest,
+ summary: backupManifest.generateSummaryReport(manifest)
+ };
+ } catch (error) {
+ logger.error('Failed to get backup manifest:', error);
+ throw error;
+ }
+}
+
+/**
+ * Validate a backup manifest file
+ */
+async function validateBackupManifest(manifestPath) {
+ try {
+ let manifest;
+
+ // Check if manifest is stored in S3
+ if (manifestPath.startsWith('s3://')) {
+ // Parse S3 path
+ const s3PathMatch = manifestPath.match(/^s3:\/\/([^\/]+)\/(.+)$/);
+ if (!s3PathMatch) {
+ throw new Error('Invalid S3 manifest path');
+ }
+
+ const [, bucket, key] = s3PathMatch;
+
+ // Get S3 configuration from backup settings
+ const config = await getBackupConfig();
+ if (!config.backup_s3_access_key || !config.backup_s3_secret_key) {
+ throw new Error('S3 credentials not configured for manifest validation');
+ }
+
+ // Initialize S3 client
+ const s3Config = {
+ bucket: bucket,
+ region: config.backup_s3_region || 'us-east-1',
+ endpoint: config.backup_s3_endpoint,
+ accessKeyId: config.backup_s3_access_key,
+ secretAccessKey: config.backup_s3_secret_key,
+ forcePathStyle: config.backup_s3_force_path_style || false,
+ sslEnabled: config.backup_s3_ssl_enabled !== false
+ };
+
+ const s3Client = new S3StorageAdapter(s3Config);
+
+ // Download manifest to temporary location
+ const tempDir = path.join(getStoragePath(), 'temp');
+ await fs.mkdir(tempDir, { recursive: true });
+
+ const tempManifestPath = path.join(tempDir, `validate-manifest-${Date.now()}.json`);
+ await s3Client.download(key, tempManifestPath);
+
+ // Load manifest
+ manifest = await backupManifest.loadManifest(tempManifestPath);
+
+ // Clean up temp file
+ await fs.unlink(tempManifestPath).catch(() => {});
+
+ } else {
+ // Load manifest from local filesystem
+ manifest = await backupManifest.loadManifest(manifestPath);
+ }
+
+ // Validate the manifest
+ backupManifest.validateManifest(manifest);
+ return { valid: true, manifest };
+ } catch (error) {
+ return { valid: false, error: error.message };
+ }
+}
+
+module.exports = {
+ startBackupService,
+ stopBackupService,
+ triggerManualBackup,
+ getBackupStatus,
+ runBackup,
+ cleanupOldBackupRuns,
+ getBackupManifest,
+ validateBackupManifest
+};
\ No newline at end of file
diff --git a/backend/src/services/backupService.original.js b/backend/src/services/backupService.original.js
new file mode 100644
index 0000000..26b2b51
--- /dev/null
+++ b/backend/src/services/backupService.original.js
@@ -0,0 +1,720 @@
+const cron = require('node-cron');
+const path = require('path');
+const fs = require('fs').promises;
+const crypto = require('crypto');
+const { exec } = require('child_process');
+const { promisify } = require('util');
+const execAsync = promisify(exec);
+const { db } = require('../database/db');
+const { queueEmail } = require('./emailProcessor');
+const logger = require('../utils/logger');
+const { formatBoolean } = require('../utils/dbCompat');
+const backupManifest = require('./backupManifest');
+
+// Backup job reference
+let backupJob = null;
+let backupConfig = null;
+let isRunning = false;
+
+// Storage paths
+const getStoragePath = () => process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
+
+/**
+ * Calculate file checksum using SHA256
+ */
+async function calculateChecksum(filePath) {
+ const hash = crypto.createHash('sha256');
+ const stream = require('fs').createReadStream(filePath);
+
+ return new Promise((resolve, reject) => {
+ stream.on('data', data => hash.update(data));
+ stream.on('end', () => resolve(hash.digest('hex')));
+ stream.on('error', reject);
+ });
+}
+
+/**
+ * Get database backup information
+ */
+async function getDatabaseBackupInfo() {
+ try {
+ // Check for recent database backup
+ const recentDbBackup = await db('database_backup_runs')
+ .where('status', 'completed')
+ .orderBy('completed_at', 'desc')
+ .first();
+
+ if (recentDbBackup && recentDbBackup.file_path) {
+ return {
+ type: recentDbBackup.backup_type,
+ backupFile: recentDbBackup.file_path,
+ size: recentDbBackup.file_size_bytes,
+ checksum: recentDbBackup.checksum,
+ tables: recentDbBackup.statistics ? JSON.parse(recentDbBackup.statistics).tables : {},
+ rowCounts: recentDbBackup.table_checksums ? JSON.parse(recentDbBackup.table_checksums) : {}
+ };
+ }
+
+ return {
+ type: process.env.DB_TYPE === 'postgresql' ? 'postgresql' : 'sqlite',
+ backupFile: null,
+ size: 0,
+ checksum: null,
+ tables: {},
+ rowCounts: {}
+ };
+ } catch (error) {
+ logger.error('Failed to get database backup info:', error);
+ return {
+ type: 'unknown',
+ backupFile: null,
+ size: 0,
+ checksum: null,
+ tables: {},
+ rowCounts: {}
+ };
+ }
+}
+
+/**
+ * Get backup configuration from database
+ */
+async function getBackupConfig() {
+ try {
+ const settings = await db('app_settings')
+ .where('setting_type', 'backup')
+ .select('setting_key', 'setting_value');
+
+ const config = {};
+ settings.forEach(setting => {
+ try {
+ config[setting.setting_key] = JSON.parse(setting.setting_value);
+ } catch (e) {
+ config[setting.setting_key] = setting.setting_value;
+ }
+ });
+
+ return config;
+ } catch (error) {
+ logger.error('Failed to get backup configuration:', error);
+ return null;
+ }
+}
+
+/**
+ * Get list of files to backup
+ */
+async function getFilesToBackup(includeArchived = true) {
+ const files = [];
+ const storagePath = getStoragePath();
+
+ try {
+ // Active events
+ const activePath = path.join(storagePath, 'events/active');
+ await scanDirectory(activePath, files, storagePath);
+
+ // Archived events (if enabled)
+ if (includeArchived) {
+ const archivePath = path.join(storagePath, 'events/archived');
+ await scanDirectory(archivePath, files, storagePath);
+ }
+
+ // Thumbnails
+ const thumbsPath = path.join(storagePath, 'thumbnails');
+ await scanDirectory(thumbsPath, files, storagePath);
+
+ // Uploads (logos, favicons, etc.)
+ const uploadsPath = path.join(storagePath, 'uploads');
+ await scanDirectory(uploadsPath, files, storagePath);
+
+ return files;
+ } catch (error) {
+ logger.error('Failed to get files to backup:', error);
+ throw error;
+ }
+}
+
+/**
+ * Recursively scan directory for files
+ */
+async function scanDirectory(dirPath, fileList, basePath, excludePatterns = []) {
+ try {
+ const entries = await fs.readdir(dirPath, { withFileTypes: true });
+
+ for (const entry of entries) {
+ const fullPath = path.join(dirPath, entry.name);
+ const relativePath = path.relative(basePath, fullPath);
+
+ // Check exclude patterns
+ if (excludePatterns.some(pattern => {
+ if (pattern.includes('*')) {
+ return new RegExp(pattern.replace(/\*/g, '.*')).test(entry.name);
+ }
+ return entry.name === pattern;
+ })) {
+ continue;
+ }
+
+ if (entry.isDirectory()) {
+ await scanDirectory(fullPath, fileList, basePath, excludePatterns);
+ } else if (entry.isFile()) {
+ const stats = await fs.stat(fullPath);
+ fileList.push({
+ path: fullPath,
+ relativePath: relativePath,
+ size: stats.size,
+ modified: stats.mtime
+ });
+ }
+ }
+ } catch (error) {
+ if (error.code !== 'ENOENT') {
+ logger.error(`Failed to scan directory ${dirPath}:`, error);
+ }
+ }
+}
+
+/**
+ * Check if file has changed since last backup
+ */
+async function hasFileChanged(filePath, checksum) {
+ try {
+ const fileState = await db('backup_file_states')
+ .where('file_path', filePath)
+ .first();
+
+ return !fileState || fileState.checksum !== checksum;
+ } catch (error) {
+ logger.error('Failed to check file state:', error);
+ return true; // Assume changed if we can't check
+ }
+}
+
+/**
+ * Update file state in database
+ */
+async function updateFileState(filePath, checksum, size, modified) {
+ try {
+ const existing = await db('backup_file_states')
+ .where('file_path', filePath)
+ .first();
+
+ const data = {
+ file_path: filePath,
+ checksum: checksum,
+ size_bytes: size,
+ last_modified: modified,
+ last_backed_up: new Date()
+ };
+
+ if (existing) {
+ await db('backup_file_states')
+ .where('id', existing.id)
+ .update(data);
+ } else {
+ await db('backup_file_states').insert(data);
+ }
+ } catch (error) {
+ logger.error('Failed to update file state:', error);
+ }
+}
+
+/**
+ * Perform local directory backup
+ */
+async function performLocalBackup(config, files) {
+ const destPath = config.backup_destination_path;
+ const storagePath = getStoragePath();
+ let backedUpCount = 0;
+ let backedUpSize = 0;
+ const backedUpFiles = [];
+
+ // Ensure destination exists
+ await fs.mkdir(destPath, { recursive: true });
+
+ for (const file of files) {
+ try {
+ // Skip large files if configured
+ const maxSizeMB = config.backup_max_file_size_mb || 5000;
+ if (file.size > maxSizeMB * 1024 * 1024) {
+ logger.warn(`Skipping large file: ${file.relativePath} (${(file.size / 1024 / 1024).toFixed(2)} MB)`);
+ continue;
+ }
+
+ // Calculate checksum
+ const checksum = await calculateChecksum(file.path);
+ file.checksum = checksum; // Add checksum to file object
+
+ // Check if file has changed
+ const changed = await hasFileChanged(file.relativePath, checksum);
+ if (!changed) {
+ continue;
+ }
+
+ // Copy file
+ const destFilePath = path.join(destPath, file.relativePath);
+ const destDir = path.dirname(destFilePath);
+ await fs.mkdir(destDir, { recursive: true });
+ await fs.copyFile(file.path, destFilePath);
+
+ // Update state
+ await updateFileState(file.relativePath, checksum, file.size, file.modified);
+
+ backedUpCount++;
+ backedUpSize += file.size;
+ backedUpFiles.push(file.relativePath);
+ } catch (error) {
+ logger.error(`Failed to backup file ${file.relativePath}:`, error);
+ }
+ }
+
+ return { backedUpCount, backedUpSize, backedUpFiles };
+}
+
+/**
+ * Perform rsync backup
+ */
+async function performRsyncBackup(config, files) {
+ const storagePath = getStoragePath();
+ const host = config.backup_rsync_host;
+ const user = config.backup_rsync_user;
+ const remotePath = config.backup_rsync_path;
+ const sshKey = config.backup_rsync_ssh_key;
+
+ if (!host || !remotePath) {
+ throw new Error('Rsync configuration incomplete');
+ }
+
+ // Build rsync command
+ const rsyncOptions = [
+ '-avz', // archive, verbose, compress
+ '--delete', // remove deleted files
+ '--stats' // show statistics
+ ];
+
+ if (sshKey) {
+ rsyncOptions.push(`-e "ssh -i ${sshKey} -o StrictHostKeyChecking=no"`);
+ }
+
+ // Add exclude patterns
+ const excludePatterns = config.backup_exclude_patterns || [];
+ excludePatterns.forEach(pattern => {
+ rsyncOptions.push(`--exclude="${pattern}"`);
+ });
+
+ const source = `${storagePath}/`;
+ const destination = user ? `${user}@${host}:${remotePath}` : `${host}:${remotePath}`;
+
+ const rsyncCommand = `rsync ${rsyncOptions.join(' ')} "${source}" "${destination}"`;
+
+ try {
+ const { stdout, stderr } = await execAsync(rsyncCommand);
+
+ // Parse rsync stats
+ const stats = parseRsyncStats(stdout);
+
+ // Update file states for successfully synced files
+ for (const file of files) {
+ try {
+ const checksum = await calculateChecksum(file.path);
+ await updateFileState(file.relativePath, checksum, file.size, file.modified);
+ } catch (error) {
+ logger.error(`Failed to update state for ${file.relativePath}:`, error);
+ }
+ }
+
+ return {
+ backedUpCount: stats.filesTransferred || files.length,
+ backedUpSize: stats.totalSize || files.reduce((sum, f) => sum + f.size, 0),
+ backedUpFiles: files.map(f => f.relativePath)
+ };
+ } catch (error) {
+ logger.error('Rsync backup failed:', error);
+ throw new Error(`Rsync backup failed: ${error.message}`);
+ }
+}
+
+/**
+ * Parse rsync statistics from output
+ */
+function parseRsyncStats(output) {
+ const stats = {};
+
+ // Extract files transferred
+ const filesMatch = output.match(/Number of files transferred: (\d+)/);
+ if (filesMatch) {
+ stats.filesTransferred = parseInt(filesMatch[1]);
+ }
+
+ // Extract total size
+ const sizeMatch = output.match(/Total file size: ([\d,]+) bytes/);
+ if (sizeMatch) {
+ stats.totalSize = parseInt(sizeMatch[1].replace(/,/g, ''));
+ }
+
+ return stats;
+}
+
+/**
+ * Perform S3-compatible backup
+ */
+async function performS3Backup(config, files) {
+ // This would require AWS SDK or similar
+ // For now, return a placeholder
+ throw new Error('S3 backup not implemented yet');
+}
+
+/**
+ * Run backup process
+ */
+async function runBackup() {
+ if (isRunning) {
+ logger.warn('Backup already running, skipping');
+ return;
+ }
+
+ isRunning = true;
+ const startTime = new Date();
+ let backupRun = null;
+
+ try {
+ // Get current configuration
+ const config = await getBackupConfig();
+ if (!config.backup_enabled) {
+ logger.info('Backup is disabled, skipping');
+ return;
+ }
+
+ // Create backup run record
+ const [runId] = await db('backup_runs').insert({
+ started_at: startTime,
+ status: 'running',
+ backup_type: 'scheduled'
+ });
+
+ backupRun = { id: runId };
+
+ // Get files to backup
+ const files = await getFilesToBackup(config.backup_include_archived);
+ logger.info(`Found ${files.length} files to check for backup`);
+
+ // Perform backup based on destination type
+ let result;
+ switch (config.backup_destination_type) {
+ case 'local':
+ result = await performLocalBackup(config, files);
+ break;
+ case 'rsync':
+ result = await performRsyncBackup(config, files);
+ break;
+ case 's3':
+ result = await performS3Backup(config, files);
+ break;
+ default:
+ throw new Error(`Unknown backup destination type: ${config.backup_destination_type}`);
+ }
+
+ // Calculate duration
+ const endTime = new Date();
+ const durationSeconds = Math.round((endTime - startTime) / 1000);
+
+ // Generate backup manifest
+ let manifestPath = null;
+ try {
+ logger.info('Generating backup manifest...');
+
+ // Get database backup info if available
+ const databaseInfo = await getDatabaseBackupInfo();
+
+ // Determine if this is an incremental backup
+ const lastSuccessfulBackup = await db('backup_runs')
+ .where('status', 'completed')
+ .whereNot('id', runId)
+ .orderBy('completed_at', 'desc')
+ .first();
+
+ let manifest;
+ const manifestOptions = {
+ backupType: lastSuccessfulBackup ? 'incremental' : 'full',
+ backupPath: config.backup_destination_path || config.backup_destination_type,
+ files: files.filter(f => result.backedUpFiles && result.backedUpFiles.includes(f.relativePath)),
+ databaseInfo: databaseInfo,
+ parentBackupId: lastSuccessfulBackup ? lastSuccessfulBackup.manifest_id : null,
+ format: config.backup_manifest_format || 'json',
+ customMetadata: {
+ backup_run_id: runId,
+ destination_type: config.backup_destination_type,
+ operator: 'system',
+ reason: 'scheduled',
+ retentionDays: config.backup_retention_days || 30
+ }
+ };
+
+ if (lastSuccessfulBackup && lastSuccessfulBackup.manifest_path) {
+ try {
+ const parentManifest = await backupManifest.loadManifest(lastSuccessfulBackup.manifest_path);
+ manifest = await backupManifest.generateIncrementalManifest(manifestOptions, parentManifest);
+ } catch (error) {
+ logger.warn('Failed to load parent manifest, generating full manifest:', error);
+ manifest = await backupManifest.generateManifest(manifestOptions);
+ }
+ } else {
+ manifest = await backupManifest.generateManifest(manifestOptions);
+ }
+
+ // Save manifest
+ const manifestDir = config.backup_manifest_path || path.join(config.backup_destination_path || '/backup', 'manifests');
+ await fs.mkdir(manifestDir, { recursive: true });
+
+ const manifestFileName = `backup-manifest-${manifest.backup.id}.${config.backup_manifest_format || 'json'}`;
+ manifestPath = path.join(manifestDir, manifestFileName);
+ await backupManifest.saveManifest(manifest, manifestPath, config.backup_manifest_format || 'json');
+
+ logger.info(`Backup manifest saved to ${manifestPath}`);
+
+ } catch (error) {
+ logger.error('Failed to generate backup manifest:', error);
+ // Don't fail the entire backup for manifest generation failure
+ }
+
+ // Update backup run record
+ await db('backup_runs')
+ .where('id', runId)
+ .update({
+ completed_at: endTime,
+ status: 'completed',
+ files_backed_up: result.backedUpCount,
+ total_size_bytes: result.backedUpSize,
+ duration_seconds: durationSeconds,
+ manifest_path: manifestPath,
+ manifest_id: manifestPath ? path.basename(manifestPath, path.extname(manifestPath)) : null,
+ statistics: JSON.stringify({
+ totalFilesChecked: files.length,
+ filesBackedUp: result.backedUpCount,
+ totalSize: result.backedUpSize,
+ averageFileSize: result.backedUpCount > 0 ? Math.round(result.backedUpSize / result.backedUpCount) : 0,
+ manifestGenerated: !!manifestPath
+ })
+ });
+
+ logger.info(`Backup completed: ${result.backedUpCount} files, ${(result.backedUpSize / 1024 / 1024).toFixed(2)} MB in ${durationSeconds}s`);
+
+ // Send success email if configured
+ if (config.backup_email_on_success) {
+ // Get admin emails
+ const admins = await db('admin_users').where('is_active', formatBoolean(true));
+ for (const admin of admins) {
+ await queueEmail(null, admin.email, 'backup_completed', {
+ start_time: startTime.toISOString(),
+ duration: `${durationSeconds} seconds`,
+ files_count: result.backedUpCount.toString(),
+ total_size: `${(result.backedUpSize / 1024 / 1024).toFixed(2)} MB`,
+ backup_type: config.backup_destination_type
+ });
+ }
+ }
+
+ } catch (error) {
+ logger.error('Backup failed:', error);
+
+ // Update backup run record
+ if (backupRun) {
+ await db('backup_runs')
+ .where('id', backupRun.id)
+ .update({
+ completed_at: new Date(),
+ status: 'failed',
+ error_message: error.message
+ });
+ }
+
+ // Send failure email
+ const config = await getBackupConfig();
+ if (config && config.backup_email_on_failure) {
+ const admins = await db('admin_users').where('is_active', formatBoolean(true));
+ for (const admin of admins) {
+ await queueEmail(null, admin.email, 'backup_failed', {
+ start_time: startTime.toISOString(),
+ backup_type: config.backup_destination_type || 'unknown',
+ error_message: error.message
+ });
+ }
+ }
+ } finally {
+ isRunning = false;
+ }
+}
+
+/**
+ * Start backup service
+ */
+async function startBackupService() {
+ try {
+ // Get configuration
+ backupConfig = await getBackupConfig();
+
+ if (!backupConfig || !backupConfig.backup_enabled) {
+ logger.info('Backup service is disabled');
+ return;
+ }
+
+ // Cancel existing job if any
+ if (backupJob) {
+ backupJob.stop();
+ }
+
+ // Schedule backup job
+ const schedule = backupConfig.backup_schedule || '0 2 * * *'; // Default: 2 AM daily
+ backupJob = cron.schedule(schedule, async () => {
+ logger.info('Starting scheduled backup');
+ await runBackup();
+ });
+
+ logger.info(`Backup service started with schedule: ${schedule}`);
+ } catch (error) {
+ logger.error('Failed to start backup service:', error);
+ }
+}
+
+/**
+ * Stop backup service
+ */
+function stopBackupService() {
+ if (backupJob) {
+ backupJob.stop();
+ backupJob = null;
+ logger.info('Backup service stopped');
+ }
+}
+
+/**
+ * Trigger manual backup
+ */
+async function triggerManualBackup() {
+ logger.info('Starting manual backup');
+ await runBackup();
+}
+
+/**
+ * Get backup status and history
+ */
+async function getBackupStatus(limit = 10) {
+ try {
+ const runs = await db('backup_runs')
+ .orderBy('started_at', 'desc')
+ .limit(limit);
+
+ const lastRun = runs[0];
+ const isHealthy = lastRun && lastRun.status === 'completed';
+
+ // Validate manifest if exists
+ let manifestValid = false;
+ if (lastRun && lastRun.manifest_path) {
+ try {
+ const manifest = await backupManifest.loadManifest(lastRun.manifest_path);
+ backupManifest.validateManifest(manifest);
+ manifestValid = true;
+ } catch (error) {
+ logger.warn('Manifest validation failed:', error);
+ }
+ }
+
+ return {
+ isRunning,
+ isHealthy,
+ lastRun: lastRun ? {
+ ...lastRun,
+ manifestValid
+ } : null,
+ recentRuns: runs,
+ nextScheduledRun: backupJob ? getNextScheduledRun() : null
+ };
+ } catch (error) {
+ logger.error('Failed to get backup status:', error);
+ return {
+ isRunning,
+ isHealthy: false,
+ error: error.message
+ };
+ }
+}
+
+/**
+ * Get next scheduled run time
+ */
+function getNextScheduledRun() {
+ // This is a simplified version - would need proper cron parsing
+ const now = new Date();
+ const tomorrow = new Date(now);
+ tomorrow.setDate(tomorrow.getDate() + 1);
+ tomorrow.setHours(2, 0, 0, 0); // Assuming default 2 AM schedule
+ return tomorrow.toISOString();
+}
+
+/**
+ * Clean up old backup runs
+ */
+async function cleanupOldBackupRuns(retentionDays = 30) {
+ try {
+ const cutoffDate = new Date();
+ cutoffDate.setDate(cutoffDate.getDate() - retentionDays);
+
+ const deleted = await db('backup_runs')
+ .where('started_at', '<', cutoffDate)
+ .delete();
+
+ if (deleted > 0) {
+ logger.info(`Cleaned up ${deleted} old backup runs`);
+ }
+ } catch (error) {
+ logger.error('Failed to cleanup old backup runs:', error);
+ }
+}
+
+/**
+ * Get backup manifest for a specific backup run
+ */
+async function getBackupManifest(backupRunId) {
+ try {
+ const run = await db('backup_runs')
+ .where('id', backupRunId)
+ .first();
+
+ if (!run || !run.manifest_path) {
+ throw new Error('Backup manifest not found');
+ }
+
+ const manifest = await backupManifest.loadManifest(run.manifest_path);
+ return {
+ manifest,
+ summary: backupManifest.generateSummaryReport(manifest)
+ };
+ } catch (error) {
+ logger.error('Failed to get backup manifest:', error);
+ throw error;
+ }
+}
+
+/**
+ * Validate a backup manifest file
+ */
+async function validateBackupManifest(manifestPath) {
+ try {
+ const manifest = await backupManifest.loadManifest(manifestPath);
+ backupManifest.validateManifest(manifest);
+ return { valid: true, manifest };
+ } catch (error) {
+ return { valid: false, error: error.message };
+ }
+}
+
+module.exports = {
+ startBackupService,
+ stopBackupService,
+ triggerManualBackup,
+ getBackupStatus,
+ runBackup,
+ cleanupOldBackupRuns,
+ getBackupManifest,
+ validateBackupManifest
+};
\ No newline at end of file
diff --git a/backend/src/services/databaseBackup.example.js b/backend/src/services/databaseBackup.example.js
new file mode 100644
index 0000000..ed0a7af
--- /dev/null
+++ b/backend/src/services/databaseBackup.example.js
@@ -0,0 +1,147 @@
+/**
+ * Database Backup Service Usage Examples
+ *
+ * This service provides comprehensive database backup functionality
+ * with support for both SQLite and PostgreSQL databases.
+ */
+
+const { databaseBackupService } = require('./databaseBackup');
+
+// Example 1: Manual backup with default settings
+async function manualBackup() {
+ try {
+ const result = await databaseBackupService.backup();
+ console.log('Backup completed:', result);
+ // Result includes: path, size, duration, checksum, compressionRatio
+ } catch (error) {
+ console.error('Backup failed:', error);
+ }
+}
+
+// Example 2: Backup with custom options
+async function customBackup() {
+ try {
+ const result = await databaseBackupService.backup({
+ destinationPath: '/custom/backup/path',
+ compress: true, // Enable gzip compression
+ validateIntegrity: true, // Validate backup after creation
+ includeChecksums: true, // Calculate table checksums
+ noTransaction: false // Use transaction for consistency (PostgreSQL)
+ });
+ console.log('Custom backup completed:', result);
+ } catch (error) {
+ console.error('Backup failed:', error);
+ }
+}
+
+// Example 3: Check backup progress (useful for long-running backups)
+async function backupWithProgress() {
+ // Start backup asynchronously
+ const backupPromise = databaseBackupService.backup();
+
+ // Poll for progress
+ const progressInterval = setInterval(() => {
+ const progress = databaseBackupService.getProgress();
+ if (progress) {
+ console.log(`Progress: ${progress.message}`, progress.details);
+ }
+ }, 1000);
+
+ try {
+ const result = await backupPromise;
+ clearInterval(progressInterval);
+ console.log('Backup completed:', result);
+ } catch (error) {
+ clearInterval(progressInterval);
+ console.error('Backup failed:', error);
+ }
+}
+
+// Example 4: Get backup history
+async function getBackupHistory() {
+ const history = await databaseBackupService.getBackupHistory(10);
+
+ history.forEach(backup => {
+ console.log(`Backup ${backup.id}:`);
+ console.log(` Started: ${backup.started_at}`);
+ console.log(` Status: ${backup.status}`);
+ console.log(` Size: ${(backup.file_size_bytes / 1024 / 1024).toFixed(2)} MB`);
+ console.log(` Duration: ${backup.duration_seconds}s`);
+ });
+}
+
+// Example 5: Clean up old backups
+async function cleanupBackups() {
+ // Delete backups older than 30 days
+ await databaseBackupService.cleanupOldBackups(30);
+ console.log('Old backups cleaned up');
+}
+
+// Example 6: Get table checksums (useful for monitoring changes)
+async function getTableChecksums() {
+ const checksums = await databaseBackupService.getTableChecksums();
+
+ console.log('Table Checksums:');
+ Object.entries(checksums).forEach(([table, info]) => {
+ console.log(` ${table}: ${info.rowCount} rows, checksum: ${info.checksum}`);
+ });
+}
+
+// Example 7: Using the scheduled backup service
+const { startScheduledBackups, stopScheduledBackups } = require('./databaseBackup');
+
+async function setupScheduledBackups() {
+ // Start scheduled backups (reads schedule from database config)
+ await startScheduledBackups();
+ console.log('Scheduled backups started');
+
+ // Later, if needed, stop scheduled backups
+ // stopScheduledBackups();
+}
+
+// Example 8: Admin API endpoints available
+/*
+ GET /api/admin/database-backup/status - Get backup status and config
+ PUT /api/admin/database-backup/config - Update backup configuration
+ POST /api/admin/database-backup/backup - Trigger manual backup
+ GET /api/admin/database-backup/progress - Get current backup progress
+ GET /api/admin/database-backup/history - Get backup history with pagination
+ DELETE /api/admin/database-backup/cleanup - Delete old backup files
+ POST /api/admin/database-backup/test - Test backup configuration
+ GET /api/admin/database-backup/checksums - Get current table checksums
+*/
+
+// Example 9: Configuration options stored in database
+/*
+ database_backup_enabled: boolean - Enable/disable scheduled backups
+ database_backup_schedule: string - Cron schedule (default: '0 3 * * *')
+ database_backup_destination_path: string - Where to store backups
+ database_backup_compress: boolean - Enable gzip compression
+ database_backup_validate_integrity: boolean - Validate after backup
+ database_backup_include_checksums: boolean - Calculate table checksums
+ database_backup_retention_days: number - Days to keep old backups
+ database_backup_email_on_failure: boolean - Send email on failure
+ database_backup_email_on_success: boolean - Send email on success
+*/
+
+// Example 10: Production considerations
+/*
+ 1. Ensure destination path has sufficient space
+ 2. For large databases, backups may take significant time
+ 3. PostgreSQL backups use single-transaction mode by default
+ 4. Compression typically reduces size by 70-90%
+ 5. Schedule backups during low-traffic periods
+ 6. Monitor backup history for failures
+ 7. Test restore procedures regularly
+ 8. Consider replication for real-time redundancy
+*/
+
+module.exports = {
+ manualBackup,
+ customBackup,
+ backupWithProgress,
+ getBackupHistory,
+ cleanupBackups,
+ getTableChecksums,
+ setupScheduledBackups
+};
\ No newline at end of file
diff --git a/backend/src/services/databaseBackup.js b/backend/src/services/databaseBackup.js
new file mode 100644
index 0000000..02e5ff2
--- /dev/null
+++ b/backend/src/services/databaseBackup.js
@@ -0,0 +1,631 @@
+const fs = require('fs').promises;
+const path = require('path');
+const { exec } = require('child_process');
+const { promisify } = require('util');
+const execAsync = promisify(exec);
+const crypto = require('crypto');
+const zlib = require('zlib');
+const { pipeline } = require('stream/promises');
+const { createReadStream, createWriteStream } = require('fs');
+const { db } = require('../database/db');
+const knexConfig = require('../../knexfile');
+const logger = require('../utils/logger');
+const { queueEmail } = require('./emailProcessor');
+const { formatBoolean } = require('../utils/dbCompat');
+
+// Constants
+const CHUNK_SIZE = 1024 * 1024; // 1MB chunks for streaming
+const PROGRESS_INTERVAL = 100; // Report progress every 100 rows
+
+/**
+ * Database Backup Service
+ * Supports both SQLite and PostgreSQL with proper escaping,
+ * compression, checksums, and validation
+ */
+class DatabaseBackupService {
+ constructor() {
+ this.isRunning = false;
+ this.currentProgress = null;
+ this.dbType = knexConfig.client === 'pg' ? 'postgresql' : 'sqlite';
+ }
+
+ /**
+ * Calculate checksum for a file
+ */
+ async calculateChecksum(filePath) {
+ const hash = crypto.createHash('sha256');
+ const stream = createReadStream(filePath);
+
+ return new Promise((resolve, reject) => {
+ stream.on('data', data => hash.update(data));
+ stream.on('end', () => resolve(hash.digest('hex')));
+ stream.on('error', reject);
+ });
+ }
+
+ /**
+ * Compress a file using gzip
+ */
+ async compressFile(inputPath, outputPath) {
+ const gzip = zlib.createGzip({ level: 6 }); // Balanced compression
+ const source = createReadStream(inputPath);
+ const destination = createWriteStream(outputPath);
+
+ await pipeline(source, gzip, destination);
+
+ // Get compression ratio
+ const inputStats = await fs.stat(inputPath);
+ const outputStats = await fs.stat(outputPath);
+ const ratio = (1 - outputStats.size / inputStats.size) * 100;
+
+ return {
+ originalSize: inputStats.size,
+ compressedSize: outputStats.size,
+ compressionRatio: ratio.toFixed(2)
+ };
+ }
+
+ /**
+ * Get database size
+ */
+ async getDatabaseSize() {
+ if (this.dbType === 'sqlite') {
+ const dbPath = knexConfig.connection.filename;
+ const stats = await fs.stat(dbPath);
+ return stats.size;
+ } else {
+ // PostgreSQL
+ const result = await db.raw(`
+ SELECT pg_database_size(current_database()) as size
+ `);
+ return parseInt(result.rows[0].size);
+ }
+ }
+
+ /**
+ * Get table checksums for change detection
+ */
+ async getTableChecksums() {
+ const checksums = {};
+ const tables = await this.getTables();
+
+ for (const table of tables) {
+ if (this.dbType === 'sqlite') {
+ // SQLite: Use aggregate of all row data
+ const result = await db.raw(`
+ SELECT
+ COUNT(*) as row_count,
+ COALESCE(SUM(LENGTH(CAST(t.* AS TEXT))), 0) as data_sum
+ FROM "${table}" t
+ `);
+
+ checksums[table] = {
+ rowCount: result[0].row_count,
+ checksum: crypto
+ .createHash('md5')
+ .update(`${result[0].row_count}-${result[0].data_sum}`)
+ .digest('hex')
+ };
+ } else {
+ // PostgreSQL: Use built-in functions
+ const result = await db.raw(`
+ SELECT
+ COUNT(*) as row_count,
+ MD5(COALESCE(STRING_AGG(MD5(t::text), ''), '')) as checksum
+ FROM "${table}" t
+ `);
+
+ checksums[table] = {
+ rowCount: parseInt(result.rows[0].row_count),
+ checksum: result.rows[0].checksum || 'empty'
+ };
+ }
+ }
+
+ return checksums;
+ }
+
+ /**
+ * Get list of tables
+ */
+ async getTables() {
+ if (this.dbType === 'sqlite') {
+ const result = await db.raw(`
+ SELECT name FROM sqlite_master
+ WHERE type='table'
+ AND name NOT LIKE 'sqlite_%'
+ AND name != 'knex_migrations'
+ AND name != 'knex_migrations_lock'
+ ORDER BY name
+ `);
+ return result.map(row => row.name);
+ } else {
+ // PostgreSQL
+ const result = await db.raw(`
+ SELECT table_name
+ FROM information_schema.tables
+ WHERE table_schema = 'public'
+ AND table_type = 'BASE TABLE'
+ AND table_name NOT IN ('knex_migrations', 'knex_migrations_lock')
+ ORDER BY table_name
+ `);
+ return result.rows.map(row => row.table_name);
+ }
+ }
+
+ /**
+ * Create SQLite backup
+ */
+ async createSQLiteBackup(outputPath, options = {}) {
+ const dbPath = knexConfig.connection.filename;
+ const tempPath = `${outputPath}.tmp`;
+
+ try {
+ // Use SQLite's backup API for consistency
+ await execAsync(`sqlite3 "${dbPath}" ".backup '${tempPath}'"`);
+
+ // Verify the backup
+ const verifyResult = await execAsync(`sqlite3 "${tempPath}" "PRAGMA integrity_check"`);
+ if (!verifyResult.stdout.includes('ok')) {
+ throw new Error('Backup integrity check failed');
+ }
+
+ // Move temp file to final location
+ await fs.rename(tempPath, outputPath);
+
+ return { success: true };
+ } catch (error) {
+ // Cleanup temp file if exists
+ try {
+ await fs.unlink(tempPath);
+ } catch (e) {
+ // Ignore
+ }
+ throw error;
+ }
+ }
+
+ /**
+ * Create PostgreSQL backup with proper escaping
+ */
+ async createPostgreSQLBackup(outputPath, options = {}) {
+ const { host, port, user, password, database } = knexConfig.connection;
+
+ // Build connection string with proper escaping
+ const connectionParts = [
+ `host=${host}`,
+ `port=${port}`,
+ `dbname=${database}`,
+ `user=${user}`
+ ];
+
+ // Set PGPASSWORD environment variable for security
+ const env = { ...process.env };
+ if (password) {
+ env.PGPASSWORD = password;
+ }
+
+ // Build pg_dump command with options
+ const pgDumpOptions = [
+ '--verbose',
+ '--no-owner',
+ '--no-privileges',
+ '--clean',
+ '--if-exists',
+ '--format=plain',
+ '--encoding=UTF8'
+ ];
+
+ // Add transaction support for consistency
+ if (!options.noTransaction) {
+ pgDumpOptions.push('--single-transaction');
+ }
+
+ // Add compression if not doing it separately
+ if (options.compress && !options.separateCompression) {
+ pgDumpOptions.push('--compress=6');
+ }
+
+ const command = `pg_dump "${connectionParts.join(' ')}" ${pgDumpOptions.join(' ')} > "${outputPath}"`;
+
+ try {
+ const { stderr } = await execAsync(command, {
+ env,
+ maxBuffer: 1024 * 1024 * 100 // 100MB buffer
+ });
+
+ // pg_dump writes progress to stderr, not an error
+ if (stderr && !stderr.includes('dump complete')) {
+ logger.warn('pg_dump warnings:', stderr);
+ }
+
+ // Verify the dump file is not empty
+ const stats = await fs.stat(outputPath);
+ if (stats.size === 0) {
+ throw new Error('Backup file is empty');
+ }
+
+ return { success: true, warnings: stderr };
+ } catch (error) {
+ throw new Error(`PostgreSQL backup failed: ${error.message}`);
+ }
+ }
+
+ /**
+ * Validate backup integrity
+ */
+ async validateBackup(backupPath, originalChecksums) {
+ const tempDbPath = `${backupPath}.validate`;
+
+ try {
+ if (this.dbType === 'sqlite') {
+ // For SQLite, we can directly check integrity
+ const result = await execAsync(`sqlite3 "${backupPath}" "PRAGMA integrity_check"`);
+ if (!result.stdout.includes('ok')) {
+ throw new Error('Backup integrity check failed');
+ }
+ } else {
+ // For PostgreSQL, we'd need to restore to a temp database
+ // This is more complex and might not be feasible in production
+ logger.info('PostgreSQL backup validation would require restore test');
+ }
+
+ return { valid: true };
+ } finally {
+ // Cleanup
+ try {
+ await fs.unlink(tempDbPath);
+ } catch (e) {
+ // Ignore
+ }
+ }
+ }
+
+ /**
+ * Main backup method
+ */
+ async backup(options = {}) {
+ if (this.isRunning) {
+ throw new Error('Backup already in progress');
+ }
+
+ this.isRunning = true;
+ const startTime = new Date();
+ let backupRun = null;
+
+ try {
+ // Get configuration
+ const config = await this.getBackupConfig();
+ const {
+ destinationPath = '/backup/database',
+ compress = true,
+ validateIntegrity = true,
+ includeChecksums = true
+ } = { ...config, ...options };
+
+ // Create backup directory
+ await fs.mkdir(destinationPath, { recursive: true });
+
+ // Generate backup filename
+ const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
+ const baseName = `picpeak-db-${this.dbType}-${timestamp}`;
+ const sqlFile = path.join(destinationPath, `${baseName}.sql`);
+ const finalFile = compress ? path.join(destinationPath, `${baseName}.sql.gz`) : sqlFile;
+
+ // Create backup run record
+ const [runId] = await db('database_backup_runs').insert({
+ started_at: startTime,
+ status: 'running',
+ backup_type: this.dbType,
+ destination_path: finalFile
+ });
+
+ backupRun = { id: runId };
+
+ // Get initial checksums
+ let tableChecksums = null;
+ if (includeChecksums) {
+ this.updateProgress('Calculating table checksums...');
+ tableChecksums = await this.getTableChecksums();
+ }
+
+ // Get database size
+ const dbSize = await this.getDatabaseSize();
+
+ // Create the backup
+ this.updateProgress('Creating database backup...');
+ if (this.dbType === 'sqlite') {
+ await this.createSQLiteBackup(sqlFile, options);
+ } else {
+ await this.createPostgreSQLBackup(sqlFile, options);
+ }
+
+ // Compress if requested
+ let compressionStats = null;
+ if (compress) {
+ this.updateProgress('Compressing backup...');
+ compressionStats = await this.compressFile(sqlFile, finalFile);
+ await fs.unlink(sqlFile); // Remove uncompressed file
+ }
+
+ // Calculate checksum
+ this.updateProgress('Calculating backup checksum...');
+ const backupChecksum = await this.calculateChecksum(finalFile);
+
+ // Validate if requested
+ if (validateIntegrity && !compress) {
+ this.updateProgress('Validating backup integrity...');
+ await this.validateBackup(finalFile, tableChecksums);
+ }
+
+ // Get final file size
+ const finalStats = await fs.stat(finalFile);
+
+ // Calculate duration
+ const endTime = new Date();
+ const durationSeconds = Math.round((endTime - startTime) / 1000);
+
+ // Update backup run record
+ await db('database_backup_runs')
+ .where('id', runId)
+ .update({
+ completed_at: endTime,
+ status: 'completed',
+ file_path: finalFile,
+ file_size_bytes: finalStats.size,
+ original_size_bytes: dbSize,
+ duration_seconds: durationSeconds,
+ checksum: backupChecksum,
+ compression_ratio: compressionStats?.compressionRatio || null,
+ table_checksums: tableChecksums ? JSON.stringify(tableChecksums) : null,
+ statistics: JSON.stringify({
+ dbType: this.dbType,
+ compressed: compress,
+ validated: validateIntegrity,
+ compressionStats,
+ tableCount: tableChecksums ? Object.keys(tableChecksums).length : null
+ })
+ });
+
+ logger.info(`Database backup completed: ${finalFile} (${(finalStats.size / 1024 / 1024).toFixed(2)} MB) in ${durationSeconds}s`);
+
+ // Send success notification if configured
+ if (config.emailOnSuccess) {
+ await this.sendBackupNotification('success', {
+ duration: durationSeconds,
+ size: finalStats.size,
+ compressionRatio: compressionStats?.compressionRatio,
+ path: finalFile
+ });
+ }
+
+ return {
+ success: true,
+ path: finalFile,
+ size: finalStats.size,
+ duration: durationSeconds,
+ checksum: backupChecksum,
+ compressionRatio: compressionStats?.compressionRatio
+ };
+
+ } catch (error) {
+ logger.error('Database backup failed:', error);
+
+ // Update backup run record
+ if (backupRun) {
+ await db('database_backup_runs')
+ .where('id', backupRun.id)
+ .update({
+ completed_at: new Date(),
+ status: 'failed',
+ error_message: error.message
+ });
+ }
+
+ // Send failure notification
+ const config = await this.getBackupConfig();
+ if (config.emailOnFailure) {
+ await this.sendBackupNotification('failure', {
+ error: error.message
+ });
+ }
+
+ throw error;
+ } finally {
+ this.isRunning = false;
+ this.currentProgress = null;
+ }
+ }
+
+ /**
+ * Update progress
+ */
+ updateProgress(message, details = {}) {
+ this.currentProgress = {
+ message,
+ details,
+ timestamp: new Date()
+ };
+ logger.info(`Backup progress: ${message}`, details);
+ }
+
+ /**
+ * Get current progress
+ */
+ getProgress() {
+ return this.currentProgress;
+ }
+
+ /**
+ * Get backup configuration
+ */
+ async getBackupConfig() {
+ const settings = await db('app_settings')
+ .where('setting_type', 'database_backup')
+ .select('setting_key', 'setting_value');
+
+ const config = {};
+ settings.forEach(setting => {
+ try {
+ config[setting.setting_key] = JSON.parse(setting.setting_value);
+ } catch (e) {
+ config[setting.setting_key] = setting.setting_value;
+ }
+ });
+
+ return config;
+ }
+
+ /**
+ * Send backup notification email
+ */
+ async sendBackupNotification(type, details) {
+ const admins = await db('admin_users').where('is_active', formatBoolean(true));
+
+ for (const admin of admins) {
+ if (type === 'success') {
+ await queueEmail(null, admin.email, 'database_backup_completed', {
+ backup_type: this.dbType,
+ duration: `${details.duration} seconds`,
+ file_size: `${(details.size / 1024 / 1024).toFixed(2)} MB`,
+ compression_ratio: details.compressionRatio ? `${details.compressionRatio}%` : 'N/A',
+ file_path: details.path
+ });
+ } else {
+ await queueEmail(null, admin.email, 'database_backup_failed', {
+ backup_type: this.dbType,
+ error_message: details.error,
+ timestamp: new Date().toISOString()
+ });
+ }
+ }
+ }
+
+ /**
+ * Clean up old backups
+ */
+ async cleanupOldBackups(retentionDays = 30) {
+ try {
+ const cutoffDate = new Date();
+ cutoffDate.setDate(cutoffDate.getDate() - retentionDays);
+
+ // Get old backup records
+ const oldBackups = await db('database_backup_runs')
+ .where('completed_at', '<', cutoffDate)
+ .where('status', 'completed')
+ .select('id', 'file_path');
+
+ let deletedCount = 0;
+
+ for (const backup of oldBackups) {
+ try {
+ // Delete the file
+ if (backup.file_path) {
+ await fs.unlink(backup.file_path);
+ }
+
+ // Delete the record
+ await db('database_backup_runs')
+ .where('id', backup.id)
+ .delete();
+
+ deletedCount++;
+ } catch (error) {
+ logger.error(`Failed to delete old backup ${backup.file_path}:`, error);
+ }
+ }
+
+ if (deletedCount > 0) {
+ logger.info(`Cleaned up ${deletedCount} old database backups`);
+ }
+
+ // Also clean up old failed runs
+ await db('database_backup_runs')
+ .where('started_at', '<', cutoffDate)
+ .where('status', 'failed')
+ .delete();
+
+ } catch (error) {
+ logger.error('Failed to cleanup old database backups:', error);
+ }
+ }
+
+ /**
+ * Get backup history
+ */
+ async getBackupHistory(limit = 10) {
+ return await db('database_backup_runs')
+ .orderBy('started_at', 'desc')
+ .limit(limit);
+ }
+
+ /**
+ * Restore from backup (careful!)
+ */
+ async restore(backupPath, options = {}) {
+ // This is a dangerous operation and should be used with extreme caution
+ throw new Error('Restore functionality not implemented for safety. Please restore manually.');
+ }
+}
+
+// Create singleton instance
+const databaseBackupService = new DatabaseBackupService();
+
+// Scheduled backup runner
+let backupSchedule = null;
+
+/**
+ * Start scheduled database backups
+ */
+async function startScheduledBackups() {
+ const cron = require('node-cron');
+
+ try {
+ const config = await databaseBackupService.getBackupConfig();
+
+ if (!config.enabled) {
+ logger.info('Database backup service is disabled');
+ return;
+ }
+
+ // Stop existing schedule
+ if (backupSchedule) {
+ backupSchedule.stop();
+ }
+
+ // Default schedule: 3 AM daily (offset from file backups at 2 AM)
+ const schedule = config.schedule || '0 3 * * *';
+
+ backupSchedule = cron.schedule(schedule, async () => {
+ logger.info('Starting scheduled database backup');
+ try {
+ await databaseBackupService.backup();
+ await databaseBackupService.cleanupOldBackups(config.retentionDays || 30);
+ } catch (error) {
+ logger.error('Scheduled database backup failed:', error);
+ }
+ });
+
+ logger.info(`Database backup service started with schedule: ${schedule}`);
+ } catch (error) {
+ logger.error('Failed to start database backup service:', error);
+ }
+}
+
+/**
+ * Stop scheduled database backups
+ */
+function stopScheduledBackups() {
+ if (backupSchedule) {
+ backupSchedule.stop();
+ backupSchedule = null;
+ logger.info('Database backup service stopped');
+ }
+}
+
+module.exports = {
+ databaseBackupService,
+ startScheduledBackups,
+ stopScheduledBackups,
+ DatabaseBackupService // Export class for testing
+};
\ No newline at end of file
diff --git a/backend/src/services/restoreService.js b/backend/src/services/restoreService.js
new file mode 100644
index 0000000..05967f3
--- /dev/null
+++ b/backend/src/services/restoreService.js
@@ -0,0 +1,1221 @@
+const fs = require('fs').promises;
+const path = require('path');
+const crypto = require('crypto');
+const zlib = require('zlib');
+const { pipeline } = require('stream/promises');
+const { createReadStream, createWriteStream } = require('fs');
+const { exec } = require('child_process');
+const { promisify } = require('util');
+const execAsync = promisify(exec);
+const { db } = require('../database/db');
+const knexConfig = require('../../knexfile');
+const logger = require('../utils/logger');
+const backupManifest = require('./backupManifest');
+const S3StorageAdapter = require('./storage/s3Storage');
+const { queueEmail } = require('./emailProcessor');
+const { formatBoolean } = require('../utils/dbCompat');
+const os = require('os');
+
+/**
+ * Restore Service with Extreme Safety Measures
+ *
+ * This service handles restoration of backups with multiple safety checks,
+ * validation, and rollback capabilities. It prioritizes data safety over speed.
+ *
+ * Features:
+ * - Pre-restore validation and integrity checks
+ * - Automatic pre-restore backup creation
+ * - Atomic operations where possible
+ * - Comprehensive rollback capability
+ * - Detailed logging of every action
+ * - Dry-run mode for testing
+ * - Multiple restore options (full, database-only, files-only, selective)
+ * - S3 support with resume capability
+ * - Post-restore verification
+ *
+ * @class RestoreService
+ */
+class RestoreService {
+ constructor() {
+ this.isRunning = false;
+ this.currentProgress = null;
+ this.restoreLog = [];
+ this.preRestoreBackupPath = null;
+ this.dbType = knexConfig.client === 'pg' ? 'postgresql' : 'sqlite';
+ this.tempDir = path.join(os.tmpdir(), 'picpeak-restore');
+ }
+
+ /**
+ * Main restore method with comprehensive safety checks
+ *
+ * @param {Object} options - Restore options
+ * @param {string} options.source - Backup source (file path or S3 URL)
+ * @param {string} options.manifestPath - Path to backup manifest
+ * @param {string} options.restoreType - 'full', 'database', 'files', or 'selective'
+ * @param {Array} options.selectedItems - For selective restore, array of items to restore
+ * @param {boolean} options.dryRun - If true, performs validation only
+ * @param {boolean} options.skipPreBackup - Skip automatic pre-restore backup (dangerous!)
+ * @param {boolean} options.force - Force restore even with warnings (dangerous!)
+ * @param {Object} options.s3Config - S3 configuration for S3-based backups
+ * @returns {Promise