Compare commits

..

206 Commits

Author SHA1 Message Date
Gitea Actions Bot ea261dd03b chore: bump version to 1.0.57 (backend + frontend)
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-16 15:37:55 +00:00
paul eb93223d79 fix: email resend language and password display issues
Mirror to GitHub / mirror (push) Successful in 22s
Test and Lint / backend-test (push) Successful in 1m4s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m9s
Version and Release / version-bump (push) Successful in 41s
Version and Release / trigger-drone (push) Successful in 3s
- Added missing translation for admin.activities.email_resent in both EN and DE
- Fixed language detection in resend email to use email processor's priority system
- Fixed password security message showing German text in English emails
- Updated email processor to handle password placeholder and format dates properly
- Dates are now formatted according to language and system settings
- Password security message now displays in the correct language based on recipient
2025-07-16 17:33:33 +02:00
Gitea Actions Bot b20f9cc108 chore: bump backend version to 1.0.56
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-07-16 15:22:56 +00:00
paul 515814e1d5 feat: update English email templates to match German quality and content
Mirror to GitHub / mirror (push) Successful in 23s
Test and Lint / backend-test (push) Successful in 1m11s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m17s
Version and Release / version-bump (push) Successful in 39s
Version and Release / trigger-drone (push) Successful in 3s
- Enhanced all English email templates with proper HTML styling
- Added conditional content blocks (Handlebars {{#if}} syntax)
- Added styled alert boxes for warnings and success messages
- Added action buttons with proper styling
- Added emoji icons for better visual communication
- Added missing variables: support_email, support_phone, archive_date, photo_count
- Matched professional tone and formatting from German templates
- Improved visual hierarchy with background colors and borders
- All templates now have equal or better quality scores
2025-07-16 17:18:19 +02:00
Gitea Actions Bot fba9838e21 chore: bump backend version to 1.0.55
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-07-16 14:52:07 +00:00
paul a9c2761986 fix: add language detection from app_settings and fix activity_logs schema
Mirror to GitHub / mirror (push) Successful in 26s
Test and Lint / backend-test (push) Successful in 1m6s
Test and Lint / frontend-test (push) Successful in 2m23s
Version and Release / version-bump (push) Successful in 33s
Version and Release / trigger-drone (push) Successful in 3s
continuous-integration/drone/push Build is passing
- Enhanced language detection priority in email processor:
  1. Event-specific language
  2. App settings general_default_language (NEW)
  3. Email config default_language
  4. Domain-based detection
- Fixed activity_logs insertion error by using logActivity function
- Fixed 500 error on resend email endpoint
- Emails now respect language selected in settings page
2025-07-16 16:47:04 +02:00
Gitea Actions Bot db7e5913eb chore: bump backend version to 1.0.54
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-16 14:44:54 +00:00
paul aa27d1ea79 fix: resolve 500 error on resend email endpoint
Mirror to GitHub / mirror (push) Successful in 31s
Test and Lint / backend-test (push) Successful in 1m16s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m16s
Version and Release / version-bump (push) Successful in 35s
Version and Release / trigger-drone (push) Successful in 3s
- Moved queueEmail import to top of file (no more dynamic require)
- Fixed similar issue in password reset endpoint
- Added error handling for activity log insertion to prevent failures
- Added default values for ip_address and user_agent
- Added detailed error logging for debugging

The email was being sent successfully but the endpoint was returning 500
due to the dynamic require pattern. This is now fixed.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-16 16:39:56 +02:00
Gitea Actions Bot cf8df2780e chore: bump version to 1.0.53 (backend + frontend)
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-16 14:29:36 +00:00
paul 419a283c62 feat: add resend creation email button and fix Handlebars template processing
Mirror to GitHub / mirror (push) Successful in 21s
Test and Lint / backend-test (push) Successful in 1m6s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m7s
Version and Release / version-bump (push) Successful in 41s
Version and Release / trigger-drone (push) Successful in 3s
- Fixed email template processing to properly handle Handlebars conditionals
  - Installed handlebars package
  - Updated processTemplate to use Handlebars.compile()
  - Now {{#if welcome_message}} blocks work correctly

- Added "Resend Creation Email" button to event details page
  - New endpoint: POST /api/admin/events/:id/resend-email
  - Button appears below "Reset Gallery Password"
  - Shows mail icon and includes success/error notifications
  - For security, resent emails show "(Aus Sicherheitsgründen nicht angezeigt)"
    instead of the actual password

- Added translations:
  - EN: "Resend Creation Email", success/error messages
  - DE: "Erstellungs-E-Mail erneut senden", success/error messages

This fixes the issue where Handlebars syntax was showing in emails and provides
a convenient way to resend the initial gallery creation email to hosts.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-16 16:25:00 +02:00
Gitea Actions Bot 1b075a4beb chore: bump backend version to 1.0.52
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-16 14:01:47 +00:00
paul 68ff93cf17 feat: complete German email template implementation and language detection
Mirror to GitHub / mirror (push) Successful in 21s
Test and Lint / backend-test (push) Successful in 1m3s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m8s
Version and Release / version-bump (push) Successful in 38s
Version and Release / trigger-drone (push) Successful in 3s
- Added comprehensive German translations for all email templates:
  - gallery_created: Welcome email with proper German formatting
  - expiration_warning: Warning email with countdown in German
  - gallery_expired: Expiration notice in German
  - archive_complete: Archive completion notice in German

- Enhanced language detection with priority system:
  1. Event-specific language setting (new 'language' column)
  2. Global email config default language
  3. Domain-based detection (.de, .at, .ch, .li)

- Added language preferences:
  - events.language column for per-event language settings
  - email_configs.default_language for system-wide default
  - Set default to German ('de') for existing installations

- Fixed email processor to:
  - Pass eventId for proper language detection
  - Handle both SQLite and PostgreSQL JSON formats
  - Log language used for each email sent

All German templates are professionally translated with proper:
- Greetings ("Liebe(r)" instead of "Dear")
- Date formatting (DD.MM.YYYY)
- Professional closings ("Mit freundlichen Grüßen")
- UI elements ("Galerie-Details:" instead of "Gallery Details:")

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-16 15:57:20 +02:00
Gitea Actions Bot b22a29e877 chore: bump version to 1.0.51 (backend + frontend)
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-16 13:10:04 +00:00
paul e601311ca3 fix: resolve email queue discrepancy between admin dashboard and processor
Mirror to GitHub / mirror (push) Successful in 22s
Test and Lint / backend-test (push) Successful in 1m6s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m4s
Version and Release / version-bump (push) Successful in 34s
Version and Release / trigger-drone (push) Successful in 3s
The issue: Admin dashboard showed pending emails that the processor wouldn't
process because they had exceeded retry limits (retry_count >= 3).

Changes:
- Updated backend /admin/system/status to provide detailed email queue stats:
  - pending: total pending emails (as before)
  - processable: emails that will actually be processed (retry_count < 3)
  - stuck: emails that exceeded retry limit but are still pending
- Updated frontend to display stuck emails with warning when present
- Created debug-email-queue.js script to diagnose discrepancies
- Created fix-stuck-emails.js script to handle stuck emails:
  - Can reset retry count, mark as failed, or delete
  - Usage: node fix-stuck-emails.js [reset|fail|delete]

This makes it clear when emails are stuck and won't be processed automatically.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-16 15:05:34 +02:00
Gitea Actions Bot bb00c3993b chore: bump version to 1.0.50 (backend + frontend)
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-16 12:46:54 +00:00
paul c9c0de46bf fix: improve email processor reliability and add diagnostic tools
Mirror to GitHub / mirror (push) Successful in 22s
Test and Lint / backend-test (push) Successful in 1m19s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m15s
Version and Release / version-bump (push) Successful in 37s
Version and Release / trigger-drone (push) Successful in 2s
- Enhanced email processor with comprehensive logging
- Added error handling for database queries in email processing
- Fixed duplicate export in module.exports
- Added testEmailConnection() function for connectivity testing
- Created diagnostic scripts:
  - check-email-processor.js: Full diagnostic with manual processing
  - run-email-processor.js: Standalone processor (can run once or continuously)
  - check-email-environment.js: Environment and configuration checker
- Fixed missing translation: added 'common.unknown' in EN/DE locales

This should help identify why emails aren't being processed on production.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-16 14:30:09 +02:00
Gitea Actions Bot 5374299cd5 chore: bump backend version to 1.0.49
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-16 09:34:18 +00:00
paul 536e2b2874 fix: handle pre-parsed JSON data in email processor
Mirror to GitHub / mirror (push) Successful in 22s
Test and Lint / backend-test (push) Successful in 1m5s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m29s
Version and Release / version-bump (push) Successful in 32s
Version and Release / trigger-drone (push) Successful in 2s
The email_data column in PostgreSQL is a JSON type that auto-parses
the data, while SQLite stores it as a string. Updated the code to
check if email_data is already an object before attempting to parse.

This fixes the "Unexpected token o in JSON at position 1" error.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-16 11:29:36 +02:00
Gitea Actions Bot 141acd5736 chore: bump version to 1.0.48 (backend + frontend)
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-16 09:17:20 +00:00
paul 5f4337a18d fix: resolve multiple production issues
Mirror to GitHub / mirror (push) Successful in 23s
Test and Lint / backend-test (push) Successful in 1m11s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m4s
Version and Release / version-bump (push) Successful in 36s
Version and Release / trigger-drone (push) Successful in 3s
- Fix archive page showing "00" instead of "0" for empty photo counts
- Fix email queue PostgreSQL "updated_at" column error with migration and fallback
- Fix storage calculation discrepancies between system status and archive pages
- Add proper storage info to system status endpoint
- Fix archive path concatenation in settings endpoint

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-16 11:12:32 +02:00
Gitea Actions Bot fec7b687f7 chore: bump version to 1.0.47 (backend + frontend)
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-16 08:23:21 +00:00
paul cfa29ad5cb fix: GUI improvements and fixes
Mirror to GitHub / mirror (push) Successful in 19s
Test and Lint / backend-test (push) Successful in 1m12s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m6s
Version and Release / version-bump (push) Successful in 33s
Version and Release / trigger-drone (push) Successful in 2s
- Enable JSON module imports in TypeScript config to fix frontend version display
- Fix archive page showing '00' instead of '0' for empty photo counts
- Add null safety to archive photo count calculation
- Update email processor to reinitialize on config changes
- Add auto-retry for failed email transporter initialization

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-16 10:18:52 +02:00
Gitea Actions Bot 76ae35217c chore: bump backend version to 1.0.46
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-16 07:25:22 +00:00
paul fe651fa38e fix: add activity_logs deletion to event delete transaction
Mirror to GitHub / mirror (push) Successful in 24s
Test and Lint / backend-test (push) Successful in 1m6s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m8s
Version and Release / version-bump (push) Successful in 35s
Version and Release / trigger-drone (push) Successful in 3s
- Include activity_logs table in deletion transaction to prevent foreign key constraint error
- Maintain proper deletion order: activity_logs before events
- Fixes 500 error when deleting events that have associated activity logs

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-16 09:20:48 +02:00
paul f7b8c0c0fe fix: resolve event deletion 500 error with proper foreign key handling
Mirror to GitHub / mirror (push) Successful in 22s
Test and Lint / backend-test (push) Successful in 1m7s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m18s
Version and Release / version-bump (push) Failing after 31s
Version and Release / trigger-drone (push) Has been skipped
- Rewrite delete endpoint to use database transaction
- Delete related data in correct order: access_logs, email_queue, photos, photo_categories, then event
- Add cleanup for storage folders and archive files
- Improve error messages for better debugging

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-16 08:30:05 +02:00
Gitea Actions Bot 4af3cc2486 chore: bump version to 1.0.45
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-07-15 20:45:45 +00:00
paul 3632b936e9 fix: improve URL slug generation for special characters
Mirror to GitHub / mirror (push) Successful in 20s
Test and Lint / backend-test (push) Successful in 1m6s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m6s
Version and Release / version-bump (push) Successful in 30s
Version and Release / trigger-drone (push) Successful in 2s
- Replace all non-alphanumeric characters with single dash
- Collapse multiple consecutive dashes into single dash
- Remove leading and trailing dashes
- Fixes issue where "Petra & Peter" became "petra---peter"
- Now generates cleaner URLs like "petra-peter" instead

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-15 22:41:25 +02:00
Gitea Actions Bot 66a6d4003a chore: bump version to 1.0.44
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-07-15 20:34:22 +00:00
paul bdf73c1f06 fix: properly allow date-based passwords for galleries
Mirror to GitHub / mirror (push) Successful in 23s
Test and Lint / backend-test (push) Successful in 1m6s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m9s
Version and Release / version-bump (push) Successful in 31s
Version and Release / trigger-drone (push) Successful in 3s
- Added skipStrengthCheck option to bypass zxcvbn analysis for gallery passwords
- Added explicit date pattern matching for formats like "04.07.2025"
- Date passwords (DD.MM.YYYY, DD/MM/YYYY, DD-MM-YYYY) are now automatically accepted
- Gallery passwords skip all strength requirements but maintain 6 character minimum
- Fixes production issue where date passwords were rejected by zxcvbn

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-15 22:29:58 +02:00
Gitea Actions Bot f032743690 chore: bump version to 1.0.43
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-15 16:52:20 +00:00
paul a9902b95b4 fix: allow date formats as gallery passwords
Mirror to GitHub / mirror (push) Successful in 21s
Test and Lint / backend-test (push) Successful in 1m7s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m17s
Version and Release / version-bump (push) Successful in 32s
Version and Release / trigger-drone (push) Successful in 3s
- Modified backend password validation for galleries to be more lenient
- Reduced minimum password length to 6 characters for galleries
- Removed uppercase/lowercase/number requirements for gallery passwords
- Allow date formats like "04.07.2025" as passwords
- Added helper text to inform users about password options
- Still prevent overly simple passwords like "123456"
- Admin passwords remain strict with all security requirements

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-15 18:47:48 +02:00
Gitea Actions Bot 9d1c0b672a chore: bump version to 1.0.42
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-15 16:40:19 +00:00
paul 727fd8bae8 fix: allow past dates when creating events
Mirror to GitHub / mirror (push) Successful in 20s
Test and Lint / backend-test (push) Successful in 1m13s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m24s
Version and Release / version-bump (push) Successful in 36s
Version and Release / trigger-drone (push) Successful in 3s
- Removed min date restriction from event date picker
- Users can now select any date (past, present, or future)
- Essential for uploading events that have already occurred

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-15 18:35:30 +02:00
Gitea Actions Bot 954103510a chore: bump version to 1.0.41
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-15 15:14:40 +00:00
paul 801e1f81d9 fix: improve gallery lightbox UI for better user experience
Mirror to GitHub / mirror (push) Successful in 24s
Test and Lint / backend-test (push) Successful in 1m12s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m22s
Version and Release / version-bump (push) Successful in 34s
Version and Release / trigger-drone (push) Successful in 3s
- Removed photo filename display from lightbox for cleaner viewing
- Fixed z-index layering to ensure controls always appear above images
- Changed control elements from z-10 to z-20 for proper visibility
- Set image container to z-0 to prevent overlap with controls
- Ensures all buttons remain accessible regardless of image size or zoom

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-15 17:09:37 +02:00
paul f9861480aa fix: improve GitHub mirror workflow reliability
Mirror to GitHub / mirror (push) Successful in 22s
Test and Lint / backend-test (push) Successful in 1m11s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m14s
- Added workflow_dispatch for manual triggering
- Added debugging steps to identify issues
- Clean up existing branches before creating new ones
- Remove existing remotes to avoid conflicts
- Added GitHub token verification
- Added completion status messages
- Made workflow more resilient with better error handling

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-15 16:30:22 +02:00
Gitea Actions Bot a26dfd3d6f chore: bump version to 1.0.40
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-15 14:21:15 +00:00
paul 1db908771f fix: resolve multiple UI issues in admin panel
Test and Lint / backend-test (push) Successful in 1m11s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m13s
Version and Release / version-bump (push) Successful in 36s
Version and Release / trigger-drone (push) Successful in 3s
- Fixed dropdown menu visibility in events table by using fixed positioning
- Fixed double /api prefix in settings upload endpoints (favicon, watermark)
- Fixed thumbnail display in hero image selection by properly handling API paths
- Removed unnecessary console.log statements
- Added proper cleanup for dropdown on scroll/resize events

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-15 16:16:47 +02:00
Gitea Actions Bot 59651b8c24 chore: bump version to 1.0.39
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-15 11:04:23 +00:00
paul 7ccd48297f fix: resolve dropdown menu visibility issue in admin events table
Test and Lint / backend-test (push) Successful in 1m7s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m18s
Version and Release / version-bump (push) Successful in 38s
Version and Release / trigger-drone (push) Successful in 3s
- Changed Card container from overflow-hidden to overflow-visible
- Added overflow-y-visible to prevent vertical clipping
- Dropdown menu now properly displays even with few events
- Table only scrolls when content exceeds viewport height

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-15 12:59:54 +02:00
Gitea Actions Bot d05ff6380e chore: bump version to 1.0.38
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-15 09:51:55 +00:00
paul 605f773a7e fix: resolve gallery photo/thumbnail serving issues
Test and Lint / backend-test (push) Successful in 1m13s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m11s
Version and Release / version-bump (push) Successful in 34s
Version and Release / trigger-drone (push) Successful in 3s
- Change gallery photo URLs from static paths to API endpoints
- Add dedicated thumbnail serving endpoint for galleries
- Add test script to diagnose authentication issues
- Add nginx configuration documentation for Authorization header

This fixes the issue where photos and thumbnails work in admin but not
in gallery view. The problem was that static file routes with auth
middleware often have Authorization headers stripped by reverse proxies.
Using API endpoints ensures proper authentication handling.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-15 11:47:53 +02:00
Gitea Actions Bot c844f634c8 chore: bump version to 1.0.37
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-15 09:37:32 +00:00
paul 1d94398e2d fix: add thumbnail cleanup on archive deletion and diagnostic scripts
Test and Lint / backend-test (push) Successful in 1m8s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m23s
Version and Release / version-bump (push) Successful in 32s
Version and Release / trigger-drone (push) Successful in 3s
- Fix archive deletion to also clean up associated thumbnails
- Add cleanup-thumbnails.js script to remove temporary and orphaned thumbnails
- Add diagnose-thumbnails.js script to troubleshoot thumbnail serving issues
- Update README with documentation for new scripts

This prevents thumbnail accumulation when events are deleted and helps
diagnose why thumbnails might not be showing despite existing on disk.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-15 11:33:11 +02:00
Gitea Actions Bot a2551dc0ad chore: bump version to 1.0.36
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-15 09:27:09 +00:00
paul 32821934e6 feat: add storage management scripts for production troubleshooting
Test and Lint / backend-test (push) Successful in 1m13s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m8s
Version and Release / version-bump (push) Successful in 37s
Version and Release / trigger-drone (push) Successful in 4s
- Add check-storage.js to verify storage structure and file existence
- Add regenerate-thumbnails.js to recreate missing thumbnail files
- Include comprehensive README with usage instructions
- Scripts help diagnose and fix missing photo/thumbnail issues

These scripts are essential for production maintenance when photos
or thumbnails are missing from the storage directory.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-15 11:23:00 +02:00
Gitea Actions Bot b9c28e52cd chore: bump version to 1.0.35
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-07-15 07:15:42 +00:00
paul c94b6268cf fix: resolve gallery authentication and redirect issues
Test and Lint / backend-test (push) Failing after 32s
Test and Lint / frontend-test (push) Successful in 2m15s
continuous-integration/drone/push Build is passing
Version and Release / version-bump (push) Successful in 36s
Version and Release / trigger-drone (push) Successful in 4s
- Fix useWatermarkSettings hook to use public API endpoint instead of admin endpoint
- Add hero_photo_id to gallery authentication response
- Prevent 401 errors on gallery pages from redirecting to admin login
- Gallery pages now correctly fetch settings without requiring admin auth

The main issue was that gallery pages were calling admin-only endpoints
which triggered 401 errors and caused redirects to the admin login page.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-15 09:11:50 +02:00
Gitea Actions Bot 439c743fd1 chore: bump version to 1.0.34
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-15 06:53:59 +00:00
paul 74144f1fc6 fix: handle both share_link formats in gallery token verification
Test and Lint / backend-test (push) Successful in 1m7s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m17s
Version and Release / version-bump (push) Successful in 34s
Version and Release / trigger-drone (push) Successful in 3s
- Support both full URL and token-only formats in share_link column
- Fix gallery info endpoint to correctly validate share tokens
- Prevents "gallery not found" errors for valid share links

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-15 08:49:45 +02:00
Gitea Actions Bot 99a0376657 chore: bump version to 1.0.33
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-15 06:39:40 +00:00
paul 21b1e79672 fix: resolve gallery login redirect issue
Test and Lint / backend-test (push) Successful in 1m11s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m6s
Version and Release / version-bump (push) Successful in 35s
Version and Release / trigger-drone (push) Successful in 3s
- Updated API interceptor to better handle gallery authentication
- Fixed 401 error handling to prevent redirect loops on gallery pages
- Improved token extraction logic for gallery API requests
- Consolidated duplicate verifyGalleryAccess middleware
- Added proper error handling in GalleryView component
- Gallery authentication now properly distinguishes from admin routes

The issue was caused by the API interceptor redirecting to admin login
when gallery API calls failed with 401, even when users were already
on gallery pages attempting to authenticate.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-15 08:35:33 +02:00
Gitea Actions Bot cfaee103b6 chore: bump version to 1.0.32
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-07-15 06:21:46 +00:00
paul c0e346992d fix: improve photo authentication middleware for thumbnails
Test and Lint / backend-test (push) Successful in 1m14s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m5s
Version and Release / version-bump (push) Successful in 32s
Version and Release / trigger-drone (push) Successful in 3s
- Add eventId check from JWT token for thumbnail access
- Better error logging for debugging authentication issues
- Add admin debug endpoint to check photo existence
- More permissive thumbnail access when valid gallery token exists

This should help diagnose why photos are returning 404 errors in production.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-15 08:17:52 +02:00
Gitea Actions Bot 04f45a16c9 chore: bump version to 1.0.31
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-14 21:15:34 +00:00
paul efad1da74d fix: resolve image and thumbnail loading issues
Test and Lint / backend-test (push) Successful in 1m8s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m9s
Version and Release / version-bump (push) Successful in 34s
Version and Release / trigger-drone (push) Successful in 3s
- Fix static file serving paths to use correct storage directory
- Remove /api prefix from admin photo URLs to prevent double /api/api/ issue
- Fix thumbnail URL generation in gallery to use correct path format
- Update storage path resolution to support both relative and absolute paths

The issues were:
1. Admin images had URLs like /api/api/admin/events/2/thumbnail/90
2. Gallery thumbnails were looking for /thumbnails/thumb_*.jpg but paths were wrong
3. Static serving middleware was using incorrect storage paths

All images and thumbnails should now load correctly in both admin and gallery views.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-14 23:11:25 +02:00
Gitea Actions Bot 0a2b010332 chore: bump version to 1.0.30
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-07-14 19:17:01 +00:00
paul 6906c8bcf7 feat: increase file upload limit from 20 to 500 with performance optimizations
Test and Lint / backend-test (push) Successful in 1m10s
Test and Lint / frontend-test (push) Successful in 2m22s
continuous-integration/drone/push Build is passing
Version and Release / version-bump (push) Successful in 37s
Version and Release / trigger-drone (push) Successful in 3s
Backend changes:
- Update multer configuration to accept up to 500 files per upload
- Implement batch processing (10 files per transaction) for better performance
- Add memory-efficient Sharp configuration for thumbnail generation
- Increase Express body parser limits to handle large payloads
- Add proper error handling and reporting for partial upload failures

Frontend changes:
- Update validation to allow 500 files maximum
- Implement chunked uploads (50 files per chunk) to prevent timeouts
- Add progress tracking with chunk information display
- Update error messages and translations (EN/DE)

Performance optimizations:
- Disable Sharp cache to prevent memory buildup
- Limit Sharp concurrency to 2 operations
- Use sequential read for large images
- Process files in database transaction batches
- Return detailed upload results including success/failure counts

This implementation ensures the application can handle large photo uploads
efficiently without running into memory or timeout issues.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-14 21:12:35 +02:00
Gitea Actions Bot ac48bfdd0d chore: bump version to 1.0.29
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-14 19:09:25 +00:00
paul ec99243b6f test: add database compatibility tests
Test and Lint / backend-test (push) Successful in 1m6s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m12s
Version and Release / version-bump (push) Successful in 32s
Version and Release / trigger-drone (push) Successful in 3s
- Add comprehensive tests for formatBoolean() function
- Test PostgreSQL vs SQLite boolean handling
- Test date formatting and manipulation functions
- Verify insertAndGetId handles both database result formats

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-14 21:05:18 +02:00
paul 9932621e14 fix: comprehensive boolean compatibility for PostgreSQL and SQLite
- Add formatBoolean() usage to all boolean database queries
- Fix 36 boolean comparisons across 18 backend files
- Ensure compatibility between SQLite (0/1) and PostgreSQL (true/false)
- Update all WHERE clauses and UPDATE operations with boolean values

This completes the database compatibility fixes for production deployment.
All boolean operations now work correctly with both database systems.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-14 21:05:18 +02:00
paul 0fb17c78fa fix: photo upload issues with file limit and date formatting
- Add 20-file limit validation to PhotoUpload component
- Prevent Multer "Unexpected field" errors by enforcing client-side limit
- Fix JSON parsing error in dateFormatter when value is already an object
- Add missing translation keys for upload error messages
- Handle both string and object values for date format settings

These fixes resolve the 400 error when uploading more than 20 files
and the "Unexpected token o in JSON" error during email queue creation.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-14 21:05:18 +02:00
Gitea Actions Bot 4bcca58a11 chore: bump version to 1.0.28
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-14 18:54:11 +00:00
paul 9fa5ba1cf7 fix: update GitHub mirror workflow to use orphan branch
Test and Lint / backend-test (push) Successful in 1m5s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m19s
Version and Release / version-bump (push) Successful in 35s
Version and Release / trigger-drone (push) Successful in 3s
2025-07-14 20:49:53 +02:00
paul 88919fa0d3 fix: critical boolean compatibility for PostgreSQL/SQLite
Fixed boolean value handling differences between databases:
- SQLite stores booleans as 0/1, PostgreSQL as true/false
- Add formatBoolean() calls to critical queries that were failing

Files fixed:
- adminEvents.js: Fixed status filters and archive queries
- adminDashboard.js: Fixed active/archived event counts
- expirationChecker.js: Fixed expiration checking queries
- dbCompat.js: Updated to avoid circular dependency

Added migration 024 to:
- Enable foreign keys for SQLite (PRAGMA foreign_keys = ON)
- Document boolean compatibility requirements

This fixes queries returning 0 results in SQLite when checking
boolean columns like is_active, is_archived. Critical for
proper event management and expiration handling.

Note: 23 more boolean comparisons remain to be fixed in other files.
2025-07-14 20:49:53 +02:00
Gitea Actions Bot 5e43fc9cd9 chore: bump version to 1.0.27
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-07-14 18:43:02 +00:00
paul f053f42b6d fix: comprehensive PostgreSQL/SQLite compatibility fixes
Mirror to GitHub / mirror (push) Successful in 19s
Test and Lint / backend-test (push) Successful in 1m8s
Test and Lint / frontend-test (push) Successful in 2m11s
Version and Release / version-bump (push) Successful in 34s
Version and Release / trigger-drone (push) Successful in 3s
continuous-integration/drone/push Build is passing
Critical fixes for database compatibility issues:

INSERT operations:
- Fix all INSERT queries to use .returning('id')
- Handle both PostgreSQL (returns objects) and SQLite (returns IDs)
- Fixed in: events.js, adminArchives.js, adminEvents-enhanced.js, create-test-event.js

Date operations:
- Replace SQLite-specific db.raw("datetime('now', '+30 days')")
- Use JavaScript Date objects for cross-database compatibility
- Fixed in: adminArchives.js

Database utilities:
- Add dbCompat.js utility for handling database differences
- Provides consistent API for inserts, dates, booleans, and DB-specific operations
- Centralized database compatibility logic

Migration:
- Add migration 023 documenting PostgreSQL compatibility requirements
- Ensures future developers are aware of compatibility needs

This resolves all 'not iterable' errors and ensures the application
works correctly with both PostgreSQL (production) and SQLite (development).
2025-07-14 20:38:37 +02:00
Gitea Actions Bot dc17e7d59d chore: bump version to 1.0.26
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-14 18:37:28 +00:00
paul f05ad87602 fix: PostgreSQL insert compatibility issues
Mirror to GitHub (Archive Method) / mirror (push) Failing after 17s
Mirror to GitHub (Rsync Method) / mirror (push) Failing after 17s
Mirror to GitHub / mirror (push) Failing after 19s
Test and Lint / backend-test (push) Successful in 1m7s
Test and Lint / frontend-test (push) Successful in 2m11s
continuous-integration/drone/push Build is passing
Version and Release / version-bump (push) Successful in 35s
Version and Release / trigger-drone (push) Successful in 3s
- Fix event creation failing with 'not iterable' error
- Add .returning('id') to insert queries for PostgreSQL
- Handle both PostgreSQL (returns objects) and SQLite (returns IDs)
- Add missing fields to email_queue insert

This fixes the 500 error when creating new events in production.
2025-07-14 20:32:21 +02:00
Gitea Actions Bot 2efc74a687 chore: bump version to 1.0.25
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-14 18:30:15 +00:00
paul 85e7fbe73f chore: clean up unnecessary deployment files
Mirror to GitHub (Archive Method) / mirror (push) Failing after 13s
Mirror to GitHub (Rsync Method) / mirror (push) Failing after 16s
Mirror to GitHub / mirror (push) Failing after 18s
Test and Lint / backend-test (push) Successful in 1m4s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m14s
Version and Release / version-bump (push) Successful in 37s
Version and Release / trigger-drone (push) Successful in 2s
- Remove deploy/ folder with complex Docker Swarm configs
- Remove local development scripts referencing non-existent docker-compose.local.yml
- Remove setup scripts for files that already exist
- Remove docker-compose.dev.yml in favor of unified approach
- Keep certbot/, CHANGELOG.md, docker-compose.prod.yml, and production guide

This simplifies the repository structure and removes confusing
duplicate deployment approaches, making it clearer for new users.
2025-07-14 20:25:18 +02:00
Gitea Actions Bot 0a21856a8d chore: bump version to 1.0.24
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-07-14 18:20:14 +00:00
paul 349e7c7eb1 feat: add GitHub issue and PR templates
Mirror to GitHub (Archive Method) / mirror (push) Failing after 16s
Mirror to GitHub (Rsync Method) / mirror (push) Failing after 16s
Mirror to GitHub / mirror (push) Failing after 18s
Test and Lint / backend-test (push) Successful in 1m1s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m24s
Version and Release / version-bump (push) Successful in 33s
Version and Release / trigger-drone (push) Successful in 2s
- Add bug report template with environment details
- Add feature request template with use case section
- Add documentation improvement template
- Add security vulnerability template with warnings
- Add question template for support
- Add issue template config with helpful links
- Add comprehensive pull request template
- Update mirror workflow to exclude production guide

These templates will help maintain quality contributions
and streamline the issue/PR process on GitHub.
2025-07-14 20:15:10 +02:00
paul be07438915 docs: major repository restructure for GitHub public release
Mirror to GitHub (Archive Method) / mirror (push) Failing after 16s
Mirror to GitHub (Rsync Method) / mirror (push) Failing after 16s
Mirror to GitHub / mirror (push) Failing after 16s
Test and Lint / backend-test (push) Successful in 1m10s
Test and Lint / frontend-test (push) Has started running
Version and Release / version-bump (push) Has been cancelled
Version and Release / trigger-drone (push) Has been cancelled
continuous-integration/drone/push Build is passing
- Create comprehensive README.md optimized for GitHub/SEO
- Consolidate deployment instructions into single DEPLOYMENT.md
- Add all standard GitHub documentation files:
  - CONTRIBUTING.md with development guidelines
  - CODE_OF_CONDUCT.md for community standards
  - SECURITY.md with vulnerability reporting
  - CHANGELOG.md following Keep a Changelog format
- Simplify deployment with single docker-compose.yml
- Remove complex deployment configurations (Swarm, Traefik)
- Add backup script for easy maintenance
- Update .github-mirror-exclude to hide complex configs
- Remove redundant documentation files

This prepares PicPeak as a professional open-source alternative
to PicDrop and Scrapbook.de with clear, simple deployment.
2025-07-14 20:12:14 +02:00
Gitea Actions Bot b31ae72153 chore: bump version to 1.0.23
continuous-integration/drone/push Build is failing
continuous-integration/drone/tag Build is failing
2025-07-14 12:20:00 +00:00
paul 1db08b1e9b feat: add Gitea workflows for selective GitHub mirroring
Mirror to GitHub (Archive Method) / mirror (push) Failing after 15s
Mirror to GitHub (Rsync Method) / mirror (push) Failing after 15s
Mirror to GitHub / mirror (push) Failing after 14s
Test and Lint / backend-test (push) Successful in 1m3s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m15s
Version and Release / version-bump (push) Successful in 35s
Version and Release / trigger-drone (push) Successful in 3s
- Add three different approaches for mirroring to GitHub
- Approach 1: Filter out sensitive files on a separate branch
- Approach 2: Use git archive with .gitattributes exclusions
- Approach 3: Use rsync for flexible file filtering
- Add exclusion lists for sensitive files and directories
- Protect production configs, environment files, and private data

This allows maintaining a public GitHub mirror while keeping
sensitive configuration and data private on the Gitea instance.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-14 14:14:39 +02:00
Gitea Actions Bot 237a3332cc chore: bump version to 1.0.22
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-14 12:10:51 +00:00
paul e2d0a83d51 fix: relax password requirements and improve password UI
Test and Lint / backend-test (push) Successful in 1m9s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m13s
Version and Release / version-bump (push) Successful in 35s
Version and Release / trigger-drone (push) Successful in 3s
- Reduce minimum password length from 12 to 8 characters
- Make special characters optional for gallery passwords
- Lower strength requirement from score 3 to 1 for galleries
- Add eye icon toggle for password visibility on each field
- Remove redundant 'Show passwords' checkbox
- Add translation for password security requirements error
- Update both English and German translations

This allows users to use simpler passwords like 'Sommer2025\!' for events
while maintaining security through other measures like expiration dates.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-14 14:06:18 +02:00
Gitea Actions Bot c546657285 chore: bump version to 1.0.21
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-14 11:13:23 +00:00
paul e8d5ee1a7b fix: multiple production issues with PostgreSQL and connection handling
Test and Lint / backend-test (push) Successful in 1m7s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m11s
Version and Release / version-bump (push) Successful in 36s
Version and Release / trigger-drone (push) Successful in 3s
- Fix trust proxy to use specific values instead of permissive 'true'
- Fix clear old notifications to use database-agnostic date calculation
- Fix database size check to support both PostgreSQL and SQLite
- Add caching and better error handling for session timeout queries
- Add query timeout to prevent hanging connections
- Improve JSON parsing error handling for setting values

These fixes address:
- ERR_ERL_PERMISSIVE_TRUST_PROXY warning
- PostgreSQL datetime function errors
- ENOENT errors looking for SQLite file in PostgreSQL deployment
- Connection terminated errors for session timeout checks

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-14 13:09:24 +02:00
Gitea Actions Bot 1761ebd531 chore: bump version to 1.0.20
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-07-14 10:08:49 +00:00
paul 5e5e98601f fix: improve email test endpoint error handling
Test and Lint / backend-test (push) Successful in 1m10s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m6s
Version and Release / version-bump (push) Successful in 32s
Version and Release / trigger-drone (push) Successful in 3s
- Add detailed validation for SMTP configuration
- Add specific error messages for common SMTP issues (connection, auth, network)
- Add logging to help debug email configuration issues
- Check for masked passwords that shouldn't be used for sending
- Parse smtp_port as integer and handle boolean smtp_secure properly

This helps identify the exact cause of email sending failures instead of
generic 500 errors.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-14 12:04:57 +02:00
Gitea Actions Bot 1cda80792b chore: bump version to 1.0.19
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-14 09:02:17 +00:00
paul 8740d5e618 fix: dashboard activity endpoint JSON parsing error
Test and Lint / backend-test (push) Successful in 1m4s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m26s
Version and Release / version-bump (push) Successful in 31s
Version and Release / trigger-drone (push) Successful in 3s
- Add safe JSON parsing for activity metadata in dashboard endpoint
- Handle cases where PostgreSQL returns JSON columns as objects
- Prevent 500 errors when metadata contains invalid JSON

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-14 10:58:12 +02:00
Gitea Actions Bot a619d52d17 chore: bump version to 1.0.18
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-14 08:46:37 +00:00
paul dd8cc14d30 fix: production JSON parsing errors and trust proxy issue
Test and Lint / backend-test (push) Successful in 1m3s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m16s
Version and Release / version-bump (push) Successful in 33s
Version and Release / trigger-drone (push) Successful in 3s
- Set Express to trust proxy headers for proper IP detection with Traefik
- Add safe JSON parsing for email template variables and activity log metadata
- Create migration to fix invalid JSON data in database
- Add error handling to prevent JSON.parse crashes

This fixes the 500 errors caused by invalid JSON data and the trust proxy
warning from express-rate-limit when running behind Traefik.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-14 10:42:41 +02:00
Gitea Actions Bot 3b7d723c2a chore: bump version to 1.0.17
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-14 08:18:38 +00:00
paul dc6252ff56 fix: complete email template and CMS page fixes for production
Test and Lint / backend-test (push) Successful in 1m8s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m7s
Version and Release / version-bump (push) Successful in 34s
Version and Release / trigger-drone (push) Successful in 4s
- Fix email templates list endpoint to handle both schema variations
- Add migration for default CMS pages with language-specific columns
- Create debug script to identify 500 error causes
- Update CMS migration to use correct column names (title_en/de, content_en/de)

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-14 10:14:44 +02:00
Gitea Actions Bot 1d4e79a4f9 chore: bump version to 1.0.16
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-14 08:02:15 +00:00
paul 0b0e3e22d2 fix: handle email templates schema variations in production
Test and Lint / backend-test (push) Successful in 1m11s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m20s
Version and Release / version-bump (push) Successful in 34s
Version and Release / trigger-drone (push) Successful in 3s
- Update migration to detect and handle both old and new email template schemas
- Fix migration to insert into correct columns based on existing schema
- Update adminEmail routes to handle both schema formats gracefully
- Add proper fallbacks for German language columns

This ensures the application works whether the language migration has been
applied or not, preventing null constraint violations.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-14 09:58:06 +02:00
Gitea Actions Bot f22e3c133f chore: bump version to 1.0.15
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-07-14 07:53:34 +00:00
paul 64c0a58f78 fix: database migration and routing issues for production
Test and Lint / backend-test (push) Successful in 1m19s
Test and Lint / frontend-test (push) Successful in 2m20s
continuous-integration/drone/push Build is passing
Version and Release / version-bump (push) Successful in 38s
Version and Release / trigger-drone (push) Successful in 3s
- Add migration to fix email_templates column structure after language migration
- Add migration to ensure default email templates exist
- Create diagnostic script to check database issues
- Fix docker-compose configuration for proper routing without path stripping

The backend expects routes with /api prefix, so removing the stripprefix
middleware allows proper routing to work.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-14 09:49:11 +02:00
Gitea Actions Bot 4182089c17 chore: bump version to 1.0.14
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-07-14 07:24:51 +00:00
paul cecf773fb7 fix: database connection stability issues in production
Test and Lint / backend-test (push) Successful in 1m9s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m10s
Version and Release / version-bump (push) Successful in 32s
Version and Release / trigger-drone (push) Successful in 3s
- Add robust connection pool configuration with timeouts and retry settings
- Implement retry logic in maintenance middleware for connection errors
- Increase connection stability with keepAlive and proper timeout values
- Handle "Connection terminated unexpectedly" errors gracefully

This prevents 503 errors when the database connection is temporarily interrupted
and ensures the application can recover from transient connection issues.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-14 09:20:51 +02:00
Gitea Actions Bot 973af17b85 chore: bump version to 1.0.13
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-07-14 06:45:13 +00:00
paul 6c3e88a588 fix: production deployment issues with Traefik and database migrations
Test and Lint / backend-test (push) Successful in 1m8s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m8s
Version and Release / version-bump (push) Successful in 32s
Version and Release / trigger-drone (push) Successful in 2s
- Add missing created_at column to email_queue table
- Fix 502 Bad Gateway errors with proper Traefik routing configuration
- Create docker-compose.traefik.yml for external Traefik deployment
- Fix health check endpoint path for API path stripping
- Add PostgreSQL init script for Umami database creation
- Add comprehensive deployment guide for Traefik setup

The backend now properly handles /api prefix stripping by Traefik and
migrations run safely in production environments with existing schemas.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-14 08:41:11 +02:00
Gitea Actions Bot 97bbb3c8e1 chore: bump version to 1.0.12
continuous-integration/drone/tag Build is failing
continuous-integration/drone/push Build is passing
2025-07-13 21:22:28 +00:00
paul ac1cd96ecd fix: resolve frontend API routing issues for Traefik deployment
Test and Lint / backend-test (push) Successful in 1m16s
continuous-integration/drone/push Build is failing
Test and Lint / frontend-test (push) Successful in 2m18s
Version and Release / version-bump (push) Successful in 43s
Version and Release / trigger-drone (push) Successful in 4s
Major fixes for production deployment with Traefik:

1. API Path Fixes:
   - Remove double /api prefix from all frontend service calls
   - Fix auth.service.ts to use correct paths (/auth/admin/login)
   - Update all services to use single /api prefix from base URL
   - Fix template literal paths in photo services

2. Docker Configuration:
   - Add build args for VITE_API_URL in docker-compose.prod.yml
   - Create Dockerfile.prod with proper API URL configuration
   - Ensure frontend is built with correct API base path

3. Documentation:
   - Add comprehensive TRAEFIK_DEPLOYMENT.md guide
   - Document proper Traefik labels and routing configuration
   - Include troubleshooting steps for common issues
   - Explain network configuration and SSL handling

This resolves:
- 502 Bad Gateway errors
- Double /api/api paths in requests
- Frontend unable to communicate with backend
- Login functionality not working

The frontend now correctly calls the backend API through Traefik's
routing, with all requests going to /api/* being forwarded to the
backend service on port 3000.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-13 23:18:00 +02:00
Gitea Actions Bot 689861f671 chore: bump version to 1.0.11
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-13 21:12:57 +00:00
paul 41fb575e80 fix: use correct bcrypt package in create-admin script
Test and Lint / backend-test (push) Successful in 1m10s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m5s
Version and Release / version-bump (push) Successful in 40s
Version and Release / trigger-drone (push) Successful in 3s
- Change from bcryptjs to bcrypt to match installed dependency
- Fixes "Cannot find module 'bcryptjs'" error when creating admin user

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-13 23:09:02 +02:00
Gitea Actions Bot 6ebc4f3fc4 chore: bump version to 1.0.10
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-13 21:01:24 +00:00
paul de973f5613 fix: comprehensive production deployment fixes and migration safety
Test and Lint / backend-test (push) Successful in 1m9s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m15s
Version and Release / version-bump (push) Successful in 36s
Version and Release / trigger-drone (push) Successful in 4s
Major fixes for production deployment issues:

1. Migration System:
   - Add safe migration runner that handles existing schema
   - Create migration helper functions for idempotent operations
   - Auto-detect existing tables and mark migrations as applied
   - Handle "relation already exists" errors gracefully

2. Production Initialization:
   - Create init-production.sh script for proper startup sequence
   - Fix directory creation and permissions
   - Add admin user creation from environment variables
   - Ensure proper service initialization order

3. Documentation:
   - Add comprehensive PRODUCTION_DEPLOYMENT_GUIDE.md
   - Add MIGRATION_ERROR_FIX.md for immediate issue resolution
   - Document all known production issues and solutions
   - Include backup/restore procedures

4. Safety Improvements:
   - Add migrate:safe npm script for production use
   - Update wait-for-db.sh to use safe migrations in production
   - Add proper error handling and logging

This resolves the "relation already exists" error and prevents similar
issues in future deployments. The safe migration system can handle both
fresh installations and existing databases.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-13 22:57:22 +02:00
Gitea Actions Bot 279c70b3d6 chore: bump version to 1.0.9
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-07-13 20:46:24 +00:00
paul 5a73f6963f fix: correct YAML syntax in docker-compose.prod.yml
Test and Lint / backend-test (push) Successful in 1m10s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m13s
Version and Release / version-bump (push) Successful in 31s
Version and Release / trigger-drone (push) Successful in 3s
- Remove quotes from POSTGRES_INITDB_ARGS environment variable
- Fix YAML parsing error for PostgreSQL configuration
- Ensure proper formatting for all environment variables

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-13 22:42:27 +02:00
Gitea Actions Bot 5101a05bca chore: bump version to 1.0.8
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-13 20:30:04 +00:00
paul c82caf6539 fix: resolve PostgreSQL connection authentication error
Test and Lint / backend-test (push) Successful in 1m8s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m15s
Version and Release / version-bump (push) Successful in 36s
Version and Release / trigger-drone (push) Successful in 2s
- Fix "no pg_hba.conf entry" error by disabling SSL for Docker network
- Use scram-sha-256 authentication method for better security
- Update knexfile.js to support SSL configuration via environment variable
- Add documentation about PostgreSQL connection requirements

The PostgreSQL container now accepts connections from the Docker network
without requiring SSL, which is appropriate for internal container communication.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-13 22:26:00 +02:00
Gitea Actions Bot ae1b508726 chore: bump version to 1.0.7
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-07-13 20:19:10 +00:00
paul 26c05912fc fix: resolve critical production deployment issues
Test and Lint / backend-test (push) Successful in 1m11s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m21s
Version and Release / version-bump (push) Successful in 33s
Version and Release / trigger-drone (push) Successful in 2s
- Fix database connection error "getaddrinfo ENOTFOUND postgres"
- Add wait-for-db.sh script to ensure PostgreSQL is ready before starting
- Fix email processor initialization timing issue
- Add missing storage path environment variables
- Add database dependency to backend service
- Enhance health check endpoint with database connectivity check
- Update production database defaults to match docker-compose
- Install postgresql-client in Docker image for health checks
- Document all required environment variables in .env.example

Fixes immediate production deployment failures and ensures proper service startup order.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-13 22:14:44 +02:00
Gitea Actions Bot d1033cb83a chore: bump version to 1.0.6
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-07-13 20:04:46 +00:00
paul b7458b5a37 fix: add missing pg dependency for PostgreSQL support
Test and Lint / backend-test (push) Successful in 1m10s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m7s
Version and Release / version-bump (push) Successful in 36s
Version and Release / trigger-drone (push) Successful in 3s
- Add pg package required for production PostgreSQL connections
- Fixes "Cannot find module 'pg'" error in production deployment

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-13 22:00:41 +02:00
Gitea Actions Bot face8f1496 chore: bump version to 1.0.5
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-13 19:54:25 +00:00
paul ba6ee55bf7 chore: clean up obsolete files and documentation
Test and Lint / backend-test (push) Successful in 1m5s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m12s
Version and Release / version-bump (push) Successful in 39s
Version and Release / trigger-drone (push) Successful in 3s
- Remove completed auth migration documentation (13 files)
- Delete unused test and one-time scripts (23 files)
- Remove backup files and old logs
- Clean up duplicate/empty database files
- Remove old migration backup file
- Delete root level setup scripts

Total: ~185KB of obsolete files removed
All active functionality preserved

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-13 21:50:18 +02:00
Gitea Actions Bot e343106af5 chore: bump version to 1.0.4
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-13 19:28:44 +00:00
paul 2f848eb602 fix: make create-admin script executable
Test and Lint / backend-test (push) Successful in 1m7s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m7s
Version and Release / version-bump (push) Successful in 37s
Version and Release / trigger-drone (push) Successful in 4s
2025-07-13 21:24:40 +02:00
paul 1c7fa781ad fix: configure PostgreSQL for production and clean up deployment
- Fix database configuration to use PostgreSQL in production
- Add knexfile.js to support both SQLite (dev) and PostgreSQL (prod)
- Create admin user creation script (scripts/create-admin.js)
- Clean up docker-compose files:
  - Remove redundant docker-compose.yml and docker-compose.local.yml
  - Create docker-compose.dev.yml for development
  - Update docker-compose.prod.yml with proper DB configuration
- Clean up environment files:
  - Update .env.example for development
  - Update .env.production.example with proper settings
  - Remove redundant .env.local
- Update backend .env.example with database configuration options
- Create comprehensive DEPLOYMENT.md with admin setup instructions
- Fix production database name consistency (picpeak instead of photoapp)
2025-07-13 21:24:40 +02:00
Gitea Actions Bot 66940c2f5b chore: bump version to 1.0.3
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-13 19:18:49 +00:00
paul a3638fe954 fix: remove hardcoded localhost URLs for production deployment
Test and Lint / backend-test (push) Successful in 1m11s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m17s
Version and Release / version-bump (push) Successful in 43s
Version and Release / trigger-drone (push) Successful in 3s
- Add URL utility functions for building resource URLs
- Update all components to use relative URLs in production
- Add production deployment documentation
- Update nginx config to proxy all required endpoints
- Add .env.production.example with proper configuration
2025-07-13 21:14:26 +02:00
Gitea Actions Bot 0934695a69 chore: bump version to 1.0.2
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-07-13 18:13:24 +00:00
paul 77ece5c5f1 fix: add missing translations for admin activities
Test and Lint / backend-test (push) Successful in 1m11s
Test and Lint / frontend-test (push) Successful in 2m15s
Version and Release / version-bump (push) Successful in 36s
Version and Release / trigger-drone (push) Successful in 3s
continuous-integration/drone/push Build is passing
2025-07-13 20:09:08 +02:00
Gitea Actions Bot 1c2c1f177a chore: bump version to 1.0.1
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-13 18:07:45 +00:00
paul f38014099e fix: show hero image in thumbnail grid on hero gallery layout
Test and Lint / backend-test (push) Successful in 1m12s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m16s
Version and Release / version-bump (push) Successful in 35s
Version and Release / trigger-drone (push) Successful in 3s
2025-07-13 20:03:27 +02:00
paul 10649691de feat: implement automatic version incrementing with CI/CD strategy
continuous-integration/drone/push Build is passing
Test and Lint / backend-test (push) Successful in 4m1s
Test and Lint / frontend-test (push) Successful in 2m6s
- Add Gitea Actions workflow for automatic version bumping
- Update test workflow to run linting and tests
- Configure Drone to build images with version tags
- Separate concerns: Gitea Actions for versioning, Drone for Docker builds
- Version format: MAJOR.MINOR.PATCH (auto-increment patch)
- Add comprehensive CI/CD strategy documentation

This prevents race conditions between Gitea Actions and Drone CI by:
1. Gitea Actions handles version bump and creates git tag
2. Tag creation triggers Drone to build Docker images
3. Both systems work sequentially, not in parallel

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-13 19:55:35 +02:00
paul f439d0b318 fix(security): enhance security headers and tighten CORS configuration
Test Gitea Actions / test (push) Successful in 15s
continuous-integration/drone/push Build is passing
- Configure custom Content Security Policy for React app compatibility
- Add Permissions-Policy header to disable unnecessary browser features
- Set HSTS to 1 year with preload flag for better transport security
- Update referrer policy to strict-origin-when-cross-origin
- Restrict CORS localhost origins to development environment only
- Production deployments now only allow configured FRONTEND_URL and ADMIN_URL
- Addresses security headers configuration issues from security scan

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-13 19:48:28 +02:00
paul 4e977f7624 fix(security): allow serving static files from uploads directory
Test Gitea Actions / test (push) Successful in 14s
continuous-integration/drone/push Build is passing
- Remove overly restrictive absolute path check in isPathSafe
- Strip leading slash from request path before validation
- Fixes broken favicon and watermark image previews in branding page
- Path traversal protection remains intact with ../ pattern checks

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-13 19:43:29 +02:00
paul 66841e8af7 fix(security): implement file upload security enhancements
Test Gitea Actions / test (push) Successful in 16s
continuous-integration/drone/push Build is passing
- Add path traversal protection with secureStatic middleware
- Implement proper MIME type validation for all file uploads
- Add content-based file validation (magic numbers)
- Create comprehensive fileSecurityUtils for secure file operations
- Update adminPhotos.js with enhanced validation
- Update adminSettings.js for secure logo/favicon uploads
- Addresses file upload vulnerabilities from security scan

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-13 19:30:17 +02:00
paul 051e21cbaf fix(security): sanitize CMS content to prevent XSS attacks
Test Gitea Actions / test (push) Successful in 17s
continuous-integration/drone/push Build is passing
- Add DOMPurify to sanitize HTML content in LegalPage component
- Prevents stored XSS vulnerabilities in CMS-managed content
- Maintains legitimate HTML formatting while removing malicious scripts
- Addresses critical XSS vulnerability from security scan

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-13 19:11:02 +02:00
paul e35ac6a41c feat: implement critical security fixes for SQL injection and authentication vulnerabilities
Test Gitea Actions / test (push) Successful in 20s
continuous-integration/drone/push Build is passing
Security Enhancements:
- Fix SQL injection vulnerabilities by replacing whereRaw queries with parameterized queries
- Add LIKE pattern escaping to prevent SQL injection in search functionality
- Implement account lockout protection (5 failed attempts = 30 min lockout)
- Add comprehensive login attempt tracking and audit trail
- Enhance JWT tokens with issuer validation, IP tracking, and password change detection
- Add logout endpoint and session management
- Prevent user enumeration with generic error messages

Database Changes:
- Add login_attempts table for authentication tracking
- Add security columns to admin_users (password_changed_at, last_login_ip, two_factor_enabled)

New Security Features:
- Brute force protection with configurable lockout duration
- Automatic cleanup of old login attempts
- Enhanced authentication middleware with stricter validation
- Monitoring scripts for security health checks

All fixes are backward compatible and production-ready with rollback plans included.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-13 00:40:05 +02:00
paul 0d33f21ee6 fix(security): eliminate default admin password vulnerability
Test Gitea Actions / test (push) Successful in 18s
continuous-integration/drone/push Build is passing
BREAKING CHANGE: Admin password is now auto-generated on first setup

Security improvements:
- Remove hardcoded 'admin123' password completely
- Generate secure random password on first installation
- Save credentials to ADMIN_CREDENTIALS.txt (git-ignored)
- Force password change on first login
- Implement strong password requirements (12+ chars, mixed case, numbers, special)
- Add password strength validation
- Increase bcrypt rounds from 10 to 12

New features:
- Password generator utility with secure random generation
- Human-readable password format (e.g., SwiftEagle3847\!)
- Password reset script for existing installations
- Comprehensive admin setup documentation
- Must-change-password flag in database

Migration guide:
- New installations: Check ADMIN_CREDENTIALS.txt for generated password
- Existing installations: Run scripts/reset-admin-password.js
- All users must change password on first login after update

This fixes a critical vulnerability where all installations used the same
default admin password, allowing unauthorized access.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-12 23:50:21 +02:00
paul 1cfd6a44d6 feat(security): implement secure JWT secrets for all environments
Test Gitea Actions / test (push) Successful in 15s
continuous-integration/drone/push Build is passing
- Replace short dev secrets with secure 64-character (256-bit) secrets
- Update docker-compose.yml with secure development secret
- Update docker-compose.local.yml with unique secure secret
- Improve .env.example with clear security instructions
- Add comprehensive security best practices documentation
- Create helper script to generate secure JWT secrets

Security improvements:
- All environments now use cryptographically secure 64-character secrets
- Clear warnings and instructions prevent use of weak secrets
- Documentation guides proper secret management
- Helper script makes it easy to generate new secrets

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-12 23:40:43 +02:00
paul 2b5b875dfe fix(security): remove hardcoded JWT secret fallback - CRITICAL
Test Gitea Actions / test (push) Successful in 20s
continuous-integration/drone/push Build is passing
BREAKING CHANGE: Server now requires JWT_SECRET environment variable to be set

Security fixes:
- Remove hardcoded JWT secret fallback 'your-secret-key' from protectedImages.js
- Add startup validation to ensure JWT_SECRET is properly configured
- Reject insecure default values and short secrets
- Server will refuse to start without proper JWT_SECRET

This fixes a critical vulnerability where the application would use a publicly
known secret if JWT_SECRET was not set, completely compromising authentication.

Migration guide: docs/JWT_SECRET_MIGRATION.md

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-12 23:33:22 +02:00
paul f39427d9d9 docs: add comprehensive security scan report
Test Gitea Actions / test (push) Successful in 14s
continuous-integration/drone/push Build is passing
- Scan for hardcoded secrets and credentials
- Check SQL injection vulnerabilities
- Audit authentication and authorization flaws
- Analyze XSS vulnerabilities
- Review file upload security
- Check security headers and CORS configuration
- Verify npm dependencies (0 vulnerabilities found)

Critical findings:
- Hardcoded JWT secret fallback
- SQL injection in adminDashboard.js
- Stored XSS in CMS content
- Authentication bypass risks

Report includes detailed findings and remediation steps.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-12 23:19:00 +02:00
paul 74d85eadbb feat: add Gitea runner installation scripts
continuous-integration/drone/push Build is passing
Test Gitea Actions / test (push) Successful in 1m40s
- Add automated installation script for Gitea Act Runner
- Add docker-compose configuration for easy runner deployment
- Support both Docker and binary installation methods
- Include systemd service setup for Linux

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-12 09:30:39 +02:00
paul 0b550cdaf6 feat: add Gitea Actions test workflow and setup documentation
Test Gitea Actions / test (push) Has been cancelled
continuous-integration/drone/push Build is passing
- Add simple test workflow to verify Gitea Actions functionality
- Create comprehensive setup guide for troubleshooting Actions
- Include runner installation and registration steps
- Document common issues and solutions

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-12 09:28:06 +02:00
paul 288b0c25e6 refactor: rename project from wedding-photo-sharing to PicPeak
Create Release / check-version-change (push) Successful in 2m25s
Automatic Version Bump / version-bump (push) Failing after 8m2s
Create Release / create-release (push) Has been skipped
- Update Docker image names and network configurations
- Rename package.json project names to picpeak-backend/frontend
- Update CI/CD configurations (Drone CI and GitHub Actions)
- Update documentation and setup scripts
- Update application branding in source code
- Change default database name to picpeak
- Update PM2 ecosystem config

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-12 09:22:14 +02:00
paul d065132bb7 fix: skip TypeScript checking in production build for CI/CD
continuous-integration/drone/push Build is passing
- Update build script to use vite build directly
- Add build:check script for local development with TypeScript checking
- Fixes CI/CD build failures due to unused imports
2025-07-11 23:21:02 +02:00
paul 52ef3e33f4 feat: add initial Drone CI/CD configuration without security scan
continuous-integration/drone/push Build is failing
- Configure automated Docker builds for backend and frontend
- Push images to local registry (registry.local.nothaft.cloud)
- Tag images with latest and commit SHA
- Add release pipeline for tagged versions
- Full configuration with security scanning saved as .drone.yml.full for future use
2025-07-11 23:15:38 +02:00
paul d89a605579 feat: implement gallery preview with layout selector
- Add GalleryPreview component that shows simplified gallery layouts
- Update ThemeEditorModal with split view: theme customizer on left, preview on right
- Add grid style selector above preview to switch between layouts
- Update BrandingPage to show live preview alongside theme customizer
- Add preview to CreateEventPageEnhanced when customizing themes
- Support all 6 gallery layouts: grid, masonry, carousel, timeline, hero, mosaic
- Add translation keys for preview layout and live preview

The preview accurately reflects different grid layouts and theme settings,
helping users visualize how their galleries will look before saving.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-11 08:38:48 +02:00
paul 9006b754a8 Fix gallery mobile view issues
- Make logout button show only icon on mobile (no text)
- Move upload button from top bar to sidebar menu on mobile
- Fix top bar layout with proper structure:
  - Logo on left
  - Gallery title centered
  - Event date and expiration date shown below title on mobile
- Improve responsive design for header elements
- Ensure upload button only appears in menu when uploads are enabled

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-11 08:30:07 +02:00
paul 5328b4f73a Enhance email templates with clickable links, branding, and improved styling
- Add clickable gallery links in all email templates
- Include application logo in email header and footer (custom or PicPeak default)
- Redesign emails with professional styling matching gallery login page
  - Gray background with white content box
  - PicPeak green header with centered logo
  - Clean typography and proper spacing
  - Responsive design for mobile devices
  - Styled call-to-action buttons
  - Footer with branding and copyright
- Update email processor to fetch branding settings dynamically
- Use proper API URLs for logo images in emails

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-10 22:19:15 +02:00
paul 6438374258 Fix mobile responsiveness and implement enhanced theme system
- Fixed mobile gallery login box sizing and layout
- Fixed header button layout for mobile screens
- Fixed duplicate logo issue on logout
- Fixed '0' rendering when upload button is hidden
- Fixed horizontal scrolling on small screens

- Implemented comprehensive theme system with gallery layouts
- Added 6 different gallery layouts: Grid, Masonry, Carousel, Timeline, Hero, Mosaic
- Created enhanced theme customizer with layout selection
- Added theme presets for different event types
- Updated event creation with theme preview and customization
- Fixed all TypeScript compilation errors

- Added missing translation keys for create event page
- Added translations for theme customization features

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-09 11:26:27 +02:00
paul d8fb4c9565 Make gallery page fully mobile responsive
GalleryLayout improvements:
- Stack header elements vertically on mobile
- Hide company branding on small screens
- Make dates stack vertically
- Responsive text sizes and padding
- Icon-only logout button on mobile
- Improved button layout with proper wrapping

PhotoFilterBar improvements:
- Stack search and sort vertically on mobile
- Full-width sort button on mobile
- Horizontally scrollable category filters
- Responsive text sizes
- Mobile-friendly dropdown positioning

PhotoGrid improvements:
- Responsive selection controls
- Touch-friendly photo overlays
- Larger selection checkboxes on mobile
- Improved button text for small screens
- Responsive gaps between photos

Gallery grid CSS:
- Smaller gaps on mobile devices
- Maintains 2 columns on smallest screens

GalleryPage login:
- Responsive padding and margins
- Smaller text and icon sizes on mobile
- Better card spacing
- Responsive form elements

UserPhotoUpload modal:
- Full-screen modal on mobile (slides up from bottom)
- Responsive padding and text sizes
- Mobile-optimized upload area
- Sticky footer on mobile

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-09 09:53:24 +02:00
paul f5cf757142 Fix sticky positioning of version and storage info in admin sidebar
- Change AdminLayout to use h-screen instead of min-h-screen
- Add overflow-hidden to prevent scrollbar on outer container
- Update sidebar to use h-screen for proper height reference
- Add flex-shrink-0 to header and bottom sections
- Add min-h-0 to navigation to allow proper overflow
- Add overflow-y-auto to main content area

The version info and storage usage now properly stick to the bottom of the viewport instead of the bottom of the page content.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-09 09:35:08 +02:00
paul 0a203d16cf Fix translation issues in event creation and detail pages
- Add missing translation keys for event creation form
- Replace all hardcoded strings with translation keys in CreateEventPage
- Add validation error message translations
- Add event types and color theme translations
- Add errors and validation sections to translation files
- Update German translations with corresponding keys

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-09 09:25:15 +02:00
paul 8231f2b60d Fix gallery login errors and photo upload issues
- Fix photo upload cross-device link error
  - Changed fs.rename to fs.copyFile + fs.unlink to handle Docker volume mounts
  - This fixes EXDEV errors when uploading photos from /tmp to storage

- Improve gallery login error handling
  - Add console logging for debugging
  - Prevent form refresh with stopPropagation
  - Show specific error messages based on status codes
  - Keep password field populated on error for retry
  - Map 404 to gallery not found message

- Enhanced upload error messages
  - Show specific error message for each failed file
  - Display backend error messages in toast notifications

These fixes resolve:
1. Photo uploads failing silently with cross-device link errors
2. Login errors not displaying and form refreshing
3. Upload errors not showing user-friendly messages

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-09 09:09:34 +02:00
paul 9c1e79b5a5 Fix gallery issues: user uploads, error messages, and translations
- Fix user upload feature not showing in gallery
  - Add allow_user_uploads and upload_category_id to auth response
  - These fields are required for the gallery to show the upload button

- Improve gallery login error messages
  - Add specific translations for wrong password vs rate limiting
  - Map backend error messages to user-friendly translations
  - Added auth.wrongPassword and auth.tooManyAttempts keys

- Gallery pages already have full translation support
  - GalleryPage and GalleryView use i18n properly
  - All text is translated based on selected language

These fixes resolve:
1. User upload button not showing even when enabled
2. Generic error messages for wrong passwords
3. No feedback for rate limiting

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-09 09:03:14 +02:00
paul 472445a2e5 Fix backend 500 errors for event creation and system version
- Fix system version endpoint to read package.json using fs instead of require
  - Prevents MODULE_NOT_FOUND error in Docker container
  - Uses path.join to find package.json reliably

- Fix event creation email queue error
  - Change email_type from 'creation' to 'gallery_created' to match template key
  - Update email_data to include all required template variables
  - Added missing created_at and updated_at columns to email_queue table

- Fix file watcher duplicate photo insertion
  - Add check to prevent re-inserting existing photos on backend restart

These fixes resolve:
1. 500 error when accessing /api/admin/system/version
2. 500 error when creating new events
3. Email queue processing errors
4. Photo duplication issue

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-09 08:53:18 +02:00
paul ec3d5a0f80 Fix critical issues: gallery authentication and photo duplication
- Fix gallery-specific authentication for images
  - Update AuthenticatedImage component to use gallery-specific tokens
  - Add isGallery prop to distinguish between admin and gallery contexts
  - Update PhotoGrid and PhotoLightbox to pass isGallery prop

- Fix photo duplication issue in fileWatcher service
  - Add check to prevent duplicate photo entries when backend restarts
  - File watcher now verifies if photo exists before inserting
  - Cleaned up 176 duplicate photos from database

These fixes resolve:
1. Gallery images not loading due to auth token errors
2. Photo count increasing without new uploads due to duplicates

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-09 08:28:15 +02:00
paul cf32b01356 fix: Implement gallery-specific authentication tokens
- Fix issue where different galleries shared authentication
- Store gallery tokens with slug-specific keys in localStorage
- Remove global gallery_token cookie approach
- Each gallery now maintains its own authentication state
- Add cleanup for legacy authentication data

This ensures that accessing different galleries requires separate authentication
and prevents cross-gallery authentication leakage.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-09 08:05:04 +02:00
paul c8cfce3e36 fix: Add localized date formatting to gallery views
- Update GalleryLayout to use localized date formatting
- Update GalleryPage to use localized date formatting
- Dates now properly display in German/English based on selected language

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-09 07:11:08 +02:00
paul 12ba91952e feat: Add comprehensive system enhancements
- Add version display above storage consumption in admin sidebar
- Fix storage consumption to stick to bottom of window using flexbox
- Add user upload settings to events (allow uploads, category selection)
- Enhance disk space tab to comprehensive system status view
- Add localized date formatting for German/English language support
- Remove quick actions from dashboard for cleaner interface
- Create user photo upload functionality for galleries
- Add database migration for user upload settings
- Update all TypeScript types and interfaces

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-08 22:47:04 +02:00
paul 69b56ed582 Implement multi-language email templates
- Add language columns to email_templates table (subject_en/de, body_html_en/de, body_text_en/de)
- Update adminEmail.js routes to support language-specific templates
- Create EmailProcessor service to handle language selection based on recipient
- Update EmailConfigPage component with language tabs similar to CMS pages
- Add German translations for all email templates
- Update all email queue usage to use proper template keys
- Add missing email templates (gallery_expired, archive_complete)
- Integrate email processor service into main server startup

The system now automatically selects the appropriate language (English/German) based on the recipient's email domain or preferences.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-08 17:57:26 +02:00
paul 1bc9b547c7 Complete translation implementation for admin interface
- Fixed all hardcoded strings in admin components to use t() function
- Updated BrandingPage.tsx to use translations for watermark settings
- Updated EventsListPage.tsx to use translations for status labels
- Added missing translation keys to both en.json and de.json
- Fixed translations for:
  - System settings (general, storage, categories tabs)
  - Branding page (watermark settings, positions, opacity)
  - Email configuration and templates
  - Event list view (status labels, filters, actions)
  - Event detail view (all sections properly translated)
- Added comprehensive German translations for all new keys
- Ensured consistent translation usage across all admin pages

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-08 17:41:02 +02:00
paul d594d00227 Fix brand theme application and add comprehensive translations
- Fixed theme not being reflected on gallery and admin login pages
- Created GlobalThemeProvider to apply themes globally
- Updated gallery and admin login pages to use dynamic CSS variables
- Added complete translations for all admin sections in English and German:
  - Notifications management
  - Event view and creation
  - Photo upload functionality
  - Category management
  - Archive page view
  - Analytics dashboard
  - Branding and theme settings
  - System settings
  - CMS page management
  - Email configuration
- Fixed admin photo management display issues
- Fixed photo upload category assignment
- Added password reset functionality for galleries
- Improved error handling and user feedback

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-08 17:07:40 +02:00
paul 2012b0bab9 Fix language setting not being saved to database on admin settings page
- Added default_language field to general settings state in SettingsPage
- Replaced LanguageSelector component with simple select dropdown on settings page
- Fixed public settings endpoint to read general_default_language from database
- Language setting now properly saved when clicking Save Settings button
- Setting is correctly used by gallery login page and legal pages

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-08 09:49:45 +02:00
paul cfa0b0da69 Fix photo upload functionality
- Use proper API instance with authentication headers
- Fix API URL to use backend port (3001) instead of frontend
- Add upload progress tracking
- Add success/error toast notifications
- Show progress bar with percentage during upload
2025-07-07 16:35:55 +02:00
paul eac573c4a5 Fix event-specific themes and branding display
- Update COLOR_THEMES to include full theme configurations
- Send theme as JSON string when creating events
- Display company branding in gallery header
- Add debug logging for theme application
- Event themes now properly override global themes
- Company name and tagline now visible in gallery header
2025-07-07 16:32:46 +02:00
paul ff370f6dbd Fix rate limiting for admin users
- Add skip function to rate limiter that bypasses limits for authenticated admins
- Verify JWT token to identify admin users
- Admin users can now make unlimited API requests
- Other users still subject to rate limits (100 requests per 15 minutes)
2025-07-07 16:26:57 +02:00
paul 7e3009cedc Fix theme preset switching issue
- Fix preset theme selection in BrandingPage
- Add debug logging to track theme changes
- Properly update currentTheme state when preset is selected
- Match saved theme to preset on initialization
- Remove unused imports
2025-07-07 16:15:21 +02:00
paul 23ec674e05 Integrate branding and theme settings with database
- Update BrandingPage to save settings to database instead of localStorage
- Add public settings endpoint for galleries to fetch branding/theme
- Update GalleryView to apply branding settings in footer
- Apply theme settings from database to gallery pages
- Support event-specific themes that override global settings
- Ensure watermark and all branding settings are stored in database
2025-07-07 15:59:48 +02:00
paul 971397c338 Fix general settings route and req.user references
- Update frontend settings service to use correct /api/admin/settings/general route
- Fix all req.user to req.admin references in adminSettings.js
- Ensures settings can be saved without authentication errors
2025-07-07 15:32:42 +02:00
paul f0768cd31b Fix email queue column error in event creation
- Remove created_at field from email_queue insert (table uses scheduled_at)
- Let scheduled_at use its default value from database schema

This fixes the 500 error that occurred when creating events due to
trying to insert a non-existent column.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-07 14:28:01 +02:00
paul 91601c77a4 Fix React error #130 with comprehensive improvements
Frontend improvements:
- Add enhanced error logging in ErrorBoundary for better debugging
- Add validation for EventDetailsPage ID parameter
- Add delay in CreateEventPage navigation to prevent race conditions
- Fallback to events list if navigation data is invalid
- Add displayName to all critical page components

These changes address the React error #130 by:
1. Preventing navigation to undefined routes
2. Validating component parameters before rendering
3. Adding proper error boundaries with detailed logging
4. Ensuring components are properly mounted before navigation

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-07 14:21:25 +02:00
paul 53704ec92e Fix event creation errors
- Fix req.user undefined error by using req.admin from middleware
- Return proper ISO date strings in event creation response
- Include created_at in response for frontend navigation

This fixes the 500 error when creating events and ensures the frontend
receives the expected data format for navigation.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-07 14:17:14 +02:00
paul fece843505 Fix date format issues in events API
- Convert Unix timestamps to ISO strings before sending to frontend
- Store dates as ISO strings in database during event creation
- Fix created_at, expires_at, and archived_at date conversions

This resolves the "Invalid time value" error that occurred when viewing
the events page due to SQLite returning dates as Unix timestamps.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-07 14:14:05 +02:00
paul 35681d5346 Fix date parsing errors in EventsListPage
- Add null checks for all date fields before parsing
- Handle cases where created_at, event_date, or expires_at might be null/undefined
- Prevent "Invalid time value" errors when viewing events list
- Sort function now handles null dates gracefully

This fixes the RangeError that occurred when navigating to the events page
after creating an event.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-07 13:58:33 +02:00
paul 193cadef27 Fix React error #130 and backend event creation
Frontend fixes:
- Disable verbatimModuleSyntax in TypeScript config to fix module imports
- Add displayName to critical React components for better production debugging
- Configure Vite build with manual chunks for better code splitting
- Enable sourcemaps for production debugging

Backend fixes:
- Remove updated_at field from events table insert (column doesn't exist)
- Fix SQL error that was causing 500 errors on event creation

These changes resolve:
- React error #130 that occurred during login and event creation
- 500 Internal Server Error when creating new events
- Better error tracking in production builds

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-07 13:30:53 +02:00
paul 2e7cba9e8a Fix storage path issues and React error #130
Backend fixes:
- Add STORAGE_PATH environment variable support
- Fix absolute path references in all backend services
- Update Docker configuration with correct storage path

Frontend fixes:
- Remove individual ErrorBoundary wrappers to fix React error #130
- Remove unused ErrorBoundary import
- Simplify route structure to prevent component mounting issues

This resolves:
- 500 errors when creating events due to storage permission issues
- React error #130 that occurred during event creation
- Consistent storage path handling across all services

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-07 13:20:40 +02:00
paul 2e10374e2c Fix React error #130 - Remove authentication race condition
- Remove setTimeout delays in AdminAuthContext login function
- Make authentication state updates synchronous
- Replace setTimeout navigation with state-based navigation in AdminLoginPage
- Add proper error handling and component lifecycle management in CreateEventPage
- Prevent navigation if component unmounts during async operations

This fixes the issue where users would see React error #130 during login
and couldn't create events or save settings.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-07 13:12:39 +02:00
paul 9dd643338b Fix React error #130 - Resolve component import issues
- Fixed wildcard exports in common/index.ts with explicit named exports
- Fixed GalleryView import path in GalleryPage.tsx
- Fixed ToastContainer props formatting in App.tsx
- Ensured all components are properly exported and imported

This should finally resolve the invalid element type error.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-07 12:56:09 +02:00
paul f75ee680a5 Fix React error #130 - Invalid element type
- Fixed malformed JSX structure in App.tsx Routes configuration
- Added missing service exports in services/index.ts
- Added ErrorBoundary wrappers to all admin routes for consistency
- Fixed indentation and nesting issues in route definitions

This resolves the login and event creation errors.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-07 11:19:59 +02:00
paul 8dad933ff1 Fix React error #130 by correcting Card component usage
- Fixed all Card components to use padding prop instead of className
- Updated padding values: p-4 -> sm, p-6 -> md, p-8 -> lg
- This resolves the React element type invalid error

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-07 11:00:15 +02:00
paul 225d017718 Fix multiple production issues and add password change functionality
- Fixed frontend API URL configuration to use correct port 3002
- Fixed create event functionality by adding proper endpoint and fixing JSON parsing
- Fixed email settings save functionality by importing logActivity correctly
- Fixed admin settings save functionality by using api client instead of direct fetch
- Implemented password change functionality with modal and backend endpoint
- Added updated_at column to admin_users table
- Fixed all mock data issues - now using real backend data throughout

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-07 10:25:38 +02:00
paul f38a8ef598 Fix admin interface issues and 404 errors
- Create adminEvents.js router to handle /api/admin/events endpoints
- Mount events router in admin.js to fix 404 errors
- Fix admin layout CSS - changed from static to flex layout
- Update AdminSidebar positioning from static to relative
- Add missing PUT endpoints for general and security settings
- Fix frontend environment variables in docker-compose.local.yml
- Add build args to Dockerfile.dev for environment variables
- Update CORS to accept requests from all dev servers
- Remove unused imports from SettingsPage

This fixes:
- Events page 404 error
- Admin layout misalignment (sidebar and content on different rows)
- Settings page not loading
- CORS issues between frontend and backend

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-07 08:40:26 +02:00
paul 024c8eac2d Complete Settings page and fix TypeScript issues
- Create comprehensive SettingsPage with General, Storage, and Security tabs
- Add formatBytes method to settings service
- Update AdminSidebar to show real storage usage from backend
- Fix TypeScript errors with react-query v5 (isPending instead of isLoading)
- Remove unused imports and fix type imports
- Add Settings route to App.tsx
- Implement real-time storage monitoring in sidebar

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-06 22:54:17 +02:00
paul 932e5e137c Replace all mock data with real backend integration
- Add database tables for email configs, settings, and activity logs
- Create backend endpoints for dashboard stats, analytics, archives, email config, and settings
- Create frontend service layer (admin, archive, email, settings services)
- Update AdminDashboard to use real statistics and activity data
- Update AnalyticsPage to fetch real analytics from backend
- Update ArchivesPage with pagination and real archive operations
- Update EmailConfigPage to manage real SMTP config and templates
- Remove all mock data and replace with API calls throughout admin interface

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-06 22:46:24 +02:00
paul 3470120a0d Fix admin login and CORS issues
- Update CORS configuration to allow frontend on port 3005
- Fix auth service to map email field to username for backend compatibility
- Add loading state handling in AdminLayout
- Add error boundary to dashboard route
- Fix unused parameter warning in login function

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-06 22:23:14 +02:00
paul 28632e8970 Implement complete frontend with admin panel and theme system
- Add admin authentication and dashboard
- Create event management pages (list, create, edit, archive)
- Implement gallery enhancements (search, sorting, bulk download)
- Add email configuration and archive management pages
- Integrate Umami analytics with tracking throughout the app
- Add comprehensive error boundaries and loading states
- Implement accessibility features (WCAG 2.1 AA compliance)
- Create theme system with preset themes and customization
- Add branding settings and company information management
- Fix backend database initialization and health check
- Configure proper API URLs and environment variables

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-06 22:04:45 +02:00
paul 6c82958c79 Add complete frontend implementation and Docker deployment setup
- Implement React frontend with TypeScript and Tailwind CSS
- Add scrappbook.de-inspired UI design with photo galleries
- Implement authentication, photo viewing, and download features
- Add Docker Swarm configuration with Traefik reverse proxy
- Set up Drone CI/CD pipeline for automated deployments
- Add monitoring stack with Prometheus and Grafana
- Create comprehensive deployment documentation
- Add simple local development setup with docker-compose.local.yml

Features:
- Password-protected galleries with expiration warnings
- Responsive photo grid with lightbox viewer
- Bulk download functionality
- Hot reload development environment
- Email testing with Mailhog
- Production-ready deployment scripts

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-06 20:23:13 +02:00
paul 032bbae50d Add complete setup script to create all remaining files 2025-07-03 16:39:49 +02:00
paul d66ff29b3e Add email service with queue processing 2025-07-03 16:37:37 +02:00
paul 0ea3ee837a Add archive service for automatic ZIP creation 2025-07-03 16:37:14 +02:00
paul 7de326c296 Add expiration checker service 2025-07-03 16:36:49 +02:00
paul fa71cad843 Add complete setup guide 2025-07-03 16:36:21 +02:00
paul ae71834aa6 Add setup script for remaining files 2025-07-03 16:35:28 +02:00
paul 128452f580 Add file watcher service 2025-07-03 16:34:19 +02:00
paul f306a2539d Add gallery routes 2025-07-03 16:33:49 +02:00
paul 160f26f104 Add events routes 2025-07-03 16:33:09 +02:00
paul 01c37098d4 Add authentication routes 2025-07-03 16:32:28 +02:00
paul f78142cda4 Add photo authentication middleware 2025-07-03 16:31:56 +02:00
paul 6c84f701ca Add auth middleware 2025-07-03 16:31:29 +02:00
paul 1802ddaebd Add database configuration 2025-07-03 16:31:05 +02:00
paul d1d48fb3da Add PM2 ecosystem configuration 2025-07-03 16:30:35 +02:00
paul 8ba72e8aa1 Add Jest setup file 2025-07-03 16:30:20 +02:00
paul feea04b1ce Add Jest configuration 2025-07-03 16:30:03 +02:00
paul c2e1d30153 Add backend ESLint configuration 2025-07-03 16:29:50 +02:00
paul 993a132d20 Add backend .dockerignore 2025-07-03 16:29:36 +02:00
paul 1c374a2f82 Add backend .env.example 2025-07-03 16:29:22 +02:00
paul 576e7c5d35 Add backend Dockerfile 2025-07-03 16:29:04 +02:00
paul 07b93636eb Add backend server.js 2025-07-03 16:28:48 +02:00
paul a2fca63d3a Add backend package.json 2025-07-03 16:28:24 +02:00
paul ebf1dabbaa Add production docker-compose configuration 2025-07-03 16:28:05 +02:00
paul 3647855163 Add development docker-compose configuration 2025-07-03 16:27:40 +02:00
paul bdc7e73523 Add environment variables example file 2025-07-03 16:27:23 +02:00
paul f3b83829ca Add .dockerignore file 2025-07-03 16:27:04 +02:00
paul 59b1b87cba Add .gitignore file 2025-07-03 16:26:50 +02:00
paul 206539f51e Initial commit 2025-07-03 16:25:53 +02:00
147 changed files with 1575 additions and 26989 deletions
-37
View File
@@ -72,43 +72,6 @@ steps:
context: frontend/
registry: registry.local.nothaft.cloud
# -------- NEW: Publish Docker images to GitHub Container Registry --------
- name: push-backend-ghcr
image: plugins/docker
settings:
repo: ghcr.io/the-luap/picpeak-backend
tags:
- ${DRONE_TAG}
- latest
dockerfile: backend/Dockerfile
context: backend/
registry: ghcr.io
username:
from_secret: GITHUB_USERNAME
password:
from_secret: GITHUB_TOKEN
build_args:
- VERSION=${DRONE_TAG}
- name: push-frontend-ghcr
image: plugins/docker
settings:
repo: ghcr.io/the-luap/picpeak-frontend
tags:
- ${DRONE_TAG}
- latest
dockerfile: frontend/Dockerfile
context: frontend/
registry: ghcr.io
username:
from_secret: GITHUB_USERNAME
password:
from_secret: GITHUB_TOKEN
build_args:
- VERSION=${DRONE_TAG}
- VITE_API_URL=${VITE_API_URL:-/api}
trigger:
event:
- tag
+18 -31
View File
@@ -1,24 +1,26 @@
# PicPeak Development Environment Configuration
# Copy this file to .env for local development
# Environment Configuration Template
# Copy this file to .env and adjust values for your environment
# SECURITY WARNING: This configuration is for development only!
# For production, use .env.production.example
# Development: Use docker-compose.dev.yml
# Production: Use docker-compose.prod.yml with .env.production.example
# JWT Secret (Change in production!)
# Generate secure secret with: openssl rand -base64 32
JWT_SECRET=dev-secret-DO-NOT-USE-IN-PRODUCTION
# JWT Secret (CRITICAL for production)
# Generate with: openssl rand -base64 32
JWT_SECRET=dev-secret-change-in-production
# Application URLs (Docker Compose development setup)
# Application URLs
ADMIN_URL=http://localhost:3005
FRONTEND_URL=http://localhost:3005
BACKEND_URL=http://localhost:3001
# Database Configuration (SQLite for development)
# Database Configuration
# SQLite is used for development by default
# For production PostgreSQL config, see .env.production.example
DATABASE_CLIENT=sqlite3
DATABASE_PATH=./data/photo_sharing.db
# Email Configuration (Mailhog for development)
# Access Mailhog UI at: http://localhost:8025
# Email Configuration
# Development: Uses Mailhog (included in docker-compose.dev.yml)
# Production: Configure real SMTP server
SMTP_HOST=mailhog
SMTP_PORT=1025
SMTP_SECURE=false
@@ -26,22 +28,7 @@ SMTP_USER=
SMTP_PASS=
EMAIL_FROM=noreply@localhost
# Backend Port Configuration
PORT=3001
# Optional: Umami Analytics Backend Config
# NOTE: Primary configuration through Admin UI > Settings > Analytics
# These are fallback values for server-side tracking
# UMAMI_URL=https://analytics.example.com
# UMAMI_WEBSITE_ID=your-website-id
# UMAMI_HASH_SALT=your-hash-salt
# Development Features
NODE_ENV=development
LOG_LEVEL=debug
# Admin Setup Notes:
# 1. Run 'npm run migrate' in backend folder
# 2. Admin credentials will be auto-generated
# 3. Check ADMIN_CREDENTIALS.txt for login details
# 4. Change password on first login (required)
# Optional: Umami Analytics
UMAMI_URL=
UMAMI_WEBSITE_ID=
UMAMI_HASH_SALT=
+28 -84
View File
@@ -1,100 +1,44 @@
# PicPeak Production Configuration
# Copy this file to .env and update with your production values
# Copy this file to .env and update with your values
# ============================================
# CRITICAL SECURITY - MUST CHANGE ALL VALUES!
# ============================================
# Required: Security
JWT_SECRET=CHANGE_THIS_TO_RANDOM_32_CHAR_STRING
# JWT Secret - REQUIRED (minimum 32 characters)
# Generate with: openssl rand -base64 32
JWT_SECRET=CHANGE-THIS-PRODUCTION-SECRET-USE-OPENSSL-COMMAND
# Application URLs - REQUIRED (your actual domain)
# Required: URLs (update with your domain)
FRONTEND_URL=https://your-domain.com
BACKEND_URL=https://your-domain.com
ADMIN_URL=https://your-domain.com
# ============================================
# DATABASE CONFIGURATION - REQUIRED
# ============================================
# Required: Email Settings
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_USER=your-email@gmail.com
SMTP_PASS=your-app-password
SMTP_FROM=your-email@gmail.com
# PostgreSQL Configuration (Recommended for production)
# Required: Initial Admin Account
ADMIN_EMAIL=admin@your-domain.com
ADMIN_PASSWORD=change-this-password
# Database (PostgreSQL recommended for production)
DATABASE_CLIENT=pg
DB_HOST=postgres # or your database host
DB_HOST=postgres
DB_PORT=5432
DB_NAME=picpeak
DB_USER=picpeak
DB_PASSWORD=CHANGE-THIS-SECURE-DATABASE-PASSWORD
DB_PASSWORD=secure-database-password
# ============================================
# EMAIL CONFIGURATION - REQUIRED
# ============================================
# Example: Gmail with App Password
# SMTP_HOST=smtp.gmail.com
# SMTP_PORT=587
# SMTP_SECURE=false
# SMTP_USER=your-email@gmail.com
# SMTP_PASS=your-16-char-app-password
# EMAIL_FROM=Your Name <your-email@gmail.com>
# Example: SendGrid
SMTP_HOST=smtp.sendgrid.net
SMTP_PORT=587
SMTP_SECURE=false
SMTP_USER=apikey
SMTP_PASS=YOUR-SENDGRID-API-KEY
EMAIL_FROM=PicPeak <noreply@your-domain.com>
# ============================================
# ADMIN SETUP - AUTO-GENERATED
# ============================================
# NOTE: Admin credentials are automatically generated during setup
# DO NOT set ADMIN_EMAIL or ADMIN_PASSWORD anymore!
# Run 'npm run migrate' and check ADMIN_CREDENTIALS.txt
# ============================================
# OPTIONAL CONFIGURATION
# ============================================
# Umami Analytics (Optional - Fallback values)
# Primary config via Admin UI > Settings > Analytics
# UMAMI_URL=https://analytics.your-domain.com
# UMAMI_WEBSITE_ID=your-website-id
# UMAMI_HASH_SALT=your-hash-salt
# Frontend Analytics (Optional - Fallback values)
# VITE_UMAMI_URL=https://analytics.your-domain.com
# VITE_UMAMI_WEBSITE_ID=your-website-id
# VITE_UMAMI_SHARE_URL=https://analytics.your-domain.com/share/xyz/gallery
# ============================================
# PERFORMANCE & SECURITY TUNING
# ============================================
NODE_ENV=production
PORT=3001
LOG_LEVEL=info
# Security Settings (Defaults are secure)
BCRYPT_ROUNDS=12
# Optional: Customization
SITE_NAME=PicPeak
DEFAULT_EXPIRATION_DAYS=30
SESSION_TIMEOUT_MINUTES=60
RATE_LIMIT_WINDOW_MS=900000 # 15 minutes
RATE_LIMIT_MAX_REQUESTS=100 # per window
# Connection Pool (Adjust based on load)
DB_POOL_MIN=5
DB_POOL_MAX=25
# Optional: Analytics (Umami)
VITE_UMAMI_URL=
VITE_UMAMI_WEBSITE_ID=
# ============================================
# DOCKER COMPOSE SPECIFIC
# ============================================
# Traefik Configuration (if using Traefik)
DOMAIN=your-domain.com
LETSENCRYPT_EMAIL=admin@your-domain.com
# Volume Paths (Docker)
STORAGE_PATH=/app/storage
EVENTS_PATH=/app/storage/events
ARCHIVE_PATH=/app/storage/events/archived
# Advanced: Performance Tuning
NODE_ENV=production
BCRYPT_ROUNDS=12
RATE_LIMIT_WINDOW_MS=900000
RATE_LIMIT_MAX_REQUESTS=100
+44 -44
View File
@@ -13,54 +13,55 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v3
with:
fetch-depth: 0 # Full history for proper mirroring
fetch-depth: 0 # Full history needed for mirroring
- name: Setup Git
run: |
git config --global user.name "the-luap"
git config --global user.email "paul-nothaft@hotmail.de"
- name: Remove sensitive files and directories
- name: Debug - Show current branch and status
run: |
echo "Current files before cleanup:"
ls -la | head -10 || true
echo "..."
echo "Current branch:"
git branch -a
echo "Git status:"
git status
echo "Remote info:"
git remote -v
- name: Create filtered branch
run: |
# Clean up any existing github-mirror branch
git branch -D github-mirror || true
# Remove sensitive files/directories if they exist
echo "Removing sensitive files..."
rm -rf .env || true
rm -rf backend/.env* || true
rm -rf frontend/.env* || true
rm -rf docker-compose.prod.yml || true
rm -rf .claudedocs/ || true
rm -rf backend/data/ || true
rm -rf backend/storage/ || true
rm -rf .gitea/ || true
rm -rf scripts/install-gitea-runner.sh || true
rm -rf .drone* || true
rm -rf .github-mirror-exclude || true
rm -rf .gitattributes-github || true
rm -rf photo-sharing-prd.md || true
rm -rf CLAUDE.md || true
rm -rf PRODUCTION_DEPLOYMENT_GUIDE.md || true
rm -rf logs/ || true
rm -rf frontend/.claudedocs/ || true
rm -rf test-maintenance.sh || true
rm -rf storage/ || true
# Create a new branch for GitHub
git checkout --orphan github-mirror
echo "Sensitive files removal completed"
# Add and commit the cleanup if there are changes
git add -A
if ! git diff --cached --quiet; then
git commit -m "chore: remove sensitive files for GitHub mirror"
echo "✅ Committed cleanup of sensitive files"
else
echo "✅ No sensitive files to remove"
fi
echo "Final file structure (top level):"
ls -la | head -10 || true
# Remove sensitive files/directories
# Example: Remove .env files, private configs, etc.
git rm -r --cached .env* || true
git rm -r --cached backend/.env* || true
git rm -r --cached frontend/.env* || true
git rm -r --cached docker-compose.prod.yml || true
git rm -r --cached .claudedocs/ || true
git rm -r --cached backend/data/ || true
git rm -r --cached backend/storage/ || true
git rm -r --cached .gitea/ || true
git rm -r --cached scripts/install-gitea-runner.sh || true
git rm -r --cached .drone* || true
git rm -r --cached .github-mirror-exclude || true
git rm -r --cached .gitattributes-github || true
git rm -r --cached photo-sharing-prd.md || true
git rm -r --cached CLAUDE.md || true
git rm -r --cached PRODUCTION_DEPLOYMENT_GUIDE.md || true
git rm -r --cached logs/ || true
git rm -r --cached frontend/.claudedocs/ || true
git rm -r --cached test-maintenance.sh || true
git rm -r --cached storage/ || true
# Commit the changes
git commit -m "Remove sensitive files for GitHub mirror" || true
- name: Check GitHub token
env:
@@ -87,13 +88,12 @@ jobs:
echo "GitHub remote added:"
git remote -v
# Push to GitHub main branch
# Force push the filtered branch to GitHub main
echo "Pushing to GitHub..."
git push github main --force
echo "Push to GitHub completed!"
git push github github-mirror:main --force
echo "Push completed successfully!"
- name: Workflow completed
run: |
echo "✅ Mirror to GitHub workflow completed successfully!"
echo "📊 Repository mirrored to: https://github.com/the-luap/picpeak"
echo "🔒 Sensitive files have been removed from the mirror"
echo "Check https://github.com/the-luap/picpeak to verify the mirror."
+3 -61
View File
@@ -164,25 +164,11 @@ jobs:
- name: Commit version bump
if: steps.version.outputs.version_changed == 'true'
run: |
set -e # Exit on any error
# First, ensure we have the latest changes
echo "Fetching latest changes..."
git fetch origin main
# Check if we're behind and need to update
LOCAL=$(git rev-parse HEAD)
REMOTE=$(git rev-parse origin/main)
if [ "$LOCAL" != "$REMOTE" ]; then
echo "Local is behind remote, pulling changes..."
git pull origin main --no-rebase
fi
COMPONENT="${{ steps.version.outputs.component_changed }}"
if [ "$COMPONENT" = "both" ]; then
git add backend/package.json backend/package-lock.json frontend/package.json frontend/package-lock.json
git add backend/package.json backend/package-lock.json
git add frontend/package.json frontend/package-lock.json
git commit -m "chore: bump version to ${{ steps.version.outputs.new_version }} (backend + frontend)"
elif [ "$COMPONENT" = "backend" ]; then
git add backend/package.json backend/package-lock.json
@@ -192,51 +178,7 @@ jobs:
git commit -m "chore: bump frontend version to ${{ steps.version.outputs.new_version }}"
fi
# Pull latest changes before pushing to avoid conflicts
echo "Pulling latest changes from origin/main..."
if ! git pull --rebase origin main; then
echo "Rebase failed, attempting to resolve..."
# If rebase fails, abort and try a regular merge
git rebase --abort || true
git pull origin main --no-rebase
fi
# Push the changes with retry logic
echo "Pushing version bump..."
PUSH_SUCCESS=false
for i in 1 2 3; do
echo "Push attempt $i of 3..."
# Try to push
if git push origin main 2>&1; then
echo "Successfully pushed version bump on attempt $i"
PUSH_SUCCESS=true
break
else
echo "Push failed on attempt $i"
if [ $i -lt 3 ]; then
echo "Waiting 5 seconds before retry..."
sleep 5
echo "Pulling latest changes..."
git fetch origin main
# Try rebase first, fall back to merge
if ! git rebase origin/main; then
echo "Rebase failed, trying merge..."
git rebase --abort 2>/dev/null || true
git pull origin main --no-rebase
fi
fi
fi
done
if [ "$PUSH_SUCCESS" = "false" ]; then
echo "ERROR: Failed to push after 3 attempts"
exit 1
fi
git push
- name: Create Git tag
if: steps.version.outputs.version_changed == 'true'
-6
View File
@@ -48,15 +48,9 @@ coverage/
*.tmp
*.temp
# Backup and test directories
backups/
test-archiver/
# Keep directory structure
!storage/events/active/.gitkeep
!storage/events/archived/.gitkeep
!storage/thumbnails/.gitkeep
!data/.gitkeep
!logs/.gitkeep
PRODUCTION_DEPLOYMENT_GUIDE.md
-152
View File
@@ -1,152 +0,0 @@
# Backup Version Tracking Implementation
## Overview
Version tracking has been added to the backup system to ensure safe restoration by tracking application versions, Node.js versions, and database schema versions at the time of backup.
## Implementation Details
### 1. Database Schema Changes (Migration 034)
Added version tracking columns to backup tables:
#### `database_backup_runs` table:
- `app_version` - Application version from package.json
- `node_version` - Node.js runtime version
- `db_schema_version` - Latest migration name
- `environment_info` - JSON with additional environment details
#### `backup_runs` table:
- `app_version` - Application version
- `node_version` - Node.js version
- `db_schema_version` - Database schema version
- `manifest_info` - Summary of manifest information
#### New `restore_history` table:
Tracks all restore attempts with comprehensive version information:
- Backup versions vs current versions
- Compatibility check results
- Warnings and errors
- Restore outcome
### 2. Version Information Captured
During each backup, the system now records:
- **Application Version**: From `package.json` (e.g., "1.0.77")
- **Node.js Version**: Runtime version (e.g., "v18.17.0")
- **Database Schema**: Latest migration file (e.g., "034_add_version_to_backups.js")
- **Environment Info**: Platform, architecture, environment mode
### 3. Backup Services Updated
#### Database Backup Service (`databaseBackup.js`):
- Records version info when creating backups
- Includes versions in statistics JSON
- New method: `checkVersionCompatibility()` for restore safety
- New method: `getCurrentSchemaVersion()` to track migrations
#### File Backup Service (`backupService.js`):
- Records version info in backup_runs table
- Integrates with manifest system
- Stores manifest summary with version details
### 4. Existing Manifest System
The `backupManifest.js` already provides comprehensive version tracking:
- Application version and Node.js version
- System information (OS, platform, architecture)
- Database schema version
- Detailed file and database metadata
### 5. Version Compatibility Checking
When restoring, the system can now:
- Compare backup version vs current version
- Detect major/minor version differences
- Identify schema mismatches
- Provide warnings and recommendations
### 6. Configuration Settings
New backup settings for version control:
- `backup_require_version_match` - Enforce exact version matching
- `backup_allow_minor_version_mismatch` - Allow same major version
- `backup_warn_on_version_mismatch` - Show warnings on mismatch
- `backup_check_schema_compatibility` - Validate schema versions
## Usage
### Creating Backups
Backups automatically capture version information - no changes needed to existing backup workflows.
### Checking Version Before Restore
1. **For Database Backups**:
```javascript
const compatibility = await databaseBackupService.checkVersionCompatibility({
app_version: '1.0.75',
node_version: 'v16.14.0',
db_schema_version: '032_add_feedback.js'
});
if (!compatibility.compatible) {
console.error('Version mismatch:', compatibility.errors);
}
```
2. **For File Backups**:
Check the manifest file which contains all version information:
```bash
cat /backup/path/manifest-backup-20250122-123456.json | jq '.application'
```
### Restore History
All restore attempts are logged in the `restore_history` table with:
- Version compatibility results
- Warnings encountered
- Success/failure status
- Who performed the restore
## Best Practices
1. **Always Check Compatibility**: Before restoring, verify version compatibility
2. **Document Version Changes**: Keep changelog updated with breaking changes
3. **Test Restores**: Regularly test restore procedures in staging
4. **Monitor Warnings**: Even if compatible, review warnings before proceeding
5. **Keep Backups Organized**: Label backups with version info in filename
## Migration Instructions
1. Run the new migration:
```bash
cd backend
npm run migrate
```
2. Existing backups will show "unknown" for version fields
3. New backups will automatically include version information
4. The system remains backward compatible with old backups
## Troubleshooting
### Version Mismatch Errors
- Check current app version: `cat backend/package.json | grep version`
- Check Node version: `node --version`
- Check latest migration: `SELECT name FROM knex_migrations ORDER BY id DESC LIMIT 1`
### Restore Failures
- Review `restore_history` table for detailed error messages
- Check version compatibility warnings
- Consider using same version environment for critical restores
## Future Enhancements
1. **Automated Version Matching**: Docker containers with specific versions
2. **Migration Rollback**: Support for downgrading schema safely
3. **Version Matrix**: Compatibility matrix for different version combinations
4. **Restore Wizard**: UI for guided restore with compatibility checks
---
**Implementation Date**: January 2025
**Current Version**: 1.0.77
**Status**: Production Ready
+1 -105
View File
@@ -39,12 +39,6 @@ docker-compose -f docker-compose.prod.yml up -d # Production deployment
pm2 start ecosystem.config.js # Alternative: PM2 deployment
```
**⚠️ CRITICAL PRODUCTION NOTICE:**
- Production runs on a SEPARATE SERVER - never assume local changes affect production
- ALWAYS request production server details before any troubleshooting
- NO trial-and-error approaches in production - data loss is unacceptable
- Every change must be thoroughly analyzed and tested locally first
## Key Product Requirements (from PRD)
### Core Features
@@ -117,7 +111,6 @@ Background services run as separate processes:
- **archiveService**: Creates ZIP archives of expired events
- **expirationChecker**: Cron job for expiration warnings
- **fileWatcher**: Monitors for new photo uploads
- **backupService**: Scheduled backups with checksum-based change detection
### API Structure
- `/api/admin/*` - Admin panel endpoints (requires adminAuth)
@@ -134,41 +127,6 @@ Background services run as separate processes:
5. **Frontend Status**: Only skeleton exists - requires full implementation based on PRD
6. **Umami Analytics**: Track password entries, downloads, views, expiration warnings
## Troubleshooting Guidelines
### Before ANY Production Troubleshooting:
1. **ALWAYS request specific details**:
- Production server URL/IP
- Current error messages/logs
- Recent changes or deployments
- Affected users/galleries
- Time of issue occurrence
2. **Thorough Analysis Required**:
- Use detailed thinking/analysis for EVERY troubleshooting task
- Review all related code before suggesting changes
- Consider all potential side effects
- Never make assumptions about production environment
3. **Safe Troubleshooting Steps**:
- First, reproduce issue in local/dev environment
- Analyze logs without modifying production
- Create detailed action plan before any changes
- Always have rollback strategy ready
- Document every step taken
### Common Issues & Safe Approaches:
- **Email not sending**: Check email_queue table, SMTP settings, service status
- **Photos not loading**: Verify file permissions, storage paths, nginx config
- **Gallery access issues**: Check JWT tokens, expiration dates, access_logs
- **Performance problems**: Analyze with monitoring tools first, never experiment
### Data Safety Rules:
- NEVER delete or modify production data without explicit backup confirmation
- ALWAYS verify backups exist before any data operations
- NO direct database modifications without transaction safety
- Log all actions for audit trail
## Environment Variables
### Backend (.env)
@@ -295,71 +253,9 @@ const { theme, setTheme, setThemeByName } = useTheme();
--border-radius: 0.5rem;
```
## Backup Service
### Overview
The backup service provides automated, scheduled backups of all photo data with checksum-based change detection to minimize transfer overhead.
### Features
- **Multiple Destinations**: Local directory, remote server (rsync), S3-compatible storage
- **Change Detection**: SHA256 checksums track file changes, only modified files are backed up
- **Scheduled Execution**: Configurable cron-based scheduling (default: 2 AM daily)
- **Email Notifications**: Alerts on backup failure, optional success notifications
- **Retention Management**: Automatic cleanup of old backup runs based on retention policy
- **Progress Tracking**: Database storage of backup history, file states, and statistics
### Configuration
Backup settings are stored in `app_settings` table with `backup_` prefix:
- `backup_enabled`: Enable/disable the service
- `backup_schedule`: Cron expression (e.g., '0 2 * * *')
- `backup_destination_type`: 'local', 'rsync', or 's3'
- `backup_retention_days`: How long to keep backup history
- `backup_include_archived`: Whether to backup archived events
- `backup_exclude_patterns`: File patterns to exclude
### API Endpoints
- `GET /api/admin/backup/config` - Get current configuration
- `PUT /api/admin/backup/config` - Update configuration
- `GET /api/admin/backup/status` - Get backup status and history
- `POST /api/admin/backup/run` - Trigger manual backup
- `POST /api/admin/backup/test-connection` - Test destination connectivity
### Testing
Run backup service test: `npm run test-backup`
### Database Tables
- `backup_runs`: Tracks each backup execution with statistics
- `backup_file_states`: Stores file checksums for change detection
## Success Metrics (from PRD)
- Time to generate gallery: <2 minutes
- Guest satisfaction: >90%
- System uptime: 99.9%
- Email delivery rate: >98%
- Successful archiving: 100%
## Documentation & Development Practices
### Documentation Guidelines:
- **NEVER create new documentation files for simple tasks**
- **ALWAYS update existing documentation (like this CLAUDE.md)**
- Only create new .md files when explicitly requested
- Avoid creating temporary scripts for one-off tasks
### Development Best Practices:
- Test all changes thoroughly in local environment first
- Use version control for all changes
- Keep commits atomic and well-described
- Review impact on all integrated services
- Consider backward compatibility
- Update tests when changing functionality
### Production Deployment Checklist:
- [ ] All tests passing locally
- [ ] Linting and type checks pass
- [ ] Database migrations tested with rollback plan
- [ ] Environment variables documented
- [ ] Backup strategy confirmed
- [ ] Monitoring alerts configured
- [ ] Rollback procedure documented
- [ ] Stakeholders notified of maintenance window
- Successful archiving: 100%
+98
View File
@@ -0,0 +1,98 @@
# Production Deployment Guide
This guide explains how to deploy PicPeak in production behind a reverse proxy like Traefik.
## Environment Configuration
### Frontend Configuration
For production deployment behind a reverse proxy (Traefik, Nginx, etc.), the frontend should use relative URLs to automatically inherit the protocol (HTTPS) and domain.
1. Copy the production environment template:
```bash
cp frontend/.env.production.example frontend/.env.production
```
2. Set the API URL to use relative path:
```env
# frontend/.env.production
VITE_API_URL=/api
```
This ensures all API calls will use the same domain and protocol as the frontend.
### Backend Configuration
Ensure your backend `.env` file has the correct URLs:
```env
# backend/.env
FRONTEND_URL=https://yourdomain.com
ADMIN_URL=https://yourdomain.com
```
## Docker Compose Production
When using Docker Compose in production:
1. Build with production environment:
```bash
docker-compose -f docker-compose.prod.yml build --build-arg NODE_ENV=production
```
2. The frontend nginx configuration already includes proper proxy settings for:
- `/api` → Backend API
- `/photos` → Protected photo access
- `/thumbnails` → Thumbnail images
- `/uploads` → Public uploads (logos, favicons)
## Traefik Configuration
Example Traefik labels for docker-compose:
```yaml
services:
frontend:
labels:
- "traefik.enable=true"
- "traefik.http.routers.picpeak.rule=Host(`yourdomain.com`)"
- "traefik.http.routers.picpeak.entrypoints=websecure"
- "traefik.http.routers.picpeak.tls.certresolver=letsencrypt"
- "traefik.http.services.picpeak.loadbalancer.server.port=80"
```
## Important Notes
1. **No Hardcoded URLs**: The application uses environment variables with relative URL fallbacks, making it production-ready.
2. **HTTPS Only**: When `VITE_API_URL=/api`, all requests will use the same protocol as the page (HTTPS in production).
3. **CORS Configuration**: The backend CORS is configured to accept requests from the URLs specified in `FRONTEND_URL` and `ADMIN_URL`.
4. **Static Assets**: All static assets (photos, thumbnails, uploads) are served through the nginx proxy, inheriting authentication headers.
## Verification
After deployment, verify:
1. Check browser console for any localhost URLs (there should be none)
2. Verify all API calls use HTTPS
3. Check that images load correctly with authentication
4. Test favicon and logo display
## Troubleshooting
If you see console errors about localhost:
1. Ensure `VITE_API_URL=/api` in frontend environment
2. Clear browser cache
3. Rebuild frontend with production environment:
```bash
cd frontend
npm run build
```
If images don't load:
1. Check that nginx proxy locations are configured
2. Verify authentication tokens are being sent
3. Check backend logs for authentication errors
+26 -92
View File
@@ -1,6 +1,6 @@
# Production Deployment Guide
This comprehensive guide addresses all production deployment scenarios and common issues.
This guide addresses all known production deployment issues and provides solutions.
## Pre-Deployment Checklist
@@ -8,80 +8,43 @@ This comprehensive guide addresses all production deployment scenarios and commo
Create a `.env` file with ALL required variables:
```bash
# CRITICAL - Must change these!
# Required
JWT_SECRET=<generate-with-openssl-rand-base64-32>
DB_PASSWORD=<strong-password>
# Application URLs (your actual domain)
ADMIN_URL=https://yourdomain.com
FRONTEND_URL=https://yourdomain.com
BACKEND_URL=https://yourdomain.com
# Database (PostgreSQL)
DATABASE_CLIENT=pg
DB_HOST=postgres # or external host
DB_PORT=5432
# Database
DB_USER=picpeak
DB_NAME=picpeak
# Email Configuration (required for notifications)
# Email (Optional but recommended)
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_SECURE=false
SMTP_USER=your-email@gmail.com
SMTP_PASS=your-app-password # Use app-specific password
EMAIL_FROM=PicPeak <noreply@yourdomain.com>
SMTP_PASS=your-app-password
EMAIL_FROM=noreply@yourdomain.com
# Port Configuration
PORT=3001
# Performance Tuning
DB_POOL_MIN=5
DB_POOL_MAX=25
NODE_ENV=production
LOG_LEVEL=info
# Optional: Umami Analytics (configured via Admin UI)
# UMAMI_URL=https://analytics.yourdomain.com
# UMAMI_WEBSITE_ID=your-website-id
# Umami Analytics (Optional)
UMAMI_URL=https://analytics.yourdomain.com
UMAMI_WEBSITE_ID=your-website-id
UMAMI_HASH_SALT=<generate-random-string>
```
### 2. Generate Secrets
```bash
# Generate JWT Secret (REQUIRED)
# Generate JWT Secret
openssl rand -base64 32
# Generate Database Password
openssl rand -base64 24
# Generate Umami Hash Salt
openssl rand -hex 32
```
## Frontend Configuration
For production deployment behind a reverse proxy:
### Frontend Environment
```bash
# frontend/.env.production
VITE_API_URL=/api # Uses relative path for reverse proxy
# Optional: Umami fallback (primary config via Admin UI)
# VITE_UMAMI_URL=https://analytics.yourdomain.com
# VITE_UMAMI_WEBSITE_ID=your-website-id
```
This ensures all API calls use the same domain/protocol as the frontend.
### Nginx Proxy Configuration
The frontend nginx configuration already includes proper proxy settings for:
- `/api` → Backend API
- `/photos` → Protected photo access
- `/thumbnails` → Thumbnail images
- `/uploads` → Public uploads (logos, favicons)
All static assets are served through the nginx proxy, inheriting authentication headers.
## Deployment Steps
### 1. Initial Setup
@@ -133,31 +96,24 @@ docker-compose -f docker-compose.prod.yml up -d
docker-compose -f docker-compose.prod.yml logs -f backend
```
### 4. Initial Admin Setup
### 4. Create Admin User
The admin user is automatically created during database migration:
After deployment, create the first admin user:
```bash
# Run migrations (this creates admin user)
docker-compose -f docker-compose.prod.yml exec backend npm run migrate
# Enter backend container
docker-compose -f docker-compose.prod.yml exec backend sh
# Admin credentials will be displayed in console and saved to ADMIN_CREDENTIALS.txt
# Example output:
# ========================================
# ✅ Admin user created successfully!
# ========================================
# Username: admin
# Password: SwiftEagle3847!
#
# ⚠️ IMPORTANT: Change password on first login
# ========================================
# Create admin
node scripts/create-admin.js \
--username admin \
--email admin@yourdomain.com \
--password <your-secure-password>
# Retrieve credentials if needed
docker-compose -f docker-compose.prod.yml exec backend cat ADMIN_CREDENTIALS.txt
# Exit container
exit
```
**Important**: You MUST change the auto-generated password on first login.
### 5. Configure Email (if using database config)
1. Login to admin panel: https://yourdomain.com/admin
@@ -233,23 +189,6 @@ docker-compose -f docker-compose.prod.yml logs backend | grep email
## SSL/HTTPS Setup
### Option 1: Using Traefik (Recommended)
Add these labels to your docker-compose override:
```yaml
services:
frontend:
labels:
- "traefik.enable=true"
- "traefik.http.routers.picpeak.rule=Host(`yourdomain.com`)"
- "traefik.http.routers.picpeak.entrypoints=websecure"
- "traefik.http.routers.picpeak.tls.certresolver=letsencrypt"
- "traefik.http.services.picpeak.loadbalancer.server.port=80"
```
### Option 2: Using Certbot
1. Update `nginx/sites-enabled/default` with your domain
2. Run certbot:
@@ -355,18 +294,13 @@ docker-compose -f docker-compose.prod.yml up -d
- [ ] Strong JWT_SECRET (min 32 chars)
- [ ] Strong database password
- [ ] Admin password changed from auto-generated one
- [ ] SSL/HTTPS enabled
- [ ] Firewall configured (only 80/443 open)
- [ ] Regular security updates
- [ ] Backup encryption
- [ ] Access logs monitored
- [ ] Rate limiting enabled (built-in)
- [ ] Rate limiting enabled
- [ ] File upload restrictions configured
- [ ] Password complexity requirements configured (Admin > Settings)
- [ ] Session timeout configured (default 60 min)
- [ ] Umami analytics configured (if using)
- [ ] SMTP credentials secured with app-specific password
## Support
-28
View File
@@ -94,20 +94,6 @@ Perfect for:
- **Email**: SMTP with customizable templates
- **Analytics**: Privacy-focused with Umami integration
## 💻 System Requirements
### Minimum Requirements
- **CPU**: 2 CPU cores
- **RAM**: 2GB minimum
- **Storage**: 20GB minimum (plus photo storage needs)
- **OS**: Linux (Ubuntu 20.04+), macOS, or Windows with WSL2
- **Node.js**: v18.0.0 or higher
- **Database**: SQLite (included) or PostgreSQL 12+
### Docker Requirements (Recommended)
- **Docker**: v20.10.0+
- **Docker Compose**: v2.0.0+
## 🤝 Contributing
We love contributions! PicPeak is built by photographers, for photographers. Whether you're fixing bugs, adding features, or improving documentation, your help is welcome.
@@ -173,20 +159,6 @@ Organize and manage your photo galleries with intuitive event management tools.
</details>
## 🗺️ Roadmap
We're constantly improving PicPeak and welcome contributions from our community! If you have ideas for new features or want to help implement existing ones, please open an issue or submit a pull request. Your contributions help make PicPeak better for everyone.
| Feature | Description | Priority | Status |
|---------|-------------|----------|---------|
| **Backup & Restore** | Comprehensive backup system with S3/MinIO support, automated scheduling, and safe restore functionality | High | ✅ Implemented |
| **Gallery Templates** | Additional gallery layouts and themes (masonry, slideshow, story-style) for different event types | Medium | 🔄 Open |
| **Face Recognition** | AI-powered face detection to help guests find their photos and create automatic person-based albums | Low | 🔄 Open |
| **Gallery Feedback** | Allow guests to like, rate, and comment on photos with admin notifications and moderation | Medium | 🔄 Open |
| **Video Support** | Upload and display videos alongside photos in galleries with streaming support | Low | 🔄 Open |
**Status Legend:** ✅ Implemented | 🚧 In Progress | 🔄 Open | 📋 Planned
## 🙏 Acknowledgments
PicPeak is inspired by the best features of commercial platforms while remaining completely open source. Special thanks to all contributors who make this project possible.
+16 -33
View File
@@ -3,56 +3,39 @@
# Application
NODE_ENV=production
PORT=3001
PORT=3000
# Security
# Generate with: openssl rand -base64 32
JWT_SECRET=your-very-secure-jwt-secret-at-least-32-characters-long-example123456
JWT_SECRET=your-very-secure-jwt-secret-at-least-32-characters-long
# URLs (adjust for your domain)
ADMIN_URL=https://photos.example.com
FRONTEND_URL=https://photos.example.com
# URLs
ADMIN_URL=https://yourdomain.com
FRONTEND_URL=https://yourdomain.com
# Database Configuration
DATABASE_CLIENT=pg
DB_HOST=localhost
DB_HOST=db
DB_PORT=5432
DB_USER=picpeak
DB_PASSWORD=your-secure-database-password-change-this
DB_PASSWORD=your-secure-database-password
DB_NAME=picpeak
# Email Configuration (Examples for common providers)
# Gmail example:
# SMTP_HOST=smtp.gmail.com
# SMTP_PORT=587
# SMTP_SECURE=false
# SMTP_USER=your-email@gmail.com
# SMTP_PASS=your-app-specific-password
# SendGrid example:
SMTP_HOST=smtp.sendgrid.net
# Email Configuration
SMTP_HOST=smtp.example.com
SMTP_PORT=587
SMTP_SECURE=false
SMTP_USER=apikey
SMTP_PASS=your-sendgrid-api-key
EMAIL_FROM=noreply@example.com
SMTP_USER=your-smtp-username
SMTP_PASS=your-smtp-password
EMAIL_FROM=noreply@yourdomain.com
# Storage Paths
# Docker deployment:
# Storage Paths (Docker)
STORAGE_PATH=/app/storage
EVENTS_PATH=/app/storage/events
ARCHIVE_PATH=/app/storage/events/archived
# Local development:
# STORAGE_PATH=./storage
# EVENTS_PATH=./storage/events
# ARCHIVE_PATH=./storage/events/archived
# Analytics Backend Configuration (OPTIONAL)
# Used for server-side tracking only
# Primary configuration should be done through Admin UI > Settings > Analytics
# UMAMI_URL=https://analytics.example.com
# UMAMI_WEBSITE_ID=b4d3c2a1-5678-90ab-cdef-1234567890ab
# Analytics (Optional)
UMAMI_URL=https://analytics.yourdomain.com
UMAMI_WEBSITE_ID=your-website-id
# Logging
LOG_LEVEL=info
-229
View File
@@ -1,229 +0,0 @@
# Enhanced Backup System Test Suite
This directory contains comprehensive tests for the enhanced backup system with S3 support.
## Test Structure
### Unit Tests
- `services/backupService.enhanced.test.js` - Unit tests for the enhanced backup service
- Configuration management
- S3 backup functionality
- Manifest generation
- Error handling and recovery
- Backward compatibility (local and rsync)
- Service lifecycle management
### Integration Tests
- `integration/backup-s3.test.js` - Integration tests for S3 backups
- Real S3/MinIO connection tests
- Full backup process with actual files
- Incremental backup verification
- Manifest storage and retrieval
- Error recovery scenarios
### Manual Integration Test Script
- `../scripts/test-backup-integration.js` - Comprehensive manual testing script
- Can test against MinIO, AWS S3, or any S3-compatible service
- Tests all backup types (S3, local, rsync)
- Performance testing with large files
- Detailed progress reporting
## Running Tests
### Prerequisites
1. **For Unit Tests**: No special setup required, all dependencies are mocked.
2. **For Integration Tests**: Requires a running S3-compatible service (MinIO recommended)
```bash
# Start MinIO using Docker
docker run -d \
-p 9000:9000 \
-p 9001:9001 \
--name minio-test \
-e MINIO_ROOT_USER=minioadmin \
-e MINIO_ROOT_PASSWORD=minioadmin \
minio/minio server /data --console-address ":9001"
```
3. **Environment Variables** (for integration tests):
```bash
# Optional - defaults work with local MinIO
export TEST_S3_ENDPOINT=http://localhost:9000
export TEST_S3_ACCESS_KEY=minioadmin
export TEST_S3_SECRET_KEY=minioadmin
# Skip S3 tests if no S3 service available
export SKIP_S3_TESTS=true
```
### Running Unit Tests
```bash
# Run all backup service tests
npm test -- __tests__/services/backupService.enhanced.test.js
# Run specific test suite
npm test -- __tests__/services/backupService.enhanced.test.js -t "S3 Backup Functionality"
# Run with coverage
npm test -- --coverage __tests__/services/backupService.enhanced.test.js
```
### Running Integration Tests
```bash
# Ensure MinIO is running first!
# Run S3 integration tests
npm test -- __tests__/integration/backup-s3.test.js
# Run with verbose output
npm test -- __tests__/integration/backup-s3.test.js --verbose
# Skip S3 tests if needed
SKIP_S3_TESTS=true npm test -- __tests__/integration/backup-s3.test.js
```
### Running Manual Integration Tests
```bash
# Test with local MinIO (default)
node scripts/test-backup-integration.js
# Test with AWS S3
node scripts/test-backup-integration.js \
--endpoint https://s3.amazonaws.com \
--access-key YOUR_ACCESS_KEY \
--secret-key YOUR_SECRET_KEY \
--bucket your-test-bucket
# Test local backup
node scripts/test-backup-integration.js --type local
# Test with cleanup after completion
node scripts/test-backup-integration.js --cleanup
# Verbose output
node scripts/test-backup-integration.js --verbose
```
## Test Coverage
The test suite covers:
### Configuration
- ✅ Database configuration retrieval
- ✅ JSON parsing and error handling
- ✅ Configuration validation
- ✅ Required field validation
### S3 Functionality
- ✅ S3 client initialization
- ✅ Connection testing
- ✅ File upload with progress tracking
- ✅ Large file handling (multipart upload)
- ✅ Metadata and custom headers
- ✅ Error handling and retries
### Backup Process
- ✅ Full backup execution
- ✅ Incremental backup (changed files only)
- ✅ File checksum calculation and comparison
- ✅ Database backup inclusion
- ✅ Archive inclusion toggle
- ✅ File size limits
### Manifest Generation
- ✅ Full manifest generation
- ✅ Incremental manifest with parent reference
- ✅ JSON and YAML format support
- ✅ Manifest validation
- ✅ S3 manifest storage and retrieval
- ✅ Checksum verification
### Error Handling
- ✅ S3 connection failures
- ✅ File read errors
- ✅ Individual file failure recovery
- ✅ Retry logic with exponential backoff
- ✅ Email notifications on failure
- ✅ Concurrent backup prevention
### Backward Compatibility
- ✅ Local directory backup
- ✅ Rsync backup
- ✅ Existing manifest format support
### Service Management
- ✅ Cron job scheduling
- ✅ Service start/stop
- ✅ Manual backup triggering
- ✅ Backup history and status
## Mock Setup
The unit tests use comprehensive mocking:
```javascript
// Database mocking
jest.mock('../../src/database/db');
// S3 client mocking
jest.mock('../../src/services/storage/s3Storage');
// File system mocking
const mockFs = require('mock-fs');
// Cron job mocking
jest.mock('node-cron');
```
## CI/CD Integration
To run tests in CI/CD pipeline:
```yaml
# Example GitHub Actions
- name: Run Unit Tests
run: npm test -- __tests__/services/backupService.enhanced.test.js
- name: Start MinIO
run: |
docker run -d \
-p 9000:9000 \
--name minio-test \
-e MINIO_ROOT_USER=minioadmin \
-e MINIO_ROOT_PASSWORD=minioadmin \
minio/minio server /data
- name: Run Integration Tests
run: npm test -- __tests__/integration/backup-s3.test.js
```
## Debugging Tests
```bash
# Run tests in debug mode
node --inspect-brk ./node_modules/.bin/jest __tests__/services/backupService.enhanced.test.js
# Run single test with console output
npm test -- __tests__/services/backupService.enhanced.test.js -t "should perform S3 backup" --verbose
```
## Performance Considerations
- Integration tests create real files and S3 objects
- Each test run creates a unique S3 bucket to avoid conflicts
- Cleanup is automatic but can be disabled for debugging
- Large file tests (10MB+) are included but can be slow
## Adding New Tests
When adding new backup features:
1. Add unit tests to `backupService.enhanced.test.js`
2. Add integration tests to `backup-s3.test.js` if S3-specific
3. Update manual test script for comprehensive testing
4. Ensure mocks are properly configured
5. Document any new environment requirements
@@ -1,506 +0,0 @@
const { describe, it, expect, jest, beforeAll, afterAll, beforeEach, afterEach } = require('@jest/globals');
const { S3Client, CreateBucketCommand, DeleteBucketCommand, ListObjectsV2Command, DeleteObjectsCommand } = require('@aws-sdk/client-s3');
const path = require('path');
const fs = require('fs').promises;
const crypto = require('crypto');
// Load services
const backupService = require('../../src/services/backupService');
const S3StorageAdapter = require('../../src/services/storage/s3Storage');
const { db, initialize: initDb } = require('../../src/database/db');
const logger = require('../../src/utils/logger');
// Test configuration
const TEST_CONFIG = {
endpoint: process.env.TEST_S3_ENDPOINT || 'http://localhost:9000',
accessKeyId: process.env.TEST_S3_ACCESS_KEY || 'minioadmin',
secretAccessKey: process.env.TEST_S3_SECRET_KEY || 'minioadmin',
bucket: 'test-backup-bucket-' + Date.now(),
region: 'us-east-1'
};
describe('S3 Backup Integration Tests', () => {
let s3Client;
let testStoragePath;
let originalEnv;
beforeAll(async () => {
// Skip if no S3 endpoint configured
if (process.env.SKIP_S3_TESTS === 'true') {
console.log('Skipping S3 integration tests (SKIP_S3_TESTS=true)');
return;
}
// Save original environment
originalEnv = { ...process.env };
// Initialize S3 client for test setup
s3Client = new S3Client({
endpoint: TEST_CONFIG.endpoint,
region: TEST_CONFIG.region,
credentials: {
accessKeyId: TEST_CONFIG.accessKeyId,
secretAccessKey: TEST_CONFIG.secretAccessKey
},
forcePathStyle: true
});
// Create test bucket
try {
await s3Client.send(new CreateBucketCommand({ Bucket: TEST_CONFIG.bucket }));
console.log(`Created test bucket: ${TEST_CONFIG.bucket}`);
} catch (error) {
if (error.name !== 'BucketAlreadyOwnedByYou') {
console.error('Failed to create test bucket:', error);
throw error;
}
}
// Initialize database
await initDb();
await db.migrate.latest();
// Create test storage directory
testStoragePath = path.join(__dirname, '../fixtures/test-storage');
await fs.mkdir(testStoragePath, { recursive: true });
process.env.STORAGE_PATH = testStoragePath;
// Set up test data
await setupTestData();
// Mock logger to reduce noise
logger.info = jest.fn();
logger.debug = jest.fn();
logger.warn = jest.fn();
logger.error = jest.fn();
});
afterAll(async () => {
if (process.env.SKIP_S3_TESTS === 'true') return;
try {
// Clean up S3 bucket
await cleanupS3Bucket();
await s3Client.send(new DeleteBucketCommand({ Bucket: TEST_CONFIG.bucket }));
console.log(`Deleted test bucket: ${TEST_CONFIG.bucket}`);
} catch (error) {
console.error('Failed to cleanup S3 bucket:', error);
}
// Clean up test storage
await fs.rm(testStoragePath, { recursive: true, force: true });
// Restore environment
process.env = originalEnv;
// Close database
await db.destroy();
});
beforeEach(async () => {
if (process.env.SKIP_S3_TESTS === 'true') {
return;
}
// Clean backup tables
await db('backup_runs').del();
await db('backup_file_states').del();
await db('database_backup_runs').del();
// Configure S3 backup settings
await configureS3Backup();
});
afterEach(async () => {
if (process.env.SKIP_S3_TESTS === 'true') return;
// Clean up S3 objects created during test
await cleanupS3Bucket();
});
describe('S3 Connection and Configuration', () => {
it('should successfully connect to S3-compatible storage', async () => {
if (process.env.SKIP_S3_TESTS === 'true') return;
const s3Adapter = new S3StorageAdapter({
...TEST_CONFIG,
bucket: TEST_CONFIG.bucket,
forcePathStyle: true,
sslEnabled: false
});
const connected = await s3Adapter.testConnection();
expect(connected).toBe(true);
});
it('should validate S3 configuration before backup', async () => {
if (process.env.SKIP_S3_TESTS === 'true') return;
// Remove required configuration
await db('app_settings')
.where('setting_key', 'backup_s3_secret_key')
.del();
await backupService.runBackup();
const lastRun = await db('backup_runs')
.orderBy('started_at', 'desc')
.first();
expect(lastRun.status).toBe('failed');
expect(lastRun.error_message).toContain('S3 backup configuration incomplete');
});
});
describe('Full S3 Backup Process', () => {
it('should perform complete S3 backup with all file types', async () => {
if (process.env.SKIP_S3_TESTS === 'true') return;
// Run backup
await backupService.runBackup();
// Verify backup run completed
const backupRun = await db('backup_runs')
.orderBy('started_at', 'desc')
.first();
expect(backupRun.status).toBe('completed');
expect(backupRun.files_backed_up).toBeGreaterThan(0);
expect(backupRun.total_size_bytes).toBeGreaterThan(0);
// Verify files in S3
const s3Objects = await listS3Objects();
expect(s3Objects.length).toBeGreaterThan(0);
// Check for expected file types
const hasPhotos = s3Objects.some(obj => obj.Key.includes('events/active'));
const hasThumbnails = s3Objects.some(obj => obj.Key.includes('thumbnails'));
const hasManifest = s3Objects.some(obj => obj.Key.includes('backup-manifest'));
const hasSummary = s3Objects.some(obj => obj.Key.includes('backup-summary.json'));
expect(hasPhotos).toBe(true);
expect(hasThumbnails).toBe(true);
expect(hasManifest).toBe(true);
expect(hasSummary).toBe(true);
});
it('should handle large file uploads with multipart', async () => {
if (process.env.SKIP_S3_TESTS === 'true') return;
// Create a large test file (15MB)
const largeFilePath = path.join(testStoragePath, 'events/active/large-photo.jpg');
const largeFileSize = 15 * 1024 * 1024; // 15MB
const largeFileContent = Buffer.alloc(largeFileSize, 'x');
await fs.writeFile(largeFilePath, largeFileContent);
// Run backup
await backupService.runBackup();
// Verify large file was uploaded
const s3Objects = await listS3Objects();
const largeFileUploaded = s3Objects.some(obj =>
obj.Key.includes('large-photo.jpg') && obj.Size === largeFileSize
);
expect(largeFileUploaded).toBe(true);
});
it('should include database backup when available', async () => {
if (process.env.SKIP_S3_TESTS === 'true') return;
// Create a mock database backup
const dbBackupPath = path.join(testStoragePath, 'backups/db-backup.sql');
await fs.mkdir(path.dirname(dbBackupPath), { recursive: true });
await fs.writeFile(dbBackupPath, 'CREATE TABLE test (id INT);');
// Record database backup
await db('database_backup_runs').insert({
started_at: new Date(),
completed_at: new Date(),
status: 'completed',
backup_type: 'sqlite',
file_path: dbBackupPath,
file_size_bytes: 100,
checksum: 'test123',
statistics: JSON.stringify({ tables: {} }),
table_checksums: JSON.stringify({})
});
// Configure to include database
await db('app_settings')
.where('setting_key', 'backup_include_database')
.update({ setting_value: 'true' });
// Run backup
await backupService.runBackup();
// Verify database backup in S3
const s3Objects = await listS3Objects();
const hasDbBackup = s3Objects.some(obj => obj.Key.includes('database/db-backup.sql'));
expect(hasDbBackup).toBe(true);
});
});
describe('Incremental Backup', () => {
it('should only upload changed files in incremental backup', async () => {
if (process.env.SKIP_S3_TESTS === 'true') return;
// First backup - full
await backupService.runBackup();
const firstRun = await db('backup_runs')
.orderBy('started_at', 'desc')
.first();
const firstObjectCount = (await listS3Objects()).length;
// Wait a moment to ensure different timestamps
await new Promise(resolve => setTimeout(resolve, 100));
// Modify one file
const modifiedFile = path.join(testStoragePath, 'events/active/event1/photo1.jpg');
await fs.writeFile(modifiedFile, 'modified content');
// Second backup - incremental
await backupService.runBackup();
const secondRun = await db('backup_runs')
.orderBy('started_at', 'desc')
.first();
expect(secondRun.id).not.toBe(firstRun.id);
expect(secondRun.files_backed_up).toBe(1); // Only modified file
// Check manifest indicates incremental
if (secondRun.manifest_path) {
const manifest = await backupService.getBackupManifest(secondRun.id);
expect(manifest.manifest.incremental).toBeDefined();
expect(manifest.manifest.incremental.modified_files_count).toBe(1);
}
});
it('should track file states across backups', async () => {
if (process.env.SKIP_S3_TESTS === 'true') return;
await backupService.runBackup();
// Check file states are recorded
const fileStates = await db('backup_file_states').select('*');
expect(fileStates.length).toBeGreaterThan(0);
// Verify checksums are stored
const hasChecksums = fileStates.every(state => state.checksum !== null);
expect(hasChecksums).toBe(true);
});
});
describe('S3 Manifest Storage', () => {
it('should upload manifest to S3 and retrieve it', async () => {
if (process.env.SKIP_S3_TESTS === 'true') return;
// Configure YAML manifest format
await db('app_settings')
.where('setting_key', 'backup_manifest_format')
.update({ setting_value: '"yaml"' });
await backupService.runBackup();
const backupRun = await db('backup_runs')
.orderBy('started_at', 'desc')
.first();
expect(backupRun.manifest_path).toMatch(/^s3:\/\//);
// Retrieve manifest
const { manifest, summary } = await backupService.getBackupManifest(backupRun.id);
expect(manifest).toBeDefined();
expect(manifest.backup.id).toBeDefined();
expect(summary).toContain('BACKUP MANIFEST SUMMARY');
});
it('should validate manifest integrity', async () => {
if (process.env.SKIP_S3_TESTS === 'true') return;
await backupService.runBackup();
const backupRun = await db('backup_runs')
.orderBy('started_at', 'desc')
.first();
const validationResult = await backupService.validateBackupManifest(backupRun.manifest_path);
expect(validationResult.valid).toBe(true);
expect(validationResult.manifest).toBeDefined();
});
});
describe('Error Recovery', () => {
it('should handle S3 connection failures gracefully', async () => {
if (process.env.SKIP_S3_TESTS === 'true') return;
// Configure with invalid endpoint
await db('app_settings')
.where('setting_key', 'backup_s3_endpoint')
.update({ setting_value: '"http://invalid-endpoint:9999"' });
await backupService.runBackup();
const backupRun = await db('backup_runs')
.orderBy('started_at', 'desc')
.first();
expect(backupRun.status).toBe('failed');
expect(backupRun.error_message).toBeDefined();
});
it('should continue backup despite individual file failures', async () => {
if (process.env.SKIP_S3_TESTS === 'true') return;
// Create a file that will be deleted during backup
const tempFile = path.join(testStoragePath, 'events/active/temp.jpg');
await fs.writeFile(tempFile, 'temporary');
// Mock file deletion during backup
const originalUpload = S3StorageAdapter.prototype.upload;
let callCount = 0;
S3StorageAdapter.prototype.upload = jest.fn(async function(localPath, s3Key, options) {
callCount++;
if (callCount === 2) {
// Delete the temp file to cause an error
await fs.unlink(tempFile).catch(() => {});
}
return originalUpload.call(this, localPath, s3Key, options);
});
await backupService.runBackup();
const backupRun = await db('backup_runs')
.orderBy('started_at', 'desc')
.first();
// Should complete despite one file error
expect(backupRun.status).toBe('completed');
expect(backupRun.files_backed_up).toBeGreaterThan(0);
// Restore original method
S3StorageAdapter.prototype.upload = originalUpload;
});
it('should retry failed uploads with exponential backoff', async () => {
if (process.env.SKIP_S3_TESTS === 'true') return;
// Mock S3 upload to fail twice then succeed
const originalUpload = S3StorageAdapter.prototype.upload;
let attemptCount = 0;
S3StorageAdapter.prototype.upload = jest.fn(async function(localPath, s3Key, options) {
attemptCount++;
if (attemptCount <= 2) {
const error = new Error('Network timeout');
error.code = 'ETIMEDOUT';
throw error;
}
return originalUpload.call(this, localPath, s3Key, options);
});
await backupService.runBackup();
const backupRun = await db('backup_runs')
.orderBy('started_at', 'desc')
.first();
// Should succeed after retries
expect(backupRun.status).toBe('completed');
expect(attemptCount).toBeGreaterThan(2);
// Restore original method
S3StorageAdapter.prototype.upload = originalUpload;
});
});
// Helper functions
async function setupTestData() {
// Create test directory structure
const dirs = [
'events/active/event1',
'events/active/event2',
'events/archived',
'thumbnails',
'uploads'
];
for (const dir of dirs) {
await fs.mkdir(path.join(testStoragePath, dir), { recursive: true });
}
// Create test files
const files = [
{ path: 'events/active/event1/photo1.jpg', content: 'photo1 content' },
{ path: 'events/active/event1/photo2.jpg', content: 'photo2 content' },
{ path: 'events/active/event2/photo3.jpg', content: 'photo3 content' },
{ path: 'events/archived/old-event.zip', content: 'archived content' },
{ path: 'thumbnails/thumb1.jpg', content: 'thumbnail content' },
{ path: 'uploads/logo.png', content: 'logo content' }
];
for (const file of files) {
await fs.writeFile(
path.join(testStoragePath, file.path),
file.content
);
}
}
async function configureS3Backup() {
const settings = [
{ setting_key: 'backup_enabled', setting_value: 'true' },
{ setting_key: 'backup_destination_type', setting_value: '"s3"' },
{ setting_key: 'backup_s3_bucket', setting_value: `"${TEST_CONFIG.bucket}"` },
{ setting_key: 'backup_s3_region', setting_value: `"${TEST_CONFIG.region}"` },
{ setting_key: 'backup_s3_endpoint', setting_value: `"${TEST_CONFIG.endpoint}"` },
{ setting_key: 'backup_s3_access_key', setting_value: `"${TEST_CONFIG.accessKeyId}"` },
{ setting_key: 'backup_s3_secret_key', setting_value: `"${TEST_CONFIG.secretAccessKey}"` },
{ setting_key: 'backup_s3_force_path_style', setting_value: 'true' },
{ setting_key: 'backup_s3_ssl_enabled', setting_value: 'false' },
{ setting_key: 'backup_include_archived', setting_value: 'true' },
{ setting_key: 'backup_incremental', setting_value: 'true' },
{ setting_key: 'backup_max_file_size_mb', setting_value: '100' }
];
for (const setting of settings) {
await db('app_settings')
.insert({
setting_type: 'backup',
...setting,
created_at: new Date(),
updated_at: new Date()
})
.onConflict(['setting_type', 'setting_key'])
.merge();
}
}
async function listS3Objects() {
const response = await s3Client.send(new ListObjectsV2Command({
Bucket: TEST_CONFIG.bucket
}));
return response.Contents || [];
}
async function cleanupS3Bucket() {
try {
const objects = await listS3Objects();
if (objects.length > 0) {
await s3Client.send(new DeleteObjectsCommand({
Bucket: TEST_CONFIG.bucket,
Delete: {
Objects: objects.map(obj => ({ Key: obj.Key }))
}
}));
}
} catch (error) {
console.error('Failed to cleanup S3 objects:', error);
}
}
});
@@ -1,751 +0,0 @@
const { describe, it, expect, jest, beforeEach, afterEach } = require('@jest/globals');
const mockFs = require('mock-fs');
const path = require('path');
const crypto = require('crypto');
const { EventEmitter } = require('events');
// Mock dependencies before requiring the module
jest.mock('../../src/database/db');
jest.mock('../../src/utils/logger');
jest.mock('../../src/services/emailProcessor');
jest.mock('node-cron');
jest.mock('../../src/services/backupManifest');
jest.mock('../../src/services/storage/s3Storage');
const backupService = require('../../src/services/backupService');
const { db } = require('../../src/database/db');
const logger = require('../../src/utils/logger');
const { queueEmail } = require('../../src/services/emailProcessor');
const cron = require('node-cron');
const backupManifest = require('../../src/services/backupManifest');
const S3StorageAdapter = require('../../src/services/storage/s3Storage');
describe('Enhanced Backup Service Tests', () => {
let mockDb;
let mockS3Client;
let mockCronJob;
beforeEach(() => {
// Reset all mocks
jest.clearAllMocks();
// Mock database
mockDb = {
select: jest.fn().mockReturnThis(),
where: jest.fn().mockReturnThis(),
orderBy: jest.fn().mockReturnThis(),
limit: jest.fn().mockReturnThis(),
first: jest.fn(),
insert: jest.fn(),
update: jest.fn(),
delete: jest.fn()
};
db.mockReturnValue(mockDb);
// Mock cron job
mockCronJob = {
stop: jest.fn()
};
cron.schedule.mockReturnValue(mockCronJob);
// Mock S3 client
mockS3Client = {
testConnection: jest.fn().mockResolvedValue(true),
upload: jest.fn().mockResolvedValue({ Location: 's3://bucket/key' }),
uploadStream: jest.fn().mockResolvedValue({ Location: 's3://bucket/key' }),
download: jest.fn().mockResolvedValue(),
exists: jest.fn().mockResolvedValue(false),
delete: jest.fn().mockResolvedValue(),
list: jest.fn().mockResolvedValue({ Contents: [] })
};
S3StorageAdapter.mockImplementation(() => mockS3Client);
// Mock backup manifest
backupManifest.generateManifest = jest.fn().mockResolvedValue({
backup: { id: 'test-backup-123' },
version: '2.0'
});
backupManifest.saveManifest = jest.fn().mockResolvedValue('/path/to/manifest.json');
backupManifest.loadManifest = jest.fn().mockResolvedValue({});
backupManifest.validateManifest = jest.fn();
backupManifest.generateSummaryReport = jest.fn().mockReturnValue('Summary report');
// Mock logger
logger.info = jest.fn();
logger.error = jest.fn();
logger.warn = jest.fn();
logger.debug = jest.fn();
});
afterEach(() => {
mockFs.restore();
});
describe('getBackupConfig', () => {
it('should retrieve and parse backup configuration from database', async () => {
const mockSettings = [
{ setting_key: 'backup_enabled', setting_value: 'true' },
{ setting_key: 'backup_destination_type', setting_value: '"s3"' },
{ setting_key: 'backup_s3_bucket', setting_value: '"test-bucket"' },
{ setting_key: 'backup_retention_days', setting_value: '30' }
];
mockDb.select.mockResolvedValue(mockSettings);
const config = await backupService.getBackupConfig();
expect(config).toEqual({
backup_enabled: true,
backup_destination_type: 's3',
backup_s3_bucket: 'test-bucket',
backup_retention_days: 30
});
expect(db).toHaveBeenCalledWith('app_settings');
expect(mockDb.where).toHaveBeenCalledWith('setting_type', 'backup');
});
it('should handle JSON parse errors gracefully', async () => {
const mockSettings = [
{ setting_key: 'backup_enabled', setting_value: 'invalid-json' }
];
mockDb.select.mockResolvedValue(mockSettings);
const config = await backupService.getBackupConfig();
expect(config).toEqual({
backup_enabled: 'invalid-json'
});
});
it('should return null on database error', async () => {
mockDb.select.mockRejectedValue(new Error('Database error'));
const config = await backupService.getBackupConfig();
expect(config).toBeNull();
expect(logger.error).toHaveBeenCalled();
});
});
describe('S3 Backup Functionality', () => {
beforeEach(() => {
// Mock file system
mockFs({
'/storage/events/active/event1': {
'photo1.jpg': Buffer.from('photo1 content'),
'photo2.jpg': Buffer.from('photo2 content')
},
'/storage/events/archived/event2.zip': Buffer.from('archived content'),
'/storage/thumbnails': {
'thumb1.jpg': Buffer.from('thumb1 content')
},
'/storage/uploads': {
'logo.png': Buffer.from('logo content')
}
});
process.env.STORAGE_PATH = '/storage';
});
it('should perform S3 backup with correct configuration', async () => {
const config = {
backup_enabled: true,
backup_destination_type: 's3',
backup_s3_bucket: 'test-bucket',
backup_s3_region: 'us-east-1',
backup_s3_endpoint: 'https://s3.amazonaws.com',
backup_s3_access_key: 'test-key',
backup_s3_secret_key: 'test-secret',
backup_include_archived: true,
backup_max_file_size_mb: 100
};
mockDb.select.mockResolvedValue([]);
mockDb.where.mockReturnThis();
mockDb.first.mockResolvedValue(null);
mockDb.insert.mockResolvedValue([1]);
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
jest.spyOn(backupService, 'getDatabaseBackupInfo').mockResolvedValue({
type: 'sqlite',
backupFile: null,
hasChanged: true
});
await backupService.runBackup();
expect(S3StorageAdapter).toHaveBeenCalledWith({
bucket: 'test-bucket',
region: 'us-east-1',
endpoint: 'https://s3.amazonaws.com',
accessKeyId: 'test-key',
secretAccessKey: 'test-secret',
forcePathStyle: false,
sslEnabled: true,
maxRetries: 3,
retryDelay: 1000
});
expect(mockS3Client.testConnection).toHaveBeenCalled();
expect(mockS3Client.upload).toHaveBeenCalled();
});
it('should handle S3 upload failures gracefully', async () => {
const config = {
backup_enabled: true,
backup_destination_type: 's3',
backup_s3_bucket: 'test-bucket',
backup_s3_access_key: 'test-key',
backup_s3_secret_key: 'test-secret'
};
mockDb.select.mockResolvedValue([]);
mockDb.insert.mockResolvedValue([1]);
mockDb.first.mockResolvedValue(null);
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
mockS3Client.testConnection.mockRejectedValue(new Error('Connection failed'));
await backupService.runBackup();
expect(logger.error).toHaveBeenCalledWith('S3 backup failed:', expect.any(Error));
expect(mockDb.update).toHaveBeenCalledWith(expect.objectContaining({
status: 'failed',
error_message: expect.stringContaining('Connection failed')
}));
});
it('should skip unchanged files in incremental backup', async () => {
const config = {
backup_enabled: true,
backup_destination_type: 's3',
backup_s3_bucket: 'test-bucket',
backup_s3_access_key: 'test-key',
backup_s3_secret_key: 'test-secret',
backup_incremental: true
};
// Mock existing file state
mockDb.first.mockImplementation((query) => {
if (query === undefined) {
return Promise.resolve({
file_path: 'events/active/event1/photo1.jpg',
checksum: crypto.createHash('sha256').update('photo1 content').digest('hex')
});
}
return Promise.resolve(null);
});
mockDb.select.mockResolvedValue([]);
mockDb.insert.mockResolvedValue([1]);
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
await backupService.runBackup();
// Should skip unchanged file
const uploadCalls = mockS3Client.upload.mock.calls;
const photo1Uploaded = uploadCalls.some(call =>
call[1].includes('photo1.jpg')
);
expect(photo1Uploaded).toBe(false);
});
it('should include database backup when configured', async () => {
const config = {
backup_enabled: true,
backup_destination_type: 's3',
backup_s3_bucket: 'test-bucket',
backup_s3_access_key: 'test-key',
backup_s3_secret_key: 'test-secret',
backup_include_database: true
};
mockDb.select.mockResolvedValue([]);
mockDb.insert.mockResolvedValue([1]);
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
jest.spyOn(backupService, 'getDatabaseBackupInfo').mockResolvedValue({
type: 'sqlite',
backupFile: '/backup/db-backup.sql',
size: 1024000,
checksum: 'abc123',
hasChanged: false
});
// Mock database backup file
mockFs({
'/storage/events/active': {},
'/backup/db-backup.sql': Buffer.from('database backup content')
});
await backupService.runBackup();
// Verify database backup was uploaded
const uploadCalls = mockS3Client.upload.mock.calls;
const dbBackupUploaded = uploadCalls.some(call =>
call[1].includes('database/db-backup.sql')
);
expect(dbBackupUploaded).toBe(true);
});
it('should validate required S3 configuration', async () => {
const config = {
backup_enabled: true,
backup_destination_type: 's3',
backup_s3_bucket: 'test-bucket'
// Missing access key and secret key
};
mockDb.select.mockResolvedValue([]);
mockDb.insert.mockResolvedValue([1]);
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
await backupService.runBackup();
expect(logger.error).toHaveBeenCalledWith(
'S3 backup failed:',
expect.objectContaining({
message: expect.stringContaining('S3 backup configuration incomplete')
})
);
});
});
describe('Manifest Generation', () => {
it('should generate and save manifest for successful backup', async () => {
const config = {
backup_enabled: true,
backup_destination_type: 'local',
backup_destination_path: '/backup',
backup_manifest_format: 'json'
};
mockDb.select.mockResolvedValue([]);
mockDb.insert.mockResolvedValue([1]);
mockDb.first.mockResolvedValue(null);
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
mockFs({
'/storage/events/active/event1': {
'photo1.jpg': Buffer.from('photo1 content')
},
'/backup': {}
});
await backupService.runBackup();
expect(backupManifest.generateManifest).toHaveBeenCalledWith(
expect.objectContaining({
backupType: 'full',
backupPath: '/backup',
format: 'json'
})
);
expect(backupManifest.saveManifest).toHaveBeenCalled();
});
it('should generate incremental manifest when parent exists', async () => {
const config = {
backup_enabled: true,
backup_destination_type: 'local',
backup_destination_path: '/backup'
};
const lastBackup = {
id: 1,
manifest_path: '/backup/manifests/previous.json',
manifest_id: 'previous-backup-123'
};
mockDb.select.mockResolvedValue([]);
mockDb.insert.mockResolvedValue([2]);
mockDb.first.mockImplementation(() => Promise.resolve(lastBackup));
mockDb.orderBy.mockReturnThis();
mockDb.where.mockReturnThis();
mockDb.whereNot = jest.fn().mockReturnThis();
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
mockFs({
'/storage/events/active': {},
'/backup': {}
});
await backupService.runBackup();
expect(backupManifest.loadManifest).toHaveBeenCalledWith('/backup/manifests/previous.json');
expect(backupManifest.generateIncrementalManifest).toHaveBeenCalled();
});
it('should upload manifest to S3 for S3 backups', async () => {
const config = {
backup_enabled: true,
backup_destination_type: 's3',
backup_s3_bucket: 'test-bucket',
backup_s3_access_key: 'test-key',
backup_s3_secret_key: 'test-secret',
backup_manifest_format: 'yaml'
};
mockDb.select.mockResolvedValue([]);
mockDb.insert.mockResolvedValue([1]);
mockDb.first.mockResolvedValue(null);
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
const manifest = {
backup: { id: 'backup-123' },
version: '2.0'
};
backupManifest.generateManifest.mockResolvedValue(manifest);
mockFs({
'/storage/events/active': {},
'/storage/temp': {}
});
await backupService.runBackup();
// Verify manifest was uploaded to S3
const uploadCalls = mockS3Client.upload.mock.calls;
const manifestUploaded = uploadCalls.some(call =>
call[1].includes('manifests/backup-manifest-backup-123.yaml')
);
expect(manifestUploaded).toBe(true);
});
});
describe('Backward Compatibility', () => {
it('should support local backup destination', async () => {
const config = {
backup_enabled: true,
backup_destination_type: 'local',
backup_destination_path: '/backup/local'
};
mockDb.select.mockResolvedValue([]);
mockDb.insert.mockResolvedValue([1]);
mockDb.first.mockResolvedValue(null);
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
mockFs({
'/storage/events/active/event1': {
'photo1.jpg': Buffer.from('photo1 content')
},
'/backup/local': {}
});
await backupService.runBackup();
// Verify files were copied to local destination
const fs = require('fs');
const destPath = '/backup/local/events/active/event1/photo1.jpg';
expect(fs.existsSync(destPath)).toBe(true);
});
it('should support rsync backup destination', async () => {
const config = {
backup_enabled: true,
backup_destination_type: 'rsync',
backup_rsync_host: 'backup.example.com',
backup_rsync_user: 'backup',
backup_rsync_path: '/remote/backup'
};
mockDb.select.mockResolvedValue([]);
mockDb.insert.mockResolvedValue([1]);
mockDb.first.mockResolvedValue(null);
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
// Mock exec for rsync
const { exec } = require('child_process');
const mockExec = jest.fn((cmd, callback) => {
callback(null, { stdout: 'Number of files transferred: 1\nTotal file size: 1024 bytes' });
});
exec.mockImplementation(mockExec);
mockFs({
'/storage/events/active': {}
});
await backupService.runBackup();
expect(mockExec).toHaveBeenCalledWith(
expect.stringContaining('rsync'),
expect.any(Function)
);
});
});
describe('Error Handling and Recovery', () => {
it('should handle file read errors gracefully', async () => {
const config = {
backup_enabled: true,
backup_destination_type: 's3',
backup_s3_bucket: 'test-bucket',
backup_s3_access_key: 'test-key',
backup_s3_secret_key: 'test-secret'
};
mockDb.select.mockResolvedValue([]);
mockDb.insert.mockResolvedValue([1]);
mockDb.first.mockResolvedValue(null);
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
// Mock file that throws error on read
const fs = require('fs');
const originalCreateReadStream = fs.createReadStream;
fs.createReadStream = jest.fn((path) => {
if (path.includes('error.jpg')) {
const stream = new EventEmitter();
process.nextTick(() => stream.emit('error', new Error('File read error')));
return stream;
}
return originalCreateReadStream(path);
});
mockFs({
'/storage/events/active': {
'error.jpg': Buffer.from('content'),
'good.jpg': Buffer.from('content')
}
});
await backupService.runBackup();
// Should continue with other files despite error
expect(logger.error).toHaveBeenCalledWith(
expect.stringContaining('Failed to backup file'),
expect.any(Error)
);
fs.createReadStream = originalCreateReadStream;
});
it('should send failure email on backup error', async () => {
const config = {
backup_enabled: true,
backup_destination_type: 's3',
backup_s3_bucket: 'test-bucket',
backup_email_on_failure: true
};
const admins = [
{ email: 'admin1@example.com', is_active: true },
{ email: 'admin2@example.com', is_active: true }
];
mockDb.select.mockResolvedValue([]);
mockDb.insert.mockResolvedValue([1]);
mockDb.where.mockReturnThis();
jest.spyOn(backupService, 'getBackupConfig')
.mockResolvedValueOnce(config)
.mockResolvedValueOnce(config);
// Force an error
jest.spyOn(backupService, 'getFilesToBackup').mockRejectedValue(new Error('Storage error'));
// Mock admin users query
db.mockImplementation((table) => {
if (table === 'admin_users') {
return {
where: jest.fn().mockResolvedValue(admins)
};
}
return mockDb;
});
await backupService.runBackup();
expect(queueEmail).toHaveBeenCalledTimes(2);
expect(queueEmail).toHaveBeenCalledWith(
null,
'admin1@example.com',
'backup_failed',
expect.objectContaining({
error_message: 'Storage error'
})
);
});
it('should handle concurrent backup attempts', async () => {
const config = {
backup_enabled: true,
backup_destination_type: 'local',
backup_destination_path: '/backup'
};
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
mockFs({
'/storage/events/active': {},
'/backup': {}
});
// Start two backups concurrently
const backup1 = backupService.runBackup();
const backup2 = backupService.runBackup();
await Promise.all([backup1, backup2]);
// Second backup should be skipped
expect(logger.warn).toHaveBeenCalledWith('Backup already running, skipping');
});
});
describe('Service Lifecycle', () => {
it('should start backup service with cron schedule', async () => {
const config = {
backup_enabled: true,
backup_schedule: '0 3 * * *' // 3 AM daily
};
mockDb.select.mockResolvedValue(
Object.entries(config).map(([key, value]) => ({
setting_key: key,
setting_value: value.toString()
}))
);
await backupService.startBackupService();
expect(cron.schedule).toHaveBeenCalledWith('0 3 * * *', expect.any(Function));
expect(logger.info).toHaveBeenCalledWith('Backup service started with schedule: 0 3 * * *');
});
it('should stop existing job when restarting service', async () => {
const config = {
backup_enabled: true,
backup_schedule: '0 2 * * *'
};
mockDb.select.mockResolvedValue(
Object.entries(config).map(([key, value]) => ({
setting_key: key,
setting_value: value.toString()
}))
);
// Start service twice
await backupService.startBackupService();
await backupService.startBackupService();
expect(mockCronJob.stop).toHaveBeenCalled();
});
it('should not start service when backup is disabled', async () => {
const config = {
backup_enabled: false
};
mockDb.select.mockResolvedValue([
{ setting_key: 'backup_enabled', setting_value: 'false' }
]);
await backupService.startBackupService();
expect(cron.schedule).not.toHaveBeenCalled();
expect(logger.info).toHaveBeenCalledWith('Backup service is disabled');
});
});
describe('Backup Status and History', () => {
it('should return backup status with recent runs', async () => {
const recentRuns = [
{
id: 1,
started_at: new Date(),
completed_at: new Date(),
status: 'completed',
files_backed_up: 100,
total_size_bytes: 1024000,
manifest_path: '/backup/manifest.json'
}
];
mockDb.limit.mockResolvedValue(recentRuns);
backupManifest.validateManifest.mockImplementation(() => true);
const status = await backupService.getBackupStatus();
expect(status).toEqual({
isRunning: false,
isHealthy: true,
lastRun: expect.objectContaining({
...recentRuns[0],
manifestValid: true
}),
recentRuns: recentRuns,
nextScheduledRun: expect.any(String)
});
});
it('should clean up old backup runs', async () => {
mockDb.delete.mockResolvedValue(5);
await backupService.cleanupOldBackupRuns(30);
expect(mockDb.where).toHaveBeenCalledWith('started_at', '<', expect.any(Date));
expect(mockDb.delete).toHaveBeenCalled();
expect(logger.info).toHaveBeenCalledWith('Cleaned up 5 old backup runs');
});
});
describe('getBackupManifest', () => {
it('should retrieve manifest from local filesystem', async () => {
const backupRun = {
id: 1,
manifest_path: '/backup/manifests/backup-123.json'
};
mockDb.first.mockResolvedValue(backupRun);
const manifest = { backup: { id: 'backup-123' } };
backupManifest.loadManifest.mockResolvedValue(manifest);
backupManifest.generateSummaryReport.mockReturnValue('Summary');
const result = await backupService.getBackupManifest(1);
expect(result).toEqual({
manifest: manifest,
summary: 'Summary'
});
});
it('should retrieve manifest from S3', async () => {
const backupRun = {
id: 1,
manifest_path: 's3://test-bucket/backups/manifests/backup-123.json'
};
mockDb.first.mockResolvedValue(backupRun);
mockDb.select.mockResolvedValue([
{ setting_key: 'backup_s3_access_key', setting_value: '"test-key"' },
{ setting_key: 'backup_s3_secret_key', setting_value: '"test-secret"' }
]);
const manifest = { backup: { id: 'backup-123' } };
backupManifest.loadManifest.mockResolvedValue(manifest);
await backupService.getBackupManifest(1);
expect(S3StorageAdapter).toHaveBeenCalled();
expect(mockS3Client.download).toHaveBeenCalledWith(
'backups/manifests/backup-123.json',
expect.any(String)
);
});
});
});
Binary file not shown.
-161
View File
@@ -1,161 +0,0 @@
# Security Logging Documentation
## Overview
This document describes the comprehensive security logging implemented in the PicPeak application to track authentication failures, rate limiting, and suspicious activities.
## Log Files
### 1. **security.log**
- Location: `logs/security.log`
- Contains: All security-related events (authentication, rate limiting, suspicious activity)
- Max Size: 20MB with rotation (keeps 10 files)
- Format: JSON with timestamp
### 2. **error.log**
- Location: `logs/error.log`
- Contains: All error-level logs including auth failures
- Max Size: 10MB with rotation (keeps 5 files)
### 3. **combined.log**
- Location: `logs/combined.log`
- Contains: All logs (info, warn, error)
- Max Size: 50MB with rotation (keeps 10 files)
## Security Events Logged
### Rate Limiting
When rate limits are exceeded, the following is logged:
```json
{
"timestamp": "2024-01-18 14:23:45.123",
"level": "warn",
"message": "Rate limit exceeded",
"security": true,
"ip": "192.168.1.1",
"path": "/api/admin/login",
"method": "POST",
"authenticated": false,
"userAgent": "Mozilla/5.0...",
"referer": "https://app.example.com",
"origin": "https://app.example.com",
"headers": {
"x-forwarded-for": "192.168.1.1",
"x-real-ip": "192.168.1.1"
},
"requestUrl": "/api/admin/login",
"rateLimitInfo": {
"limit": 5,
"current": 6,
"remaining": 0,
"resetTime": "2024-01-18T14:38:45.123Z"
}
}
```
### Authentication Failures
#### Admin Login Failures
- Tracked in `login_attempts` table
- Logged with: IP address, username, user agent, timestamp
- Account lockout after 5 failures in 15 minutes
#### Gallery Password Failures
- Tracked in `access_logs` table with action='login_fail'
- Logged with: event_id, IP address, user agent
- Gallery lockout after 5 failures in 15 minutes
### JWT Validation Failures
```json
{
"timestamp": "2024-01-18 14:23:45.123",
"level": "warn",
"message": "JWT validation failed",
"ip": "192.168.1.1",
"path": "/api/admin/events",
"method": "GET",
"userAgent": "Mozilla/5.0...",
"error": "TokenExpiredError",
"message": "jwt expired"
}
```
### Suspicious Activity
- Multiple IPs attempting login for same account
- Token usage from different IP than issued
- Token usage after password change
- Revoked token usage attempts
## Configuration Settings
All rate limiting settings are configurable via the admin panel:
| Setting | Default | Range | Description |
|---------|---------|-------|-------------|
| rate_limit_enabled | true | - | Enable/disable rate limiting |
| rate_limit_window_minutes | 15 | 1-60 | Time window for rate limit |
| rate_limit_max_requests | 1000 | 10-10000 | Max requests for general endpoints |
| rate_limit_auth_max_requests | 5 | 1-100 | Max requests for auth endpoints |
| rate_limit_skip_authenticated | true | - | Skip rate limit for authenticated requests |
| rate_limit_public_endpoints_only | false | - | Only rate limit public endpoints |
## Database Tables
### login_attempts
```sql
- id
- username
- ip_address
- user_agent
- success (boolean)
- created_at
```
### access_logs
```sql
- id
- event_id
- ip_address
- user_agent
- action ('view', 'download', 'login_success', 'login_fail')
- photo_id (nullable)
- created_at
```
## Environment Variables
- `LOG_LEVEL`: Set logging level (default: 'info')
- `LOG_TO_CONSOLE`: Enable console logging in production (default: false)
## Monitoring Recommendations
1. **Set up alerts for:**
- Rate limit exceeded events (possible DDoS)
- Multiple failed login attempts from same IP
- Account lockout events
- JWT validation failures spike
2. **Regular review:**
- Check security.log for patterns
- Review login_attempts table for brute force attempts
- Monitor access_logs for suspicious gallery access patterns
3. **Log analysis tools:**
- Use log aggregation tools (ELK stack, Splunk)
- Set up dashboards for security metrics
- Configure alerts for threshold breaches
## Production Deployment Notes
1. Ensure logs directory has proper permissions
2. Set up log rotation outside of application if needed
3. Consider shipping logs to centralized logging service
4. Monitor disk space for log files
5. Set `LOG_TO_CONSOLE=true` for container deployments
## Security Best Practices
1. Never log sensitive data (passwords, tokens)
2. Use generic error messages to prevent user enumeration
3. Clean up old login attempts regularly (7 days retention)
4. Monitor for unusual patterns in real-time
5. Keep rate limit settings appropriate for your usage
+4 -4
View File
@@ -40,10 +40,10 @@ const config = {
keepAliveInitialDelayMillis: 0
},
pool: {
min: 5,
max: 25,
acquireTimeoutMillis: 60000,
createTimeoutMillis: 60000,
min: 2,
max: 10,
acquireTimeoutMillis: 30000,
createTimeoutMillis: 30000,
idleTimeoutMillis: 30000,
reapIntervalMillis: 1000,
createRetryIntervalMillis: 200,
@@ -8,7 +8,8 @@ exports.up = async function(knex) {
await knex('app_settings').insert({
setting_key: 'general_default_welcome_message',
setting_value: JSON.stringify('Thank you for using our photo sharing service! We hope you enjoy your photos.'),
setting_type: 'general'
setting_type: 'general',
updated_at: new Date()
});
}
@@ -1,62 +0,0 @@
exports.up = async function(knex) {
// Add rate limit settings to app_settings
const rateLimitSettings = [
{
setting_key: 'rate_limit_enabled',
setting_value: JSON.stringify(true),
setting_type: 'security'
},
{
setting_key: 'rate_limit_window_minutes',
setting_value: JSON.stringify(15),
setting_type: 'security'
},
{
setting_key: 'rate_limit_max_requests',
setting_value: JSON.stringify(1000),
setting_type: 'security'
},
{
setting_key: 'rate_limit_auth_max_requests',
setting_value: JSON.stringify(5),
setting_type: 'security'
},
{
setting_key: 'rate_limit_skip_authenticated',
setting_value: JSON.stringify(true),
setting_type: 'security'
},
{
setting_key: 'rate_limit_public_endpoints_only',
setting_value: JSON.stringify(false),
setting_type: 'security'
}
];
// Insert settings if they don't exist
for (const setting of rateLimitSettings) {
const exists = await knex('app_settings')
.where('setting_key', setting.setting_key)
.first();
if (!exists) {
await knex('app_settings').insert({
...setting
});
}
}
};
exports.down = async function(knex) {
// Remove rate limit settings
await knex('app_settings')
.whereIn('setting_key', [
'rate_limit_enabled',
'rate_limit_window_minutes',
'rate_limit_max_requests',
'rate_limit_auth_max_requests',
'rate_limit_skip_authenticated',
'rate_limit_public_endpoints_only'
])
.del();
};
@@ -1,243 +0,0 @@
const { db } = require('../src/database/db');
async function up() {
console.log('Adding backup service tables and settings...');
// Create backup_runs table to track backup history
const hasBackupRunsTable = await db.schema.hasTable('backup_runs');
if (!hasBackupRunsTable) {
await db.schema.createTable('backup_runs', (table) => {
table.increments('id').primary();
table.datetime('started_at').notNullable();
table.datetime('completed_at');
table.string('status').defaultTo('running'); // running, completed, failed
table.string('backup_type'); // full, incremental
table.integer('files_backed_up').defaultTo(0);
table.bigInteger('total_size_bytes').defaultTo(0);
table.integer('duration_seconds');
table.text('error_message');
table.json('statistics'); // Detailed stats about the backup
table.json('file_checksums'); // Store checksums for change detection
});
}
// Create backup_file_states table to track individual file states
const hasBackupFileStatesTable = await db.schema.hasTable('backup_file_states');
if (!hasBackupFileStatesTable) {
await db.schema.createTable('backup_file_states', (table) => {
table.increments('id').primary();
table.string('file_path').notNullable();
table.string('checksum').notNullable();
table.bigInteger('size_bytes');
table.datetime('last_modified');
table.datetime('last_backed_up');
table.boolean('is_archived').defaultTo(false);
table.index(['file_path'], 'idx_backup_file_path');
table.index(['checksum'], 'idx_backup_checksum');
});
}
// Add backup-related settings to app_settings
const backupSettings = [
{
setting_key: 'backup_enabled',
setting_value: JSON.stringify(false),
setting_type: 'backup'
},
{
setting_key: 'backup_schedule',
setting_value: JSON.stringify('0 2 * * *'), // Default: 2 AM daily
setting_type: 'backup'
},
{
setting_key: 'backup_destination_type',
setting_value: JSON.stringify('local'), // local, rsync, s3
setting_type: 'backup'
},
{
setting_key: 'backup_destination_path',
setting_value: JSON.stringify('/backup/picpeak'),
setting_type: 'backup'
},
{
setting_key: 'backup_rsync_host',
setting_value: JSON.stringify(''),
setting_type: 'backup'
},
{
setting_key: 'backup_rsync_user',
setting_value: JSON.stringify(''),
setting_type: 'backup'
},
{
setting_key: 'backup_rsync_path',
setting_value: JSON.stringify(''),
setting_type: 'backup'
},
{
setting_key: 'backup_rsync_ssh_key',
setting_value: JSON.stringify(''),
setting_type: 'backup'
},
{
setting_key: 'backup_s3_endpoint',
setting_value: JSON.stringify(''),
setting_type: 'backup'
},
{
setting_key: 'backup_s3_bucket',
setting_value: JSON.stringify(''),
setting_type: 'backup'
},
{
setting_key: 'backup_s3_access_key',
setting_value: JSON.stringify(''),
setting_type: 'backup'
},
{
setting_key: 'backup_s3_secret_key',
setting_value: JSON.stringify(''),
setting_type: 'backup'
},
{
setting_key: 'backup_s3_region',
setting_value: JSON.stringify('us-east-1'),
setting_type: 'backup'
},
{
setting_key: 'backup_retention_days',
setting_value: JSON.stringify(30),
setting_type: 'backup'
},
{
setting_key: 'backup_include_archived',
setting_value: JSON.stringify(true),
setting_type: 'backup'
},
{
setting_key: 'backup_compression',
setting_value: JSON.stringify(true),
setting_type: 'backup'
},
{
setting_key: 'backup_email_on_failure',
setting_value: JSON.stringify(true),
setting_type: 'backup'
},
{
setting_key: 'backup_email_on_success',
setting_value: JSON.stringify(false),
setting_type: 'backup'
},
{
setting_key: 'backup_max_file_size_mb',
setting_value: JSON.stringify(5000), // Skip files larger than 5GB
setting_type: 'backup'
},
{
setting_key: 'backup_exclude_patterns',
setting_value: JSON.stringify(['*.tmp', '.DS_Store', 'Thumbs.db']),
setting_type: 'backup'
}
];
// Insert backup settings if they don't exist
for (const setting of backupSettings) {
const exists = await db('app_settings')
.where('setting_key', setting.setting_key)
.first();
if (!exists) {
await db('app_settings').insert(setting);
}
}
// Add backup-related email templates
const backupEmailTemplates = [
{
template_key: 'backup_failed',
subject_en: 'Backup Failed - Immediate Attention Required',
subject_de: 'Backup fehlgeschlagen - Sofortige Aufmerksamkeit erforderlich',
body_html_en: `<h2>Backup Failed</h2>
<p>The scheduled backup has failed and requires immediate attention.</p>
<p><strong>Error Details:</strong></p>
<ul>
<li>Start Time: {{start_time}}</li>
<li>Backup Type: {{backup_type}}</li>
<li>Error: {{error_message}}</li>
</ul>
<p>Please check the system logs for more details and resolve the issue as soon as possible.</p>`,
body_html_de: `<h2>Backup fehlgeschlagen</h2>
<p>Das geplante Backup ist fehlgeschlagen und erfordert sofortige Aufmerksamkeit.</p>
<p><strong>Fehlerdetails:</strong></p>
<ul>
<li>Startzeit: {{start_time}}</li>
<li>Backup-Typ: {{backup_type}}</li>
<li>Fehler: {{error_message}}</li>
</ul>
<p>Bitte überprüfen Sie die Systemprotokolle für weitere Details und beheben Sie das Problem so schnell wie möglich.</p>`,
body_text_en: 'Backup Failed\n\nThe scheduled backup has failed and requires immediate attention.\n\nStart Time: {{start_time}}\nBackup Type: {{backup_type}}\nError: {{error_message}}\n\nPlease check the system logs for more details.',
body_text_de: 'Backup fehlgeschlagen\n\nDas geplante Backup ist fehlgeschlagen und erfordert sofortige Aufmerksamkeit.\n\nStartzeit: {{start_time}}\nBackup-Typ: {{backup_type}}\nFehler: {{error_message}}\n\nBitte überprüfen Sie die Systemprotokolle für weitere Details.',
variables: JSON.stringify(['start_time', 'backup_type', 'error_message'])
},
{
template_key: 'backup_completed',
subject_en: 'Backup Completed Successfully',
subject_de: 'Backup erfolgreich abgeschlossen',
body_html_en: `<h2>Backup Completed</h2>
<p>The scheduled backup has been completed successfully.</p>
<p><strong>Backup Summary:</strong></p>
<ul>
<li>Start Time: {{start_time}}</li>
<li>Duration: {{duration}}</li>
<li>Files Backed Up: {{files_count}}</li>
<li>Total Size: {{total_size}}</li>
<li>Backup Type: {{backup_type}}</li>
</ul>`,
body_html_de: `<h2>Backup abgeschlossen</h2>
<p>Das geplante Backup wurde erfolgreich abgeschlossen.</p>
<p><strong>Backup-Zusammenfassung:</strong></p>
<ul>
<li>Startzeit: {{start_time}}</li>
<li>Dauer: {{duration}}</li>
<li>Gesicherte Dateien: {{files_count}}</li>
<li>Gesamtgröße: {{total_size}}</li>
<li>Backup-Typ: {{backup_type}}</li>
</ul>`,
body_text_en: 'Backup Completed\n\nThe scheduled backup has been completed successfully.\n\nStart Time: {{start_time}}\nDuration: {{duration}}\nFiles Backed Up: {{files_count}}\nTotal Size: {{total_size}}\nBackup Type: {{backup_type}}',
body_text_de: 'Backup abgeschlossen\n\nDas geplante Backup wurde erfolgreich abgeschlossen.\n\nStartzeit: {{start_time}}\nDauer: {{duration}}\nGesicherte Dateien: {{files_count}}\nGesamtgröße: {{total_size}}\nBackup-Typ: {{backup_type}}',
variables: JSON.stringify(['start_time', 'duration', 'files_count', 'total_size', 'backup_type'])
}
];
// Insert backup email templates if they don't exist
for (const template of backupEmailTemplates) {
const exists = await db('email_templates')
.where('template_key', template.template_key)
.first();
if (!exists) {
await db('email_templates').insert(template);
}
}
console.log('Backup service tables and settings added successfully');
}
async function down() {
// Remove backup tables
await db.schema.dropTableIfExists('backup_file_states');
await db.schema.dropTableIfExists('backup_runs');
// Remove backup settings
await db('app_settings')
.where('setting_type', 'backup')
.delete();
// Remove backup email templates
await db('email_templates')
.whereIn('template_key', ['backup_failed', 'backup_completed'])
.delete();
}
module.exports = { up, down };
@@ -1,182 +0,0 @@
const { db } = require('../src/database/db');
async function up() {
console.log('Adding database backup tables and settings...');
// Create database_backup_runs table to track database backup history
const hasDatabaseBackupRunsTable = await db.schema.hasTable('database_backup_runs');
if (!hasDatabaseBackupRunsTable) {
await db.schema.createTable('database_backup_runs', (table) => {
table.increments('id').primary();
table.datetime('started_at').notNullable();
table.datetime('completed_at');
table.string('status').defaultTo('running'); // running, completed, failed
table.string('backup_type'); // sqlite, postgresql
table.string('destination_path');
table.string('file_path');
table.bigInteger('file_size_bytes').defaultTo(0);
table.bigInteger('original_size_bytes').defaultTo(0);
table.integer('duration_seconds');
table.string('checksum'); // SHA256 checksum of backup file
table.float('compression_ratio'); // Compression percentage
table.json('table_checksums'); // Individual table checksums
table.text('error_message');
table.json('statistics'); // Detailed stats about the backup
table.index(['started_at'], 'idx_db_backup_started');
table.index(['status'], 'idx_db_backup_status');
});
}
// Add database backup-related settings to app_settings
const databaseBackupSettings = [
{
setting_key: 'database_backup_enabled',
setting_value: JSON.stringify(false),
setting_type: 'database_backup'
},
{
setting_key: 'database_backup_schedule',
setting_value: JSON.stringify('0 3 * * *'), // Default: 3 AM daily
setting_type: 'database_backup'
},
{
setting_key: 'database_backup_destination_path',
setting_value: JSON.stringify('/backup/database'),
setting_type: 'database_backup'
},
{
setting_key: 'database_backup_compress',
setting_value: JSON.stringify(true),
setting_type: 'database_backup'
},
{
setting_key: 'database_backup_validate_integrity',
setting_value: JSON.stringify(true),
setting_type: 'database_backup'
},
{
setting_key: 'database_backup_include_checksums',
setting_value: JSON.stringify(true),
setting_type: 'database_backup'
},
{
setting_key: 'database_backup_retention_days',
setting_value: JSON.stringify(30),
setting_type: 'database_backup'
},
{
setting_key: 'database_backup_email_on_failure',
setting_value: JSON.stringify(true),
setting_type: 'database_backup'
},
{
setting_key: 'database_backup_email_on_success',
setting_value: JSON.stringify(false),
setting_type: 'database_backup'
},
{
setting_key: 'database_backup_max_retries',
setting_value: JSON.stringify(3),
setting_type: 'database_backup'
}
];
// Insert database backup settings if they don't exist
for (const setting of databaseBackupSettings) {
const exists = await db('app_settings')
.where('setting_key', setting.setting_key)
.first();
if (!exists) {
await db('app_settings').insert(setting);
}
}
// Add database backup-related email templates
const databaseBackupEmailTemplates = [
{
template_key: 'database_backup_failed',
subject_en: 'Database Backup Failed - Critical Alert',
subject_de: 'Datenbank-Backup fehlgeschlagen - Kritische Warnung',
body_html_en: `<h2>Database Backup Failed</h2>
<p>The scheduled database backup has failed and requires immediate attention.</p>
<p><strong>Error Details:</strong></p>
<ul>
<li>Backup Type: {{backup_type}}</li>
<li>Timestamp: {{timestamp}}</li>
<li>Error: {{error_message}}</li>
</ul>
<p>This is a critical issue that could affect disaster recovery. Please investigate immediately.</p>`,
body_html_de: `<h2>Datenbank-Backup fehlgeschlagen</h2>
<p>Das geplante Datenbank-Backup ist fehlgeschlagen und erfordert sofortige Aufmerksamkeit.</p>
<p><strong>Fehlerdetails:</strong></p>
<ul>
<li>Backup-Typ: {{backup_type}}</li>
<li>Zeitstempel: {{timestamp}}</li>
<li>Fehler: {{error_message}}</li>
</ul>
<p>Dies ist ein kritisches Problem, das die Disaster-Recovery beeinträchtigen könnte. Bitte untersuchen Sie es sofort.</p>`,
body_text_en: 'Database Backup Failed\n\nThe scheduled database backup has failed.\n\nBackup Type: {{backup_type}}\nTimestamp: {{timestamp}}\nError: {{error_message}}\n\nThis is critical - please investigate immediately.',
body_text_de: 'Datenbank-Backup fehlgeschlagen\n\nDas geplante Datenbank-Backup ist fehlgeschlagen.\n\nBackup-Typ: {{backup_type}}\nZeitstempel: {{timestamp}}\nFehler: {{error_message}}\n\nDies ist kritisch - bitte sofort untersuchen.',
variables: JSON.stringify(['backup_type', 'timestamp', 'error_message'])
},
{
template_key: 'database_backup_completed',
subject_en: 'Database Backup Completed Successfully',
subject_de: 'Datenbank-Backup erfolgreich abgeschlossen',
body_html_en: `<h2>Database Backup Completed</h2>
<p>The scheduled database backup has been completed successfully.</p>
<p><strong>Backup Summary:</strong></p>
<ul>
<li>Backup Type: {{backup_type}}</li>
<li>Duration: {{duration}}</li>
<li>File Size: {{file_size}}</li>
<li>Compression Ratio: {{compression_ratio}}</li>
<li>File Path: {{file_path}}</li>
</ul>`,
body_html_de: `<h2>Datenbank-Backup abgeschlossen</h2>
<p>Das geplante Datenbank-Backup wurde erfolgreich abgeschlossen.</p>
<p><strong>Backup-Zusammenfassung:</strong></p>
<ul>
<li>Backup-Typ: {{backup_type}}</li>
<li>Dauer: {{duration}}</li>
<li>Dateigröße: {{file_size}}</li>
<li>Komprimierungsverhältnis: {{compression_ratio}}</li>
<li>Dateipfad: {{file_path}}</li>
</ul>`,
body_text_en: 'Database Backup Completed\n\nThe scheduled database backup has been completed successfully.\n\nBackup Type: {{backup_type}}\nDuration: {{duration}}\nFile Size: {{file_size}}\nCompression Ratio: {{compression_ratio}}\nFile Path: {{file_path}}',
body_text_de: 'Datenbank-Backup abgeschlossen\n\nDas geplante Datenbank-Backup wurde erfolgreich abgeschlossen.\n\nBackup-Typ: {{backup_type}}\nDauer: {{duration}}\nDateigröße: {{file_size}}\nKomprimierungsverhältnis: {{compression_ratio}}\nDateipfad: {{file_path}}',
variables: JSON.stringify(['backup_type', 'duration', 'file_size', 'compression_ratio', 'file_path'])
}
];
// Insert database backup email templates if they don't exist
for (const template of databaseBackupEmailTemplates) {
const exists = await db('email_templates')
.where('template_key', template.template_key)
.first();
if (!exists) {
await db('email_templates').insert(template);
}
}
console.log('Database backup tables and settings added successfully');
}
async function down() {
// Remove database backup tables
await db.schema.dropTableIfExists('database_backup_runs');
// Remove database backup settings
await db('app_settings')
.where('setting_type', 'database_backup')
.delete();
// Remove database backup email templates
await db('email_templates')
.whereIn('template_key', ['database_backup_failed', 'database_backup_completed'])
.delete();
}
module.exports = { up, down };
@@ -1,86 +0,0 @@
const { db } = require('../src/database/db');
const logger = require('../src/utils/logger');
async function up() {
console.log('Adding backup manifest columns...');
// Add manifest columns to backup_runs table
const hasManifestPath = await db.schema.hasColumn('backup_runs', 'manifest_path');
if (!hasManifestPath) {
await db.schema.alterTable('backup_runs', (table) => {
table.string('manifest_path'); // Path to the manifest file
table.string('manifest_id'); // Unique manifest ID
table.string('manifest_format').defaultTo('json'); // json or yaml
});
}
// Add backup manifest-related settings to app_settings
const manifestSettings = [
{
setting_key: 'backup_manifest_enabled',
setting_value: JSON.stringify(true),
setting_type: 'backup'
},
{
setting_key: 'backup_manifest_format',
setting_value: JSON.stringify('json'), // json or yaml
setting_type: 'backup'
},
{
setting_key: 'backup_manifest_path',
setting_value: JSON.stringify('/backup/manifests'),
setting_type: 'backup'
},
{
setting_key: 'backup_manifest_validate',
setting_value: JSON.stringify(true),
setting_type: 'backup'
},
{
setting_key: 'backup_manifest_include_checksums',
setting_value: JSON.stringify(true),
setting_type: 'backup'
}
];
// Insert manifest settings if they don't exist
for (const setting of manifestSettings) {
const exists = await db('app_settings')
.where('setting_key', setting.setting_key)
.first();
if (!exists) {
await db('app_settings').insert(setting);
}
}
console.log('✓ Backup manifest columns and settings added');
}
async function down() {
// Remove manifest columns from backup_runs table
const hasManifestPath = await db.schema.hasColumn('backup_runs', 'manifest_path');
if (hasManifestPath) {
await db.schema.alterTable('backup_runs', (table) => {
table.dropColumn('manifest_path');
table.dropColumn('manifest_id');
table.dropColumn('manifest_format');
});
}
// Remove manifest settings
await db('app_settings')
.where('setting_type', 'backup')
.whereIn('setting_key', [
'backup_manifest_enabled',
'backup_manifest_format',
'backup_manifest_path',
'backup_manifest_validate',
'backup_manifest_include_checksums'
])
.delete();
console.log('✓ Backup manifest columns and settings removed');
}
module.exports = { up, down };
@@ -1,259 +0,0 @@
// No helpers needed for this migration
/**
* Add restore_runs table for tracking restore operations
*/
exports.up = async function(knex) {
// Create restore_runs table
await knex.schema.createTable('restore_runs', table => {
table.increments('id').primary();
// Timing
table.timestamp('started_at').notNullable().defaultTo(knex.fn.now());
table.timestamp('completed_at');
table.integer('duration_seconds');
// Status and type
table.string('status', 50).notNullable().defaultTo('running');
table.string('restore_type', 50).notNullable(); // full, database, files, selective
// Source information
table.string('source', 500).notNullable(); // Backup source path or S3 URL
table.string('manifest_path', 500); // Path to manifest file
// Results
table.text('error_message');
table.text('statistics'); // JSON object with detailed statistics
table.text('restore_log'); // JSON array of log entries
// Safety backup
table.string('pre_restore_backup_path', 500); // Path to pre-restore safety backup
// Flags
table.boolean('is_dry_run').defaultTo(false);
table.boolean('was_rollback_attempted').defaultTo(false);
table.boolean('was_successful').defaultTo(false);
// Operator information
table.string('operator_type', 50).defaultTo('manual'); // manual, scheduled, api
table.integer('operator_user_id').references('id').inTable('admin_users').onDelete('SET NULL');
table.string('operator_ip', 50);
// Metadata
table.text('metadata'); // JSON object for additional data
table.index(['status', 'started_at']);
table.index(['restore_type', 'started_at']);
});
// Create restore_file_operations table for tracking individual file operations
await knex.schema.createTable('restore_file_operations', table => {
table.increments('id').primary();
table.integer('restore_run_id').notNullable()
.references('id').inTable('restore_runs').onDelete('CASCADE');
table.string('file_path', 500).notNullable();
table.string('operation', 50).notNullable(); // restore, skip, error
table.string('status', 50).notNullable(); // pending, in_progress, completed, failed
table.bigInteger('file_size');
table.string('checksum', 64);
table.boolean('checksum_verified').defaultTo(false);
table.text('error_message');
table.timestamp('started_at');
table.timestamp('completed_at');
table.index(['restore_run_id', 'status']);
table.index(['file_path']);
});
// Create restore_validation_results table
await knex.schema.createTable('restore_validation_results', table => {
table.increments('id').primary();
table.integer('restore_run_id').notNullable()
.references('id').inTable('restore_runs').onDelete('CASCADE');
table.string('validation_type', 50).notNullable(); // pre-restore, post-restore
table.boolean('is_valid').notNullable();
table.text('errors'); // JSON array of errors
table.text('warnings'); // JSON array of warnings
table.text('checksums'); // JSON object with checksum comparisons
table.timestamp('validated_at').notNullable().defaultTo(knex.fn.now());
table.index(['restore_run_id', 'validation_type']);
});
// Add restore-related settings to app_settings
await knex('app_settings').insert([
{
setting_key: 'restore_allow_force',
setting_value: JSON.stringify(false),
setting_type: 'restore'
},
{
setting_key: 'restore_require_pre_backup',
setting_value: JSON.stringify(true),
setting_type: 'restore'
},
{
setting_key: 'restore_max_file_size_mb',
setting_value: '5000',
setting_type: 'restore'
},
{
setting_key: 'restore_verify_checksums',
setting_value: JSON.stringify(true),
setting_type: 'restore'
},
{
setting_key: 'restore_email_on_completion',
setting_value: JSON.stringify(true),
setting_type: 'restore'
},
{
setting_key: 'restore_retention_days',
setting_value: '30',
setting_type: 'restore'
}
]);
// Add new email templates for restore notifications
const emailTemplates = [
{
template_key: 'restore_completed',
subject_en: '✅ Restore Completed Successfully',
subject_de: '✅ Wiederherstellung erfolgreich abgeschlossen',
body_html_en: `<h2>Restore Operation Completed</h2>
<p>A restore operation has completed successfully.</p>
<h3>Details:</h3>
<ul>
<li><strong>Restore Type:</strong> {{restore_type}}</li>
<li><strong>Duration:</strong> {{duration}}</li>
<li><strong>Files Restored:</strong> {{files_restored}}</li>
<li><strong>Backup ID:</strong> {{backup_id}}</li>
<li><strong>Timestamp:</strong> {{timestamp}}</li>
</ul>
<p>Please verify that all systems are functioning correctly after the restore.</p>`,
body_html_de: `<h2>Wiederherstellungsvorgang abgeschlossen</h2>
<p>Ein Wiederherstellungsvorgang wurde erfolgreich abgeschlossen.</p>
<h3>Details:</h3>
<ul>
<li><strong>Wiederherstellungstyp:</strong> {{restore_type}}</li>
<li><strong>Dauer:</strong> {{duration}}</li>
<li><strong>Wiederhergestellte Dateien:</strong> {{files_restored}}</li>
<li><strong>Backup-ID:</strong> {{backup_id}}</li>
<li><strong>Zeitstempel:</strong> {{timestamp}}</li>
</ul>
<p>Bitte überprüfen Sie, ob alle Systeme nach der Wiederherstellung ordnungsgemäß funktionieren.</p>`,
body_text_en: `Restore Operation Completed
A restore operation has completed successfully.
Details:
- Restore Type: {{restore_type}}
- Duration: {{duration}}
- Files Restored: {{files_restored}}
- Backup ID: {{backup_id}}
- Timestamp: {{timestamp}}
Please verify that all systems are functioning correctly after the restore.`,
body_text_de: `Wiederherstellungsvorgang abgeschlossen
Ein Wiederherstellungsvorgang wurde erfolgreich abgeschlossen.
Details:
- Wiederherstellungstyp: {{restore_type}}
- Dauer: {{duration}}
- Wiederhergestellte Dateien: {{files_restored}}
- Backup-ID: {{backup_id}}
- Zeitstempel: {{timestamp}}
Bitte überprüfen Sie, ob alle Systeme nach der Wiederherstellung ordnungsgemäß funktionieren.`,
variables: JSON.stringify(['restore_type', 'duration', 'files_restored', 'backup_id', 'timestamp'])
},
{
template_key: 'restore_failed',
subject_en: '❌ Restore Operation Failed',
subject_de: '❌ Wiederherstellungsvorgang fehlgeschlagen',
body_html_en: `<h2>Restore Operation Failed</h2>
<p>A restore operation has failed and requires attention.</p>
<h3>Details:</h3>
<ul>
<li><strong>Restore Type:</strong> {{restore_type}}</li>
<li><strong>Error:</strong> {{error_message}}</li>
<li><strong>Timestamp:</strong> {{timestamp}}</li>
</ul>
<p>Please check the system logs for more details and take appropriate action.</p>
<p><strong>Important:</strong> If a pre-restore backup was created, it may be used for recovery.</p>`,
body_html_de: `<h2>Wiederherstellungsvorgang fehlgeschlagen</h2>
<p>Ein Wiederherstellungsvorgang ist fehlgeschlagen und erfordert Ihre Aufmerksamkeit.</p>
<h3>Details:</h3>
<ul>
<li><strong>Wiederherstellungstyp:</strong> {{restore_type}}</li>
<li><strong>Fehler:</strong> {{error_message}}</li>
<li><strong>Zeitstempel:</strong> {{timestamp}}</li>
</ul>
<p>Bitte überprüfen Sie die Systemprotokolle für weitere Details und ergreifen Sie entsprechende Maßnahmen.</p>
<p><strong>Wichtig:</strong> Falls ein Backup vor der Wiederherstellung erstellt wurde, kann es zur Wiederherstellung verwendet werden.</p>`,
body_text_en: `Restore Operation Failed
A restore operation has failed and requires attention.
Details:
- Restore Type: {{restore_type}}
- Error: {{error_message}}
- Timestamp: {{timestamp}}
Please check the system logs for more details and take appropriate action.
Important: If a pre-restore backup was created, it may be used for recovery.`,
body_text_de: `Wiederherstellungsvorgang fehlgeschlagen
Ein Wiederherstellungsvorgang ist fehlgeschlagen und erfordert Ihre Aufmerksamkeit.
Details:
- Wiederherstellungstyp: {{restore_type}}
- Fehler: {{error_message}}
- Zeitstempel: {{timestamp}}
Bitte überprüfen Sie die Systemprotokolle für weitere Details und ergreifen Sie entsprechende Maßnahmen.
Wichtig: Falls ein Backup vor der Wiederherstellung erstellt wurde, kann es zur Wiederherstellung verwendet werden.`,
variables: JSON.stringify(['restore_type', 'error_message', 'timestamp'])
}
];
await knex('email_templates').insert(emailTemplates);
};
exports.down = async function(knex) {
// Remove email templates
await knex('email_templates')
.whereIn('template_key', ['restore_completed', 'restore_failed'])
.delete();
// Remove settings
await knex('app_settings')
.where('setting_type', 'restore')
.delete();
// Drop tables
await knex.schema.dropTableIfExists('restore_validation_results');
await knex.schema.dropTableIfExists('restore_file_operations');
await knex.schema.dropTableIfExists('restore_runs');
};
@@ -1,129 +0,0 @@
// No helpers needed for boolean values
exports.up = async function(knex) {
console.log('Adding gallery feedback tables...');
// Create event_feedback_settings table
await knex.schema.createTable('event_feedback_settings', (table) => {
table.increments('id').primary();
table.integer('event_id').references('id').inTable('events').onDelete('CASCADE');
table.boolean('feedback_enabled').defaultTo(false);
table.boolean('allow_ratings').defaultTo(true);
table.boolean('allow_likes').defaultTo(true);
table.boolean('allow_comments').defaultTo(false);
table.boolean('allow_favorites').defaultTo(true);
table.boolean('require_name_email').defaultTo(false);
table.boolean('moderate_comments').defaultTo(true);
table.boolean('show_feedback_to_guests').defaultTo(true);
table.timestamp('created_at').defaultTo(knex.fn.now());
table.timestamp('updated_at').defaultTo(knex.fn.now());
table.unique(['event_id']);
});
// Create photo_feedback table
await knex.schema.createTable('photo_feedback', (table) => {
table.increments('id').primary();
table.integer('photo_id').references('id').inTable('photos').onDelete('CASCADE');
table.integer('event_id').references('id').inTable('events').onDelete('CASCADE');
table.string('feedback_type', 20).notNullable();
table.integer('rating');
table.text('comment_text');
table.string('guest_name', 100);
table.string('guest_email', 255);
table.string('guest_identifier', 64);
table.string('ip_address', 45);
table.text('user_agent');
table.boolean('is_approved').defaultTo(true);
table.boolean('is_hidden').defaultTo(false);
table.timestamp('created_at').defaultTo(knex.fn.now());
table.timestamp('updated_at').defaultTo(knex.fn.now());
// Add indexes
table.index(['photo_id']);
table.index(['event_id']);
table.index(['feedback_type']);
table.index(['guest_identifier']);
// Add check constraint for rating (PostgreSQL)
if (knex.client.config.client === 'pg') {
table.check('?? >= 1 AND ?? <= 5', ['rating', 'rating']);
}
});
// Create feedback_rate_limits table
await knex.schema.createTable('feedback_rate_limits', (table) => {
table.increments('id').primary();
table.string('identifier', 64).notNullable();
table.integer('event_id').references('id').inTable('events').onDelete('CASCADE');
table.string('action_type', 20).notNullable();
table.integer('action_count').defaultTo(1);
table.timestamp('window_start').defaultTo(knex.fn.now());
// Add indexes
table.index(['identifier', 'event_id', 'action_type']);
table.index(['window_start']);
});
// Create feedback_word_filters table
await knex.schema.createTable('feedback_word_filters', (table) => {
table.increments('id').primary();
table.string('word', 100).notNullable();
table.string('severity', 20).defaultTo('moderate');
table.boolean('is_active').defaultTo(true);
table.timestamp('created_at').defaultTo(knex.fn.now());
table.unique(['word']);
});
// Add feedback summary columns to photos table
await knex.schema.alterTable('photos', (table) => {
table.integer('feedback_count').defaultTo(0);
table.integer('like_count').defaultTo(0);
table.decimal('average_rating', 3, 2).defaultTo(0);
table.integer('favorite_count').defaultTo(0);
});
// Add feedback notification settings to app_settings
await knex('app_settings').insert([
{
setting_key: 'feedback_notification_email',
setting_value: JSON.stringify(''),
setting_type: 'feedback'
},
{
setting_key: 'feedback_rate_limits',
setting_value: JSON.stringify({
rating: { max: 100, window: 3600 }, // 100 ratings per hour
comment: { max: 20, window: 3600 }, // 20 comments per hour
like: { max: 200, window: 3600 } // 200 likes per hour
}),
setting_type: 'feedback'
}
]);
console.log('Gallery feedback tables created successfully');
};
exports.down = async function(knex) {
console.log('Removing gallery feedback tables...');
// Remove feedback settings from app_settings
await knex('app_settings')
.whereIn('setting_key', ['feedback_notification_email', 'feedback_rate_limits'])
.delete();
// Remove feedback columns from photos table
await knex.schema.alterTable('photos', (table) => {
table.dropColumn('feedback_count');
table.dropColumn('like_count');
table.dropColumn('average_rating');
table.dropColumn('favorite_count');
});
// Drop tables in reverse order
await knex.schema.dropTableIfExists('feedback_word_filters');
await knex.schema.dropTableIfExists('feedback_rate_limits');
await knex.schema.dropTableIfExists('photo_feedback');
await knex.schema.dropTableIfExists('event_feedback_settings');
console.log('Gallery feedback tables removed');
};
@@ -1,139 +0,0 @@
const { db } = require('../src/database/db');
async function up() {
console.log('Adding version tracking to backup tables...');
// Add version columns to database_backup_runs table
const hasDatabaseBackupRunsTable = await db.schema.hasTable('database_backup_runs');
if (hasDatabaseBackupRunsTable) {
const hasAppVersion = await db.schema.hasColumn('database_backup_runs', 'app_version');
if (!hasAppVersion) {
await db.schema.alterTable('database_backup_runs', (table) => {
table.string('app_version'); // Application version
table.string('node_version'); // Node.js version
table.string('db_schema_version'); // Database schema version (migration name)
table.json('environment_info'); // Additional environment information
});
console.log('Added version columns to database_backup_runs table');
}
}
// Add version columns to backup_runs table (file backups)
const hasBackupRunsTable = await db.schema.hasTable('backup_runs');
if (hasBackupRunsTable) {
const hasAppVersion = await db.schema.hasColumn('backup_runs', 'app_version');
if (!hasAppVersion) {
await db.schema.alterTable('backup_runs', (table) => {
table.string('app_version'); // Application version
table.string('node_version'); // Node.js version
table.string('db_schema_version'); // Database schema version
table.json('manifest_info'); // Manifest summary information
});
console.log('Added version columns to backup_runs table');
}
}
// Add restore tracking table
const hasRestoreHistoryTable = await db.schema.hasTable('restore_history');
if (!hasRestoreHistoryTable) {
await db.schema.createTable('restore_history', (table) => {
table.increments('id').primary();
table.datetime('started_at').notNullable();
table.datetime('completed_at');
table.string('status').defaultTo('running'); // running, completed, failed, partial
table.string('restore_type'); // database, files, full
table.string('backup_id'); // Reference to the backup that was restored
table.string('backup_app_version'); // Version of app that created the backup
table.string('restore_app_version'); // Version of app performing the restore
table.string('backup_node_version'); // Node version that created the backup
table.string('restore_node_version'); // Node version performing the restore
table.string('backup_schema_version'); // Schema version in the backup
table.string('restore_schema_version'); // Current schema version
table.json('version_compatibility'); // Compatibility check results
table.json('restore_options'); // Options used during restore
table.json('statistics'); // Restore statistics
table.text('warnings'); // Any warnings during restore
table.text('error_message'); // Error details if failed
table.string('restored_by'); // User who initiated the restore
table.index(['started_at'], 'idx_restore_started');
table.index(['backup_id'], 'idx_restore_backup_id');
});
console.log('Created restore_history table');
}
// Add version compatibility settings
const versionSettings = [
{
setting_key: 'backup_require_version_match',
setting_value: JSON.stringify(false), // If true, exact version match required for restore
setting_type: 'backup'
},
{
setting_key: 'backup_allow_minor_version_mismatch',
setting_value: JSON.stringify(true), // Allow restoring from same major version
setting_type: 'backup'
},
{
setting_key: 'backup_warn_on_version_mismatch',
setting_value: JSON.stringify(true), // Show warning when versions don't match
setting_type: 'backup'
},
{
setting_key: 'backup_check_schema_compatibility',
setting_value: JSON.stringify(true), // Check if migrations are compatible
setting_type: 'backup'
}
];
// Insert version settings if they don't exist
for (const setting of versionSettings) {
const exists = await db('app_settings')
.where('setting_key', setting.setting_key)
.first();
if (!exists) {
await db('app_settings').insert(setting);
}
}
console.log('Version tracking for backups added successfully');
}
async function down() {
// Remove version columns from database_backup_runs
const hasDatabaseBackupRunsTable = await db.schema.hasTable('database_backup_runs');
if (hasDatabaseBackupRunsTable) {
await db.schema.alterTable('database_backup_runs', (table) => {
table.dropColumn('app_version');
table.dropColumn('node_version');
table.dropColumn('db_schema_version');
table.dropColumn('environment_info');
});
}
// Remove version columns from backup_runs
const hasBackupRunsTable = await db.schema.hasTable('backup_runs');
if (hasBackupRunsTable) {
await db.schema.alterTable('backup_runs', (table) => {
table.dropColumn('app_version');
table.dropColumn('node_version');
table.dropColumn('db_schema_version');
table.dropColumn('manifest_info');
});
}
// Drop restore_history table
await db.schema.dropTableIfExists('restore_history');
// Remove version settings
await db('app_settings')
.whereIn('setting_key', [
'backup_require_version_match',
'backup_allow_minor_version_mismatch',
'backup_warn_on_version_mismatch',
'backup_check_schema_compatibility'
])
.delete();
}
module.exports = { up, down };
@@ -1,221 +0,0 @@
const { db } = require('../src/database/db');
async function up() {
console.log('Enhancing backup system...');
// Add new settings to app_settings table if they don't exist
const backupSettings = [
{
setting_key: 'backup_s3_force_path_style',
setting_value: JSON.stringify(false),
setting_type: 'backup'
},
{
setting_key: 'backup_s3_ssl_enabled',
setting_value: JSON.stringify(true),
setting_type: 'backup'
},
{
setting_key: 'backup_s3_prefix',
setting_value: JSON.stringify(''),
setting_type: 'backup'
},
{
setting_key: 'backup_incremental',
setting_value: JSON.stringify(false),
setting_type: 'backup'
},
{
setting_key: 'backup_include_database',
setting_value: JSON.stringify(true),
setting_type: 'backup'
},
{
setting_key: 'backup_encryption_enabled',
setting_value: JSON.stringify(false),
setting_type: 'backup'
},
{
setting_key: 'backup_database_schedule',
setting_value: JSON.stringify(''),
setting_type: 'backup'
}
];
// Insert settings if they don't exist
for (const setting of backupSettings) {
const exists = await db('app_settings')
.where('setting_key', setting.setting_key)
.first();
if (!exists) {
await db('app_settings').insert(setting);
}
}
// Check and add columns to backup_runs table
const hasBackupRunsTable = await db.schema.hasTable('backup_runs');
if (hasBackupRunsTable) {
// Check for existing columns before adding
const hasManifestPath = await db.schema.hasColumn('backup_runs', 'manifest_path');
const hasManifestId = await db.schema.hasColumn('backup_runs', 'manifest_id');
const hasManifestFormat = await db.schema.hasColumn('backup_runs', 'manifest_format');
const hasParentBackupId = await db.schema.hasColumn('backup_runs', 'parent_backup_id');
const hasBackupMode = await db.schema.hasColumn('backup_runs', 'backup_mode');
if (!hasManifestPath || !hasManifestId || !hasManifestFormat || !hasParentBackupId || !hasBackupMode) {
await db.schema.alterTable('backup_runs', (table) => {
if (!hasManifestPath) {
table.string('manifest_path', 500).comment('Path to backup manifest file');
}
if (!hasManifestId) {
table.uuid('manifest_id').comment('Unique identifier for the manifest');
}
if (!hasManifestFormat) {
table.enum('manifest_format', ['json', 'yaml']).comment('Format of the manifest file');
}
if (!hasParentBackupId) {
table.integer('parent_backup_id').unsigned().references('id').inTable('backup_runs').onDelete('SET NULL').comment('Parent backup for incremental backups');
}
if (!hasBackupMode) {
table.enum('backup_mode', ['full', 'incremental', 'database']).defaultTo('full').comment('Type of backup performed');
}
});
}
// Add indexes if they don't exist
try {
await db.raw('CREATE INDEX IF NOT EXISTS idx_backup_runs_mode_status ON backup_runs(backup_mode, status)');
await db.raw('CREATE INDEX IF NOT EXISTS idx_backup_runs_parent ON backup_runs(parent_backup_id)');
await db.raw('CREATE INDEX IF NOT EXISTS idx_backup_runs_created_mode ON backup_runs(created_at, backup_mode)');
} catch (error) {
console.log('Note: Some indexes may already exist, continuing...');
}
}
// Create backup_manifest table if it doesn't exist
const hasManifestTable = await db.schema.hasTable('backup_manifest');
if (!hasManifestTable) {
await db.schema.createTable('backup_manifest', (table) => {
table.increments('id').primary();
table.integer('backup_run_id').unsigned().notNullable().references('id').inTable('backup_runs').onDelete('CASCADE');
table.uuid('manifest_id').notNullable().unique().comment('Unique identifier matching backup_runs.manifest_id');
table.string('version', 20).notNullable().defaultTo('1.0.0').comment('Manifest schema version');
table.enum('format', ['json', 'yaml']).notNullable().defaultTo('json');
// Backup metadata
table.timestamp('backup_start').notNullable();
table.timestamp('backup_end').notNullable();
table.bigInteger('total_size').unsigned().comment('Total size of backup in bytes');
table.integer('file_count').unsigned().comment('Number of files in backup');
table.integer('photo_count').unsigned().comment('Number of photos backed up');
table.integer('event_count').unsigned().comment('Number of events backed up');
// Incremental backup metadata
table.boolean('is_incremental').defaultTo(false);
table.uuid('parent_manifest_id').comment('Parent manifest ID for incremental backups');
table.timestamp('incremental_since').comment('Timestamp for incremental backup baseline');
// Content checksums
table.string('checksum_algorithm', 50).defaultTo('sha256').comment('Algorithm used for checksums');
table.text('manifest_checksum').comment('Checksum of the manifest file itself');
// Storage information
table.string('storage_location', 500).comment('Primary storage location (local path or S3 URI)');
table.string('storage_provider', 50).comment('Storage provider (local, s3, etc.)');
// Encryption metadata
table.boolean('is_encrypted').defaultTo(false);
table.string('encryption_algorithm', 100).comment('Encryption algorithm used');
table.string('encryption_key_id', 255).comment('ID of encryption key used');
// Additional metadata as JSON
table.json('metadata').comment('Additional metadata as JSON');
// Timestamps
table.timestamps(true, true);
// Indexes
table.index(['backup_run_id'], 'idx_manifest_backup_run');
table.index(['manifest_id'], 'idx_manifest_uuid');
table.index(['parent_manifest_id'], 'idx_manifest_parent');
table.index(['backup_start', 'backup_end'], 'idx_manifest_time_range');
table.index(['is_incremental', 'created_at'], 'idx_manifest_incremental_created');
});
}
// Add composite indexes for common query patterns
try {
await db.raw(`
CREATE INDEX IF NOT EXISTS idx_backup_runs_recent_successful
ON backup_runs(created_at DESC)
WHERE status = 'completed' AND backup_mode = 'full';
`);
await db.raw(`
CREATE INDEX IF NOT EXISTS idx_backup_runs_incremental_chain
ON backup_runs(parent_backup_id, created_at)
WHERE backup_mode = 'incremental';
`);
} catch (error) {
console.log('Note: Some composite indexes may already exist, continuing...');
}
console.log('Backup system enhancements completed');
}
async function down() {
// Drop indexes first
try {
await db.raw('DROP INDEX IF EXISTS idx_backup_runs_incremental_chain;');
await db.raw('DROP INDEX IF EXISTS idx_backup_runs_recent_successful;');
} catch (error) {
// Ignore errors if indexes don't exist
}
// Drop backup_manifest table
await db.schema.dropTableIfExists('backup_manifest');
// Remove columns from backup_runs if they exist
const hasBackupRunsTable = await db.schema.hasTable('backup_runs');
if (hasBackupRunsTable) {
const hasBackupMode = await db.schema.hasColumn('backup_runs', 'backup_mode');
const hasParentBackupId = await db.schema.hasColumn('backup_runs', 'parent_backup_id');
const hasManifestFormat = await db.schema.hasColumn('backup_runs', 'manifest_format');
const hasManifestId = await db.schema.hasColumn('backup_runs', 'manifest_id');
const hasManifestPath = await db.schema.hasColumn('backup_runs', 'manifest_path');
if (hasBackupMode || hasParentBackupId || hasManifestFormat || hasManifestId || hasManifestPath) {
await db.schema.alterTable('backup_runs', (table) => {
if (hasBackupMode) table.dropColumn('backup_mode');
if (hasParentBackupId) table.dropColumn('parent_backup_id');
if (hasManifestFormat) table.dropColumn('manifest_format');
if (hasManifestId) table.dropColumn('manifest_id');
if (hasManifestPath) table.dropColumn('manifest_path');
});
}
// Drop indexes
try {
await db.raw('DROP INDEX IF EXISTS idx_backup_runs_mode_status');
await db.raw('DROP INDEX IF EXISTS idx_backup_runs_parent');
await db.raw('DROP INDEX IF EXISTS idx_backup_runs_created_mode');
} catch (error) {
// Ignore errors if indexes don't exist
}
}
// Remove settings
await db('app_settings')
.whereIn('setting_key', [
'backup_s3_force_path_style',
'backup_s3_ssl_enabled',
'backup_s3_prefix',
'backup_incremental',
'backup_include_database',
'backup_encryption_enabled',
'backup_database_schedule'
])
.delete();
}
module.exports = { up, down };
+307 -2401
View File
File diff suppressed because it is too large Load Diff
+9 -17
View File
@@ -1,6 +1,6 @@
{
"name": "picpeak-backend",
"version": "1.0.93",
"version": "1.0.57",
"description": "Backend for PicPeak event photo sharing platform",
"main": "server.js",
"scripts": {
@@ -8,43 +8,35 @@
"dev": "nodemon server.js",
"migrate": "node migrations/run-migrations.js",
"migrate:safe": "node migrations/run-migrations-safe.js",
"fix-temp-photos": "node scripts/fix-temp-photos.js",
"test": "jest",
"lint": "eslint src/",
"test-backup": "node scripts/test-backup-service.js",
"test-restore": "node scripts/test-restore-service.js"
"lint": "eslint src/"
},
"dependencies": {
"@aws-sdk/client-s3": "^3.850.0",
"@aws-sdk/lib-storage": "^3.850.0",
"@aws-sdk/s3-request-presigner": "^3.850.0",
"adm-zip": "^0.5.16",
"archiver": "^5.3.1",
"axios": "^1.10.0",
"bcrypt": "6.0.0",
"chokidar": "4.0.3",
"bcrypt": "^5.1.0",
"chokidar": "^3.5.3",
"cors": "^2.8.5",
"dotenv": "^16.0.3",
"express": "^4.18.2",
"express-rate-limit": "^6.7.0",
"express-validator": "^7.0.1",
"form-data": "^4.0.4",
"form-data": "^4.0.3",
"handlebars": "^4.7.8",
"helmet": "^7.0.0",
"i18next": "25.3.2",
"i18next": "^25.3.1",
"i18next-browser-languagedetector": "^8.2.0",
"i18next-http-backend": "^3.0.2",
"joi": "^17.9.1",
"js-yaml": "^4.1.0",
"jsonwebtoken": "^9.0.0",
"knex": "^2.4.2",
"mime-types": "^3.0.1",
"multer": "^2.0.2",
"multer": "^2.0.1",
"node-cron": "^3.0.2",
"nodemailer": "7.0.5",
"nodemailer": "^6.9.1",
"pg": "^8.16.3",
"react-i18next": "^15.6.0",
"sharp": "0.34.3",
"sharp": "^0.32.0",
"sqlite3": "^5.1.6",
"uuid": "^11.1.0",
"winston": "^3.8.2",
-171
View File
@@ -1,171 +0,0 @@
require('dotenv').config({ path: '../.env' });
const path = require('path');
const fs = require('fs').promises;
const { db } = require('../src/database/db');
const { generatePhotoFilename } = require('../src/utils/filenameSanitizer');
async function fixTempPhotos() {
console.log('Starting to fix temporary photo files...\n');
try {
// Find all photos with temp_ filenames
const tempPhotos = await db('photos')
.where('filename', 'like', 'temp_%')
.orderBy('event_id', 'asc')
.orderBy('category_id', 'asc')
.orderBy('id', 'asc');
console.log(`Found ${tempPhotos.length} photos with temporary filenames\n`);
if (tempPhotos.length === 0) {
console.log('No temporary photos found. Exiting.');
return;
}
// Group photos by event and category
const grouped = {};
for (const photo of tempPhotos) {
const key = `${photo.event_id}_${photo.category_id || 'null'}`;
if (!grouped[key]) {
grouped[key] = [];
}
grouped[key].push(photo);
}
console.log(`Processing ${Object.keys(grouped).length} event/category groups...\n`);
// Process each group
for (const [key, photos] of Object.entries(grouped)) {
const [eventId, categoryIdStr] = key.split('_');
const categoryId = categoryIdStr === 'null' ? null : parseInt(categoryIdStr);
console.log(`\nProcessing Event ID: ${eventId}, Category ID: ${categoryId || 'uncategorized'}`);
console.log(`Photos in group: ${photos.length}`);
// Get event details
const event = await db('events').where({ id: eventId }).first();
if (!event) {
console.error(`Event ${eventId} not found! Skipping...`);
continue;
}
// Get category details if applicable
let category = null;
let startCounter = 1;
if (categoryId) {
category = await db('photo_categories').where({ id: categoryId }).first();
if (!category) {
console.error(`Category ${categoryId} not found! Treating as uncategorized...`);
} else {
// Get the highest counter for this category
const maxPhoto = await db('photos')
.where({ event_id: eventId, category_id: categoryId })
.whereNot('filename', 'like', 'temp_%')
.orderBy('id', 'desc')
.first();
if (maxPhoto && maxPhoto.filename) {
// Extract counter from filename
const match = maxPhoto.filename.match(/_(\d+)\.[^.]+$/);
if (match) {
startCounter = parseInt(match[1]) + 1;
}
}
}
} else {
// For uncategorized, get the highest counter
const maxPhoto = await db('photos')
.where({ event_id: eventId })
.whereNull('category_id')
.whereNot('filename', 'like', 'temp_%')
.orderBy('id', 'desc')
.first();
if (maxPhoto && maxPhoto.filename) {
const match = maxPhoto.filename.match(/_(\d+)\.[^.]+$/);
if (match) {
startCounter = parseInt(match[1]) + 1;
}
}
}
console.log(`Starting counter: ${startCounter}`);
// Process each photo in the group
let successCount = 0;
let errorCount = 0;
for (let i = 0; i < photos.length; i++) {
const photo = photos[i];
const counter = startCounter + i;
try {
// Generate new filename
const extension = path.extname(photo.filename);
const newFilename = generatePhotoFilename(
event.event_name,
category ? category.name : 'uncategorized',
counter,
extension
);
// Build full paths
const storagePath = process.env.STORAGE_PATH || path.join(__dirname, '../../storage');
const oldPath = path.join(storagePath, 'events/active', photo.path);
const newPath = path.join(path.dirname(oldPath), newFilename);
// Check if old file exists
try {
await fs.access(oldPath);
} catch (e) {
console.error(`File not found: ${oldPath}`);
errorCount++;
continue;
}
// Rename the file
await fs.rename(oldPath, newPath);
// Update database
const newRelativePath = path.relative(path.join(storagePath, 'events/active'), newPath);
await db('photos')
.where({ id: photo.id })
.update({
filename: newFilename,
path: newRelativePath
});
console.log(`✓ Renamed: ${photo.filename}${newFilename}`);
successCount++;
} catch (error) {
console.error(`✗ Failed to process photo ${photo.id}: ${error.message}`);
errorCount++;
}
}
// Update category counter if needed
if (category && successCount > 0) {
const newCounter = startCounter + photos.length - 1;
await db('photo_categories')
.where({ id: categoryId })
.update({ photo_counter: newCounter });
console.log(`Updated category counter to ${newCounter}`);
}
console.log(`\nGroup summary: ${successCount} successful, ${errorCount} errors`);
}
console.log('\n=== COMPLETE ===');
console.log('All temporary photos have been processed.');
} catch (error) {
console.error('Fatal error:', error);
} finally {
await db.destroy();
}
}
// Run the script
fixTempPhotos().catch(console.error);
-577
View File
@@ -1,577 +0,0 @@
#!/usr/bin/env node
/**
* Manual Integration Test Script for Enhanced Backup System
*
* This script provides a comprehensive test of the backup system with real services.
* It can be used to test against MinIO, AWS S3, or other S3-compatible services.
*
* Usage:
* node scripts/test-backup-integration.js [options]
*
* Options:
* --endpoint <url> S3 endpoint URL (default: http://localhost:9000)
* --access-key <key> S3 access key (default: minioadmin)
* --secret-key <key> S3 secret key (default: minioadmin)
* --bucket <name> S3 bucket name (default: test-backup-<timestamp>)
* --type <type> Backup type: s3, local, rsync (default: s3)
* --cleanup Clean up test data after completion
* --verbose Enable verbose logging
* --help Show this help message
*
* Examples:
* # Test with local MinIO
* node scripts/test-backup-integration.js
*
* # Test with AWS S3
* node scripts/test-backup-integration.js \
* --endpoint https://s3.amazonaws.com \
* --access-key AKIAIOSFODNN7EXAMPLE \
* --secret-key wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY \
* --bucket my-test-bucket
*
* # Test local backup
* node scripts/test-backup-integration.js --type local
*/
const path = require('path');
const fs = require('fs').promises;
const crypto = require('crypto');
const { S3Client, CreateBucketCommand, HeadBucketCommand, ListObjectsV2Command, GetObjectCommand, DeleteObjectsCommand, DeleteBucketCommand } = require('@aws-sdk/client-s3');
// Parse command line arguments
const args = process.argv.slice(2);
const options = {
endpoint: 'http://localhost:9000',
accessKey: 'minioadmin',
secretKey: 'minioadmin',
bucket: `test-backup-${Date.now()}`,
type: 's3',
cleanup: false,
verbose: false
};
for (let i = 0; i < args.length; i++) {
switch (args[i]) {
case '--endpoint':
options.endpoint = args[++i];
break;
case '--access-key':
options.accessKey = args[++i];
break;
case '--secret-key':
options.secretKey = args[++i];
break;
case '--bucket':
options.bucket = args[++i];
break;
case '--type':
options.type = args[++i];
break;
case '--cleanup':
options.cleanup = true;
break;
case '--verbose':
options.verbose = true;
break;
case '--help':
console.log(module.exports.description || 'Manual Integration Test Script');
process.exit(0);
}
}
// Load environment and services
require('dotenv').config();
const { db, initialize: initDb } = require('../src/database/db');
const backupService = require('../src/services/backupService');
const S3StorageAdapter = require('../src/services/storage/s3Storage');
const logger = require('../src/utils/logger');
// Configure logger based on verbose flag
if (!options.verbose) {
logger.info = () => {};
logger.debug = () => {};
}
// Test results
const results = {
passed: 0,
failed: 0,
skipped: 0,
tests: []
};
// Test utilities
async function runTest(name, testFn) {
console.log(`\n📋 Running: ${name}`);
try {
const startTime = Date.now();
await testFn();
const duration = Date.now() - startTime;
console.log(`✅ PASSED: ${name} (${duration}ms)`);
results.passed++;
results.tests.push({ name, status: 'passed', duration });
} catch (error) {
console.error(`❌ FAILED: ${name}`);
console.error(` Error: ${error.message}`);
if (options.verbose) {
console.error(error.stack);
}
results.failed++;
results.tests.push({ name, status: 'failed', error: error.message });
}
}
async function skipTest(name, reason) {
console.log(`\n⏭️ Skipping: ${name}`);
console.log(` Reason: ${reason}`);
results.skipped++;
results.tests.push({ name, status: 'skipped', reason });
}
// Test functions
async function testS3Connection() {
const s3Adapter = new S3StorageAdapter({
bucket: options.bucket,
endpoint: options.endpoint,
accessKeyId: options.accessKey,
secretAccessKey: options.secretKey,
region: 'us-east-1',
forcePathStyle: true,
sslEnabled: options.endpoint.startsWith('https')
});
await s3Adapter.testConnection();
console.log(` ✓ Connected to S3 endpoint: ${options.endpoint}`);
console.log(` ✓ Bucket accessible: ${options.bucket}`);
}
async function setupTestData() {
const storagePath = path.join(__dirname, '../test-storage');
process.env.STORAGE_PATH = storagePath;
// Create directory structure
const dirs = [
'events/active/wedding-2024',
'events/active/birthday-2024',
'events/archived',
'thumbnails',
'uploads',
'backups'
];
for (const dir of dirs) {
await fs.mkdir(path.join(storagePath, dir), { recursive: true });
}
// Create test files with various sizes
const files = [
{ path: 'events/active/wedding-2024/photo1.jpg', size: 1024 * 1024 }, // 1MB
{ path: 'events/active/wedding-2024/photo2.jpg', size: 512 * 1024 }, // 512KB
{ path: 'events/active/birthday-2024/photo1.jpg', size: 2 * 1024 * 1024 }, // 2MB
{ path: 'events/archived/old-event.zip', size: 5 * 1024 * 1024 }, // 5MB
{ path: 'thumbnails/thumb1.jpg', size: 50 * 1024 }, // 50KB
{ path: 'uploads/logo.png', size: 100 * 1024 } // 100KB
];
let totalSize = 0;
for (const file of files) {
const content = crypto.randomBytes(file.size);
await fs.writeFile(path.join(storagePath, file.path), content);
totalSize += file.size;
}
console.log(` ✓ Created ${files.length} test files`);
console.log(` ✓ Total size: ${(totalSize / 1024 / 1024).toFixed(2)} MB`);
return { storagePath, fileCount: files.length, totalSize };
}
async function configureBackup(type) {
const baseSettings = [
{ setting_key: 'backup_enabled', setting_value: 'true' },
{ setting_key: 'backup_destination_type', setting_value: `"${type}"` },
{ setting_key: 'backup_include_archived', setting_value: 'true' },
{ setting_key: 'backup_include_database', setting_value: 'true' },
{ setting_key: 'backup_incremental', setting_value: 'true' },
{ setting_key: 'backup_manifest_format', setting_value: '"json"' },
{ setting_key: 'backup_max_file_size_mb', setting_value: '100' }
];
const typeSpecificSettings = {
s3: [
{ setting_key: 'backup_s3_bucket', setting_value: `"${options.bucket}"` },
{ setting_key: 'backup_s3_endpoint', setting_value: `"${options.endpoint}"` },
{ setting_key: 'backup_s3_access_key', setting_value: `"${options.accessKey}"` },
{ setting_key: 'backup_s3_secret_key', setting_value: `"${options.secretKey}"` },
{ setting_key: 'backup_s3_region', setting_value: '"us-east-1"' },
{ setting_key: 'backup_s3_force_path_style', setting_value: 'true' },
{ setting_key: 'backup_s3_ssl_enabled', setting_value: options.endpoint.startsWith('https') ? 'true' : 'false' }
],
local: [
{ setting_key: 'backup_destination_path', setting_value: `"${path.join(__dirname, '../test-backup')}"` }
],
rsync: [
{ setting_key: 'backup_rsync_host', setting_value: '"localhost"' },
{ setting_key: 'backup_rsync_path', setting_value: `"${path.join(__dirname, '../test-backup-rsync')}"` }
]
};
const settings = [...baseSettings, ...(typeSpecificSettings[type] || [])];
// Clear existing settings
await db('app_settings').where('setting_type', 'backup').del();
// Insert new settings
for (const setting of settings) {
await db('app_settings').insert({
setting_type: 'backup',
...setting,
created_at: new Date(),
updated_at: new Date()
});
}
console.log(` ✓ Configured ${type} backup with ${settings.length} settings`);
}
async function performBackup() {
const startTime = Date.now();
// Run the backup
await backupService.runBackup();
// Get backup results
const backupRun = await db('backup_runs')
.orderBy('started_at', 'desc')
.first();
if (!backupRun) {
throw new Error('No backup run found');
}
if (backupRun.status !== 'completed') {
throw new Error(`Backup failed with status: ${backupRun.status}, error: ${backupRun.error_message}`);
}
const duration = Date.now() - startTime;
console.log(` ✓ Backup completed in ${duration}ms`);
console.log(` ✓ Files backed up: ${backupRun.files_backed_up}`);
console.log(` ✓ Total size: ${(backupRun.total_size_bytes / 1024 / 1024).toFixed(2)} MB`);
console.log(` ✓ Manifest: ${backupRun.manifest_path ? 'Generated' : 'Not generated'}`);
return backupRun;
}
async function verifyS3Backup(backupRun) {
const s3Client = new S3Client({
endpoint: options.endpoint,
region: 'us-east-1',
credentials: {
accessKeyId: options.accessKey,
secretAccessKey: options.secretKey
},
forcePathStyle: true
});
// List objects in bucket
const listResponse = await s3Client.send(new ListObjectsV2Command({
Bucket: options.bucket
}));
const objects = listResponse.Contents || [];
console.log(` ✓ Objects in S3: ${objects.length}`);
// Verify key components
const hasBackupFolder = objects.some(obj => obj.Key.includes('backup-'));
const hasManifest = objects.some(obj => obj.Key.includes('backup-manifest'));
const hasSummary = objects.some(obj => obj.Key.includes('backup-summary.json'));
const hasPhotos = objects.some(obj => obj.Key.includes('events/active'));
if (!hasBackupFolder) throw new Error('No backup folder found in S3');
if (!hasManifest) throw new Error('No manifest found in S3');
if (!hasSummary) throw new Error('No summary found in S3');
if (!hasPhotos) throw new Error('No photos found in S3');
console.log(` ✓ Backup structure verified`);
// Download and verify a file
const photoObject = objects.find(obj => obj.Key.includes('photo1.jpg'));
if (photoObject) {
const getResponse = await s3Client.send(new GetObjectCommand({
Bucket: options.bucket,
Key: photoObject.Key
}));
const chunks = [];
for await (const chunk of getResponse.Body) {
chunks.push(chunk);
}
const content = Buffer.concat(chunks);
console.log(` ✓ Downloaded test file: ${photoObject.Key} (${content.length} bytes)`);
}
}
async function testIncrementalBackup(testData) {
// Modify a file
const modifiedFile = path.join(testData.storagePath, 'events/active/wedding-2024/photo1.jpg');
const newContent = crypto.randomBytes(1024 * 1024 + 100); // Slightly larger
await fs.writeFile(modifiedFile, newContent);
console.log(` ✓ Modified test file`);
// Perform incremental backup
const backupRun = await performBackup();
if (backupRun.files_backed_up !== 1) {
throw new Error(`Expected 1 file in incremental backup, got ${backupRun.files_backed_up}`);
}
console.log(` ✓ Incremental backup correctly identified changed file`);
// Verify manifest indicates incremental
if (backupRun.manifest_path) {
const { manifest } = await backupService.getBackupManifest(backupRun.id);
if (!manifest.incremental) {
throw new Error('Manifest does not indicate incremental backup');
}
console.log(` ✓ Manifest correctly marked as incremental`);
}
return backupRun;
}
async function testManifestValidation(backupRun) {
if (!backupRun.manifest_path) {
throw new Error('No manifest path in backup run');
}
const result = await backupService.validateBackupManifest(backupRun.manifest_path);
if (!result.valid) {
throw new Error(`Manifest validation failed: ${result.error}`);
}
console.log(` ✓ Manifest validation passed`);
console.log(` ✓ Manifest version: ${result.manifest.manifest.version}`);
console.log(` ✓ Files in manifest: ${result.manifest.files.count}`);
}
async function testBackupStatus() {
const status = await backupService.getBackupStatus(5);
console.log(` ✓ Backup service running: ${status.isRunning}`);
console.log(` ✓ Backup service healthy: ${status.isHealthy}`);
console.log(` ✓ Recent runs: ${status.recentRuns.length}`);
if (status.lastRun) {
console.log(` ✓ Last run status: ${status.lastRun.status}`);
console.log(` ✓ Manifest valid: ${status.lastRun.manifestValid}`);
}
}
async function cleanupTestData() {
if (!options.cleanup) {
console.log('\n📌 Test data retained for inspection');
console.log(` Storage: ${process.env.STORAGE_PATH}`);
if (options.type === 's3') {
console.log(` S3 Bucket: ${options.bucket}`);
}
return;
}
console.log('\n🧹 Cleaning up test data...');
// Clean storage directory
if (process.env.STORAGE_PATH) {
await fs.rm(process.env.STORAGE_PATH, { recursive: true, force: true });
console.log(' ✓ Removed test storage directory');
}
// Clean S3 bucket if used
if (options.type === 's3') {
const s3Client = new S3Client({
endpoint: options.endpoint,
region: 'us-east-1',
credentials: {
accessKeyId: options.accessKey,
secretAccessKey: options.secretKey
},
forcePathStyle: true
});
try {
// List and delete all objects
const listResponse = await s3Client.send(new ListObjectsV2Command({
Bucket: options.bucket
}));
if (listResponse.Contents && listResponse.Contents.length > 0) {
await s3Client.send(new DeleteObjectsCommand({
Bucket: options.bucket,
Delete: {
Objects: listResponse.Contents.map(obj => ({ Key: obj.Key }))
}
}));
console.log(` ✓ Deleted ${listResponse.Contents.length} objects from S3`);
}
// Delete bucket
await s3Client.send(new DeleteBucketCommand({
Bucket: options.bucket
}));
console.log(` ✓ Deleted S3 bucket: ${options.bucket}`);
} catch (error) {
console.error(` ⚠️ Failed to cleanup S3: ${error.message}`);
}
}
// Clean backup directories
const backupDirs = [
path.join(__dirname, '../test-backup'),
path.join(__dirname, '../test-backup-rsync')
];
for (const dir of backupDirs) {
await fs.rm(dir, { recursive: true, force: true }).catch(() => {});
}
console.log(' ✓ Removed backup directories');
}
// Main test runner
async function main() {
console.log('🚀 Enhanced Backup System Integration Test');
console.log('==========================================');
console.log(`Type: ${options.type}`);
console.log(`Endpoint: ${options.endpoint}`);
console.log(`Bucket: ${options.bucket}`);
console.log('');
let s3Client;
let testData;
try {
// Initialize database
console.log('📦 Initializing database...');
await initDb();
await db.migrate.latest();
console.log(' ✓ Database initialized');
// S3-specific setup
if (options.type === 's3') {
// Test S3 connection
await runTest('S3 Connection Test', testS3Connection);
// Create S3 bucket if needed
s3Client = new S3Client({
endpoint: options.endpoint,
region: 'us-east-1',
credentials: {
accessKeyId: options.accessKey,
secretAccessKey: options.secretKey
},
forcePathStyle: true
});
try {
await s3Client.send(new HeadBucketCommand({ Bucket: options.bucket }));
console.log(`\n📦 Using existing bucket: ${options.bucket}`);
} catch (error) {
if (error.name === 'NotFound') {
await s3Client.send(new CreateBucketCommand({ Bucket: options.bucket }));
console.log(`\n📦 Created new bucket: ${options.bucket}`);
} else {
throw error;
}
}
}
// Setup test data
console.log('\n📁 Setting up test data...');
testData = await setupTestData();
// Configure backup
console.log(`\n⚙️ Configuring ${options.type} backup...`);
await configureBackup(options.type);
// Run tests based on backup type
await runTest('Initial Full Backup', performBackup);
if (options.type === 's3') {
await runTest('Verify S3 Backup Contents', async () => {
const lastRun = await db('backup_runs').orderBy('started_at', 'desc').first();
await verifyS3Backup(lastRun);
});
}
await runTest('Incremental Backup', () => testIncrementalBackup(testData));
await runTest('Manifest Validation', async () => {
const lastRun = await db('backup_runs').orderBy('started_at', 'desc').first();
await testManifestValidation(lastRun);
});
await runTest('Backup Status Check', testBackupStatus);
// Performance test with larger files
if (options.type === 's3') {
await runTest('Large File Backup (10MB)', async () => {
const largeFile = path.join(testData.storagePath, 'events/active/large.jpg');
await fs.writeFile(largeFile, crypto.randomBytes(10 * 1024 * 1024));
await performBackup();
});
}
// Test backup service lifecycle
await runTest('Backup Service Start/Stop', async () => {
await backupService.startBackupService();
console.log(' ✓ Service started');
backupService.stopBackupService();
console.log(' ✓ Service stopped');
});
// Print results summary
console.log('\n📊 Test Results Summary');
console.log('======================');
console.log(`✅ Passed: ${results.passed}`);
console.log(`❌ Failed: ${results.failed}`);
console.log(`⏭️ Skipped: ${results.skipped}`);
console.log(`📋 Total: ${results.tests.length}`);
if (results.failed > 0) {
console.log('\nFailed Tests:');
results.tests
.filter(t => t.status === 'failed')
.forEach(t => console.log(` - ${t.name}: ${t.error}`));
}
} catch (error) {
console.error('\n💥 Fatal error:', error.message);
if (options.verbose) {
console.error(error.stack);
}
results.failed++;
} finally {
// Cleanup
await cleanupTestData();
// Close database
await db.destroy();
// Exit with appropriate code
process.exit(results.failed > 0 ? 1 : 0);
}
}
// Run if called directly
if (require.main === module) {
main().catch(error => {
console.error('Unhandled error:', error);
process.exit(1);
});
}
module.exports = { runTest, skipTest };
-192
View File
@@ -1,192 +0,0 @@
#!/usr/bin/env node
/**
* Test script for backup manifest generator
* Demonstrates all features of the manifest generator
*/
const path = require('path');
const fs = require('fs').promises;
const backupManifest = require('../src/services/backupManifest');
const logger = require('../src/utils/logger');
async function testManifestGeneration() {
console.log('=== Testing Backup Manifest Generator ===\n');
try {
// 1. Generate a full backup manifest
console.log('1. Generating full backup manifest...');
const fullManifestOptions = {
backupType: 'full',
backupPath: '/backup/full/2025-01-21',
files: [
{
path: '/storage/events/active/wedding-smith-2025/DSC_001.jpg',
relativePath: 'events/active/wedding-smith-2025/DSC_001.jpg',
size: 2456789,
modified: new Date('2025-01-20T10:30:00Z'),
checksum: 'a1b2c3d4e5f6789012345678901234567890123456789012345678901234567890',
permissions: '644'
},
{
path: '/storage/events/active/wedding-smith-2025/DSC_002.jpg',
relativePath: 'events/active/wedding-smith-2025/DSC_002.jpg',
size: 2156789,
modified: new Date('2025-01-20T10:31:00Z'),
checksum: 'b2c3d4e5f67890123456789012345678901234567890123456789012345678901',
permissions: '644'
},
{
path: '/storage/thumbnails/wedding-smith-2025/thumb_DSC_001.jpg',
relativePath: 'thumbnails/wedding-smith-2025/thumb_DSC_001.jpg',
size: 45678,
modified: new Date('2025-01-20T10:35:00Z'),
checksum: 'c3d4e5f678901234567890123456789012345678901234567890123456789012',
permissions: '644'
}
],
databaseInfo: {
type: 'sqlite',
backupFile: 'database-backup-20250121-103000.sql.gz',
size: 1048576,
checksum: 'd4e5f6789012345678901234567890123456789012345678901234567890123',
tables: {
events: 156,
photos: 4523,
access_logs: 12456,
admin_users: 3
},
rowCounts: {
events: 156,
photos: 4523,
access_logs: 12456,
admin_users: 3
}
},
format: 'json',
customMetadata: {
operator: 'admin@example.com',
reason: 'Scheduled daily backup',
retentionDays: 30,
compressionType: 'gzip'
}
};
const fullManifest = await backupManifest.generateManifest(fullManifestOptions);
// Save in both formats
const jsonPath = path.join(__dirname, 'test-manifest-full.json');
const yamlPath = path.join(__dirname, 'test-manifest-full.yaml');
await backupManifest.saveManifest(fullManifest, jsonPath, 'json');
await backupManifest.saveManifest(fullManifest, yamlPath, 'yaml');
console.log('✓ Full backup manifest generated and saved\n');
// 2. Generate summary report
console.log('2. Generating summary report...');
const summaryReport = backupManifest.generateSummaryReport(fullManifest);
console.log(summaryReport);
console.log('\n');
// 3. Load and validate manifest
console.log('3. Loading and validating manifest...');
const loadedManifest = await backupManifest.loadManifest(jsonPath);
console.log('✓ Manifest loaded and validated successfully\n');
// 4. Generate incremental backup manifest
console.log('4. Generating incremental backup manifest...');
const incrementalOptions = {
backupType: 'incremental',
backupPath: '/backup/incremental/2025-01-22',
parentBackupId: fullManifest.backup.id,
files: [
// Original files with same checksums (unchanged)
fullManifestOptions.files[0],
fullManifestOptions.files[2],
// Modified file
{
...fullManifestOptions.files[1],
size: 2256789,
modified: new Date('2025-01-21T14:00:00Z'),
checksum: 'e5f678901234567890123456789012345678901234567890123456789012345'
},
// New file
{
path: '/storage/events/active/wedding-smith-2025/DSC_003.jpg',
relativePath: 'events/active/wedding-smith-2025/DSC_003.jpg',
size: 2356789,
modified: new Date('2025-01-21T14:30:00Z'),
checksum: 'f6789012345678901234567890123456789012345678901234567890123456',
permissions: '644'
}
],
databaseInfo: {
...fullManifestOptions.databaseInfo,
size: 1148576,
checksum: 'g7890123456789012345678901234567890123456789012345678901234567',
rowCounts: {
events: 158,
photos: 4567,
access_logs: 12789,
admin_users: 3
}
}
};
const incrementalManifest = await backupManifest.generateIncrementalManifest(
incrementalOptions,
fullManifest
);
const incrementalJsonPath = path.join(__dirname, 'test-manifest-incremental.json');
await backupManifest.saveManifest(incrementalManifest, incrementalJsonPath, 'json');
console.log('✓ Incremental backup manifest generated');
console.log(` - Added files: ${incrementalManifest.incremental.changes.added_files_count}`);
console.log(` - Modified files: ${incrementalManifest.incremental.changes.modified_files_count}`);
console.log(` - Deleted files: ${incrementalManifest.incremental.changes.deleted_files_count}`);
console.log(` - Size difference: ${(incrementalManifest.incremental.changes.size_difference / 1024).toFixed(2)} KB\n`);
// 5. Compare manifests
console.log('5. Comparing manifests...');
const comparison = backupManifest.compareManifests(incrementalManifest, fullManifest);
console.log('Comparison results:');
console.log(` - Added: ${comparison.added_files.length} files`);
console.log(` - Modified: ${comparison.modified_files.length} files`);
console.log(` - Deleted: ${comparison.deleted_files.length} files`);
console.log(` - Unchanged: ${comparison.unchanged_files.length} files`);
console.log(` - Database changed: ${comparison.database_changes.checksum_changed ? 'Yes' : 'No'}\n`);
// 6. Test manifest integrity
console.log('6. Testing manifest integrity...');
// Corrupt the manifest
const corruptedManifest = JSON.parse(JSON.stringify(incrementalManifest));
corruptedManifest.files.manifest[0].size = 9999999; // Change a file size
try {
backupManifest.validateManifest(corruptedManifest);
console.log('✗ Validation should have failed for corrupted manifest');
} catch (error) {
console.log('✓ Correctly detected corrupted manifest:', error.message);
}
console.log('\n=== All tests completed successfully! ===');
// Clean up test files
await fs.unlink(jsonPath).catch(() => {});
await fs.unlink(yamlPath).catch(() => {});
await fs.unlink(incrementalJsonPath).catch(() => {});
} catch (error) {
console.error('Test failed:', error);
logger.error('Manifest test failed:', error);
process.exit(1);
}
}
// Run tests
testManifestGeneration().catch(console.error);
-46
View File
@@ -1,46 +0,0 @@
#!/usr/bin/env node
require('dotenv').config();
const { initializeDatabase } = require('../src/database/db');
const { runBackup, getBackupStatus } = require('../src/services/backupService');
const logger = require('../src/utils/logger');
async function testBackupService() {
try {
console.log('Testing backup service...\n');
// Initialize database
await initializeDatabase();
// Get current backup status
console.log('Getting backup status...');
const statusBefore = await getBackupStatus();
console.log('Last run:', statusBefore.lastRun ? statusBefore.lastRun.started_at : 'Never');
console.log('Is healthy:', statusBefore.isHealthy);
console.log('');
// Run backup
console.log('Running backup...');
await runBackup();
// Get status after backup
console.log('\nGetting status after backup...');
const statusAfter = await getBackupStatus();
console.log('Last run:', statusAfter.lastRun ? statusAfter.lastRun.started_at : 'Never');
console.log('Status:', statusAfter.lastRun ? statusAfter.lastRun.status : 'Unknown');
console.log('Files backed up:', statusAfter.lastRun ? statusAfter.lastRun.files_backed_up : 0);
console.log('Total size:', statusAfter.lastRun ? `${(statusAfter.lastRun.total_size_bytes / 1024 / 1024).toFixed(2)} MB` : '0 MB');
if (statusAfter.lastRun && statusAfter.lastRun.error_message) {
console.log('Error:', statusAfter.lastRun.error_message);
}
console.log('\nBackup test completed!');
process.exit(0);
} catch (error) {
console.error('Test failed:', error);
process.exit(1);
}
}
testBackupService();
-52
View File
@@ -1,52 +0,0 @@
/**
* Test script to verify CMS and email formatting improvements
*/
const { formatWelcomeMessage, nl2br } = require('../src/utils/formatters');
console.log('Testing CMS and Email Formatting Improvements\n');
// Test 1: Basic line break conversion
console.log('Test 1: Basic line break conversion');
const basicText = `Hello,
This is line 1.
This is line 2.
This is line 4 with an extra break.`;
console.log('Input:');
console.log(basicText);
console.log('\nOutput (nl2br):');
console.log(nl2br(basicText));
console.log('\n---\n');
// Test 2: Welcome message formatting
console.log('Test 2: Welcome message formatting');
const welcomeMessage = `Dear guests,
We're so excited to share these special moments with you!
Please note:
- Download your photos before the expiration date
- The password is case-sensitive
- Contact us if you have any issues
Thank you for being part of our special day!
Best regards,
Sarah & John`;
console.log('Input:');
console.log(welcomeMessage);
console.log('\nOutput (formatWelcomeMessage):');
console.log(formatWelcomeMessage(welcomeMessage));
console.log('\n---\n');
// Test 3: Empty and edge cases
console.log('Test 3: Edge cases');
console.log('Empty string:', formatWelcomeMessage(''));
console.log('Null:', formatWelcomeMessage(null));
console.log('Only spaces:', formatWelcomeMessage(' \n \n '));
console.log('Single line:', formatWelcomeMessage('This is a single line message'));
console.log('\nAll tests completed!');
-325
View File
@@ -1,325 +0,0 @@
/**
* Test script for the restore service
*
* This script demonstrates the restore service functionality with safety checks
*
* Usage:
* node scripts/test-restore-service.js [options]
*
* Options:
* --dry-run Perform validation only without actual restore
* --force Force restore even with warnings
* --type Restore type: full, database, files, selective (default: full)
* --source Backup source path or S3 URL
* --manifest Path to backup manifest
*/
require('dotenv').config();
const { restoreService } = require('../src/services/restoreService');
const { db } = require('../src/database/db');
const logger = require('../src/utils/logger');
const path = require('path');
const fs = require('fs').promises;
// Parse command line arguments
const args = process.argv.slice(2);
const options = {
dryRun: args.includes('--dry-run'),
force: args.includes('--force'),
restoreType: 'full',
source: null,
manifestPath: null
};
// Parse restore type
const typeIndex = args.indexOf('--type');
if (typeIndex !== -1 && args[typeIndex + 1]) {
options.restoreType = args[typeIndex + 1];
}
// Parse source
const sourceIndex = args.indexOf('--source');
if (sourceIndex !== -1 && args[sourceIndex + 1]) {
options.source = args[sourceIndex + 1];
}
// Parse manifest
const manifestIndex = args.indexOf('--manifest');
if (manifestIndex !== -1 && args[manifestIndex + 1]) {
options.manifestPath = args[manifestIndex + 1];
}
async function testRestore() {
console.log('=== PicPeak Restore Service Test ===\n');
try {
// If no source/manifest provided, try to find a recent backup
if (!options.source || !options.manifestPath) {
console.log('No backup source specified. Looking for recent backups...\n');
const recentBackup = await db('backup_runs')
.where('status', 'completed')
.whereNotNull('manifest_path')
.orderBy('completed_at', 'desc')
.first();
if (!recentBackup) {
console.error('❌ No completed backups found in the database');
console.log('\nPlease run a backup first or specify --source and --manifest');
process.exit(1);
}
console.log(`Found recent backup from ${recentBackup.completed_at}`);
console.log(`Backup ID: ${recentBackup.manifest_id}`);
console.log(`Files backed up: ${recentBackup.files_backed_up}`);
console.log(`Total size: ${(recentBackup.total_size_bytes / 1024 / 1024).toFixed(2)} MB`);
console.log(`Manifest: ${recentBackup.manifest_path}\n`);
// For this test, we'll create a mock scenario
console.log('⚠️ This is a TEST MODE - using mock data for safety\n');
// Create test backup directory
const testBackupDir = path.join(__dirname, '../temp/test-backup');
await fs.mkdir(testBackupDir, { recursive: true });
// Create test manifest
const testManifest = {
manifest: {
version: '2.0',
created: new Date().toISOString(),
generator: 'Test Script',
format: 'json'
},
backup: {
id: 'test-backup-' + Date.now(),
type: 'full',
timestamp: new Date().toISOString(),
path: testBackupDir,
parent_backup_id: null,
retention_days: 30
},
system: {
hostname: require('os').hostname(),
platform: process.platform,
os_release: require('os').release(),
architecture: require('os').arch()
},
application: {
name: 'PicPeak',
version: require('../package.json').version,
node_version: process.version,
environment: 'test'
},
files: {
count: 0,
total_size: 0,
checksums: {},
manifest: []
},
database: {
type: process.env.DB_TYPE === 'postgresql' ? 'postgresql' : 'sqlite',
backup_file: null,
size: 0,
checksum: null,
tables: {},
row_counts: {}
},
verification: {
total_checksum: null,
file_count_check: 0,
size_check: 0,
integrity_timestamp: new Date().toISOString()
},
metadata: {
test_mode: true
}
};
// Calculate checksum
const crypto = require('crypto');
const manifestCopy = JSON.parse(JSON.stringify(testManifest));
delete manifestCopy.verification.total_checksum;
testManifest.verification.total_checksum = crypto
.createHash('sha256')
.update(JSON.stringify(manifestCopy, Object.keys(manifestCopy).sort()))
.digest('hex');
// Save test manifest
const testManifestPath = path.join(testBackupDir, 'test-manifest.json');
await fs.writeFile(testManifestPath, JSON.stringify(testManifest, null, 2));
options.source = testBackupDir;
options.manifestPath = testManifestPath;
}
// Display restore options
console.log('Restore Options:');
console.log(`- Type: ${options.restoreType}`);
console.log(`- Source: ${options.source}`);
console.log(`- Manifest: ${options.manifestPath}`);
console.log(`- Dry Run: ${options.dryRun ? 'Yes' : 'No'}`);
console.log(`- Force: ${options.force ? 'Yes' : 'No'}`);
console.log('');
// Add S3 config if source is S3
if (options.source.startsWith('s3://')) {
options.s3Config = {
accessKeyId: process.env.BACKUP_S3_ACCESS_KEY,
secretAccessKey: process.env.BACKUP_S3_SECRET_KEY,
region: process.env.BACKUP_S3_REGION || 'us-east-1',
endpoint: process.env.BACKUP_S3_ENDPOINT
};
if (!options.s3Config.accessKeyId || !options.s3Config.secretAccessKey) {
console.error('❌ S3 credentials not configured in environment');
process.exit(1);
}
}
// Confirm before proceeding (unless dry run)
if (!options.dryRun) {
console.log('⚠️ WARNING: This will restore data from the backup!');
console.log('⚠️ Current data may be overwritten!');
console.log('');
console.log('Press Ctrl+C to cancel, or wait 5 seconds to continue...');
await new Promise(resolve => setTimeout(resolve, 5000));
}
console.log('\nStarting restore operation...\n');
// Perform restore
const result = await restoreService.restore(options);
if (options.dryRun) {
console.log('\n=== DRY RUN RESULTS ===\n');
console.log('Validation:');
console.log(`- Valid: ${result.validation.isValid ? '✅ Yes' : '❌ No'}`);
if (result.validation.errors.length > 0) {
console.log('- Errors:');
result.validation.errors.forEach(err => console.log(`${err}`));
}
if (result.validation.warnings.length > 0) {
console.log('- Warnings:');
result.validation.warnings.forEach(warn => console.log(` ⚠️ ${warn}`));
}
console.log('\nDisk Space:');
console.log(`- Required: ${result.spaceCheck.requiredFormatted}`);
console.log(`- Available: ${result.spaceCheck.availableFormatted}`);
console.log(`- Sufficient: ${result.spaceCheck.hasEnoughSpace ? '✅ Yes' : '❌ No'}`);
} else {
console.log('\n=== RESTORE RESULTS ===\n');
console.log(`Status: ${result.success ? '✅ SUCCESS' : '❌ FAILED'}`);
console.log(`Duration: ${result.duration}s`);
if (result.result) {
console.log('\nItems Restored:');
if (result.result.databaseRestored !== undefined) {
console.log(`- Database: ${result.result.databaseRestored ? '✅' : '❌'}`);
}
if (result.result.filesRestored !== undefined) {
console.log(`- Files: ${result.result.filesRestored}`);
}
if (result.result.errors && result.result.errors.length > 0) {
console.log('- Errors:');
result.result.errors.forEach(err => console.log(`${err}`));
}
}
if (result.verification) {
console.log('\nVerification:');
console.log(`- Valid: ${result.verification.isValid ? '✅ Yes' : '❌ No'}`);
if (result.verification.errors.length > 0) {
console.log('- Errors:');
result.verification.errors.forEach(err => console.log(`${err}`));
}
}
if (result.preRestoreBackup) {
console.log('\nSafety Backup:');
console.log(`- Location: ${result.preRestoreBackup}`);
console.log('- This backup can be used to rollback if needed');
}
}
// Show recent log entries
console.log('\nRecent Log Entries:');
result.logs.slice(-10).forEach(log => {
const icon = log.level === 'error' ? '❌' : log.level === 'warn' ? '⚠️ ' : '️ ';
console.log(`${icon} [${log.timestamp}] ${log.message}`);
});
// Clean up test files
if (options.source && options.source.includes('test-backup')) {
await fs.rmdir(path.dirname(options.source), { recursive: true }).catch(() => {});
}
} catch (error) {
console.error('\n❌ Restore operation failed:', error.message);
// Show logs if available
if (restoreService.restoreLog && restoreService.restoreLog.length > 0) {
console.log('\nError Log:');
restoreService.restoreLog.slice(-10).forEach(log => {
if (log.level === 'error' || log.level === 'warn') {
console.log(`[${log.timestamp}] ${log.level.toUpperCase()}: ${log.message}`);
}
});
}
process.exit(1);
}
// Cleanup
await db.destroy();
process.exit(0);
}
// Show help if requested
if (args.includes('--help') || args.includes('-h')) {
console.log(`
PicPeak Restore Service Test
This script tests the restore service functionality with safety checks.
Usage:
node scripts/test-restore-service.js [options]
Options:
--dry-run Perform validation only without actual restore
--force Force restore even with warnings
--type Restore type: full, database, files, selective (default: full)
--source Backup source path or S3 URL
--manifest Path to backup manifest
--help Show this help message
Examples:
# Dry run with automatic backup selection
node scripts/test-restore-service.js --dry-run
# Full restore from specific backup
node scripts/test-restore-service.js --source /backup/2024-01-20 --manifest /backup/2024-01-20/manifest.json
# Database-only restore with force
node scripts/test-restore-service.js --type database --force --source /backup/2024-01-20 --manifest /backup/2024-01-20/manifest.json
# Restore from S3
node scripts/test-restore-service.js --source s3://my-bucket/backups/2024-01-20 --manifest s3://my-bucket/backups/2024-01-20/manifest.json
Safety Features:
- Pre-restore validation checks compatibility and warns about potential issues
- Automatic pre-restore backup is created (unless skipped)
- Post-restore verification ensures data integrity
- Rollback capability if restore fails
- Detailed logging of all operations
`);
process.exit(0);
}
// Run the test
testRestore();
-95
View File
@@ -1,95 +0,0 @@
#!/usr/bin/env node
/**
* Test script to verify security logging is working correctly
* Run with: node scripts/test-security-logging.js
*/
require('dotenv').config({ path: '../.env' });
const logger = require('../src/utils/logger');
console.log('Testing Security Logging...\n');
// Test 1: Basic logging
console.log('1. Testing basic logging levels:');
logger.info('Test info message', { test: true });
logger.warn('Test warning message', { test: true });
logger.error('Test error message', { test: true });
// Test 2: Security event logging
console.log('\n2. Testing security event logging:');
// Rate limit exceeded
logger.warn('Rate limit exceeded', {
ip: '192.168.1.100',
path: '/api/admin/login',
method: 'POST',
authenticated: false,
userAgent: 'Mozilla/5.0 Test',
timestamp: new Date().toISOString(),
rateLimitInfo: {
limit: 5,
current: 6,
remaining: 0,
resetTime: new Date(Date.now() + 900000).toISOString()
}
});
// Auth rate limit
logger.warn('Auth rate limit exceeded', {
ip: '192.168.1.101',
path: '/api/auth/admin/login',
method: 'POST',
userAgent: 'Mozilla/5.0 Test',
authType: 'admin',
timestamp: new Date().toISOString()
});
// Failed login
logger.warn('Failed login attempt', {
username: 'testuser',
ip: '192.168.1.102',
userAgent: 'Mozilla/5.0 Test',
reason: 'invalid_credentials',
timestamp: new Date().toISOString()
});
// JWT validation failure
logger.warn('JWT validation failed', {
ip: '192.168.1.103',
path: '/api/admin/events',
method: 'GET',
userAgent: 'Mozilla/5.0 Test',
error: 'TokenExpiredError',
message: 'jwt expired',
timestamp: new Date().toISOString()
});
// Account lockout
logger.warn('Login attempt on locked account', {
username: 'lockeduser',
ip: '192.168.1.104',
remainingLockTime: 1200,
timestamp: new Date().toISOString()
});
// Suspicious activity
logger.warn('Suspicious login activity detected', {
username: 'suspicioususer',
ips: ['192.168.1.105', '192.168.1.106', '192.168.1.107'],
timeWindow: '15 minutes',
timestamp: new Date().toISOString()
});
console.log('\n3. Check log files:');
console.log('- logs/security.log - Should contain all security warnings');
console.log('- logs/error.log - Should contain error messages');
console.log('- logs/combined.log - Should contain all messages');
console.log('\n✅ Security logging test complete!');
console.log('Review the log files to ensure all events are properly captured.');
// Give logger time to flush
setTimeout(() => {
process.exit(0);
}, 1000);
+35 -73
View File
@@ -4,27 +4,19 @@ require('dotenv').config();
const { validateEnvironment } = require('./src/config/validateEnv');
validateEnvironment();
// Initialize logger early to capture startup logs
const logger = require('./src/utils/logger');
logger.info('Server starting up', {
nodeVersion: process.version,
environment: process.env.NODE_ENV || 'development',
timestamp: new Date().toISOString()
});
const express = require('express');
const helmet = require('helmet');
const cors = require('cors');
const rateLimit = require('express-rate-limit');
const jwt = require('jsonwebtoken');
const path = require('path');
const { initializeDatabase, db } = require('./src/database/db');
const { startFileWatcher } = require('./src/services/fileWatcher');
const { startExpirationChecker } = require('./src/services/expirationChecker');
const { initializeTransporter, startEmailQueueProcessor } = require('./src/services/emailProcessor');
const { startBackupService } = require('./src/services/backupService');
const { startScheduledBackups } = require('./src/services/databaseBackup');
const { maintenanceMiddleware } = require('./src/middleware/maintenance');
const { sessionTimeoutMiddleware } = require('./src/middleware/sessionTimeout');
const { createRateLimiter, createAuthRateLimiter } = require('./src/services/rateLimitService');
const logger = require('./src/utils/logger');
// Import routes
const authRoutes = require('./src/routes/auth-enhanced');
@@ -101,23 +93,37 @@ const corsOptions = {
app.use(cors(corsOptions));
// Initialize rate limiters (they will be created dynamically)
let generalRateLimiter;
let authRateLimiter;
// Rate limiting with admin bypass
const limiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: process.env.NODE_ENV === 'development' ? 1000 : 100, // More lenient in development
skip: (req) => {
// Skip rate limiting for authenticated admin users
if (req.path.startsWith('/api/admin/') && req.headers.authorization) {
const token = req.headers.authorization.replace('Bearer ', '');
try {
const decoded = jwt.verify(token, process.env.JWT_SECRET);
return decoded.type === 'admin';
} catch (err) {
return false;
}
}
// Also skip rate limiting for public settings endpoint in development
if (process.env.NODE_ENV === 'development' && req.path === '/api/public/settings') {
return true;
}
return false;
}
});
// Function to initialize rate limiters
async function initializeRateLimiters() {
generalRateLimiter = await createRateLimiter();
authRateLimiter = await createAuthRateLimiter();
// Apply rate limiting
app.use('/api/', generalRateLimiter);
app.use('/api/auth', authRateLimiter);
app.use('/api/gallery/:slug/verify', authRateLimiter);
app.use('/api/admin/auth/login', authRateLimiter);
}
const authLimiter = rateLimit({
windowMs: 15 * 60 * 1000,
max: 5 // limit auth attempts
});
// Note: Rate limiters will be initialized after database connection
// Apply rate limiting - admin routes check will skip for valid admin tokens
app.use('/api/', limiter);
app.use('/api/auth', authLimiter);
// Body parsing middleware with increased limits for large uploads
app.use(express.json({ limit: '100mb' }));
@@ -152,28 +158,6 @@ app.use('/thumbnails', require('./src/middleware/photoAuth'), setCorsHeaders, se
// Static file serving for uploads (public - logos, favicons)
app.use('/uploads', setCorsHeaders, secureStatic(path.join(storagePath, 'uploads')));
// Debug endpoint to check IP detection (only in development)
if (process.env.NODE_ENV === 'development') {
app.get('/api/debug/ip', (req, res) => {
const clientIp = req.headers['x-forwarded-for']?.split(',')[0]?.trim() ||
req.headers['x-real-ip'] ||
req.connection.remoteAddress ||
req.ip;
res.json({
detectedIp: clientIp,
reqIp: req.ip,
headers: {
'x-forwarded-for': req.headers['x-forwarded-for'],
'x-real-ip': req.headers['x-real-ip'],
'x-forwarded-proto': req.headers['x-forwarded-proto'],
'x-forwarded-host': req.headers['x-forwarded-host']
},
trustProxy: app.get('trust proxy')
});
});
}
// Health check endpoint
app.get('/health', async (req, res) => {
try {
@@ -203,10 +187,6 @@ app.use('/api/gallery', galleryRoutes);
app.use('/api/admin', adminRoutes);
app.use('/api/admin/auth', adminAuthRoutes);
app.use('/api/admin/system', require('./src/routes/adminSystem'));
app.use('/api/admin/backup', require('./src/routes/adminBackup'));
app.use('/api/admin/database-backup', require('./src/routes/adminDatabaseBackup'));
app.use('/api/admin/feedback', require('./src/routes/adminFeedback'));
app.use('/api/gallery', require('./src/routes/galleryFeedback'));
app.use('/api/public/settings', require('./src/routes/publicSettings'));
app.use('/api/public', require('./src/routes/publicCMS'));
app.use('/api/images', require('./src/routes/protectedImages'));
@@ -223,21 +203,9 @@ async function startServer() {
// Initialize database
await initializeDatabase();
// Initialize rate limiters after database is ready
await initializeRateLimiters();
logger.info('Rate limiters initialized with database configuration');
// Initialize auth security cleanup job
const { initializeCleanupJob } = require('./src/utils/authSecurity');
initializeCleanupJob();
// Initialize temp upload cleanup job
const { cleanupTempUploads } = require('./src/utils/cleanupTempUploads');
// Run cleanup on startup
cleanupTempUploads();
// Schedule periodic cleanup every hour
setInterval(cleanupTempUploads, 60 * 60 * 1000);
logger.info('Temp upload cleanup scheduled');
// Initialize auth security cleanup job
const { initializeCleanupJob } = require('./src/utils/authSecurity');
initializeCleanupJob();
// Start file watcher
startFileWatcher();
@@ -249,12 +217,6 @@ async function startServer() {
await initializeTransporter();
startEmailQueueProcessor();
// Start backup service
await startBackupService();
// Start database backup service
await startScheduledBackups();
app.listen(PORT, () => {
logger.info(`Server running on port ${PORT}`);
logger.info(`Admin interface: ${process.env.ADMIN_URL || 'http://localhost:3000'}`);
+1 -1
View File
@@ -34,7 +34,7 @@ function validateEnvironment() {
if (name === 'JWT_SECRET' && value) {
// Check for the insecure default value
if (value === 'your-secret-key') {
errors.push('CRITICAL: JWT_SECRET is set to the insecure default value. Please set a secure secret key.');
errors.push(`CRITICAL: JWT_SECRET is set to the insecure default value. Please set a secure secret key.`);
}
// Check minimum length (should be at least 32 characters for security)
+4 -31
View File
@@ -4,33 +4,6 @@ const knexConfig = require('../../knexfile');
// Create database connection with built-in retry logic
const db = knex(knexConfig);
// Connection retry configuration
const MAX_RETRIES = 3;
const RETRY_DELAY = 1000;
// Wrapper function to handle connection retries
async function withRetry(queryFn, retries = MAX_RETRIES) {
for (let i = 0; i < retries; i++) {
try {
return await queryFn();
} catch (error) {
const isConnectionError = error.message && (
error.message.includes('Connection terminated unexpectedly') ||
error.message.includes('Connection ended unexpectedly') ||
error.message.includes('ECONNREFUSED') ||
error.message.includes('ETIMEDOUT')
);
if (isConnectionError && i < retries - 1) {
console.log(`Database connection error, retrying in ${RETRY_DELAY}ms... (attempt ${i + 1}/${retries})`);
await new Promise(resolve => setTimeout(resolve, RETRY_DELAY * (i + 1)));
continue;
}
throw error;
}
}
}
async function initializeDatabase() {
// Events table
const hasEventsTable = await db.schema.hasTable('events');
@@ -86,9 +59,9 @@ async function initializeDatabase() {
)
`);
await db.raw('INSERT INTO events_new SELECT * FROM events');
await db.raw('DROP TABLE events');
await db.raw('ALTER TABLE events_new RENAME TO events');
await db.raw(`INSERT INTO events_new SELECT * FROM events`);
await db.raw(`DROP TABLE events`);
await db.raw(`ALTER TABLE events_new RENAME TO events`);
} catch (error) {
// If the migration fails, it might already have been applied
console.log('Color theme migration may have already been applied');
@@ -252,4 +225,4 @@ async function logActivity(activityType, metadata = {}, eventId = null, actor =
}
}
module.exports = { db, initializeDatabase, logActivity, withRetry };
module.exports = { db, initializeDatabase, logActivity };
+1 -60
View File
@@ -1,83 +1,24 @@
const jwt = require('jsonwebtoken');
const { db } = require('../database/db');
const { formatBoolean } = require('../utils/dbCompat');
const logger = require('../utils/logger');
async function adminAuth(req, res, next) {
try {
const token = req.headers.authorization?.split(' ')[1];
if (!token) {
const clientIp = req.headers['x-forwarded-for']?.split(',')[0]?.trim() ||
req.headers['x-real-ip'] ||
req.connection.remoteAddress ||
req.ip;
logger.warn('Admin auth attempt without token', {
ip: clientIp,
path: req.path,
method: req.method,
userAgent: req.headers['user-agent']
});
return res.status(401).json({ error: 'No token provided' });
}
let decoded;
try {
decoded = jwt.verify(token, process.env.JWT_SECRET);
} catch (jwtError) {
const clientIp = req.headers['x-forwarded-for']?.split(',')[0]?.trim() ||
req.headers['x-real-ip'] ||
req.connection.remoteAddress ||
req.ip;
logger.warn('JWT validation failed', {
ip: clientIp,
path: req.path,
method: req.method,
userAgent: req.headers['user-agent'],
error: jwtError.name,
message: jwtError.message,
timestamp: new Date().toISOString()
});
if (jwtError.name === 'TokenExpiredError') {
return res.status(401).json({ error: 'Token expired' });
}
return res.status(401).json({ error: 'Invalid token' });
}
const decoded = jwt.verify(token, process.env.JWT_SECRET);
const admin = await db('admin_users').where({ id: decoded.id, is_active: formatBoolean(true) }).first();
if (!admin) {
const clientIp = req.headers['x-forwarded-for']?.split(',')[0]?.trim() ||
req.headers['x-real-ip'] ||
req.connection.remoteAddress ||
req.ip;
logger.warn('Admin auth failed - user not found or inactive', {
ip: clientIp,
userId: decoded.id,
path: req.path,
method: req.method,
timestamp: new Date().toISOString()
});
return res.status(401).json({ error: 'Invalid token' });
}
req.admin = admin;
next();
} catch (error) {
const clientIp = req.headers['x-forwarded-for']?.split(',')[0]?.trim() ||
req.headers['x-real-ip'] ||
req.connection.remoteAddress ||
req.ip;
logger.error('Admin auth middleware error', {
ip: clientIp,
path: req.path,
error: error.message,
stack: error.stack,
timestamp: new Date().toISOString()
});
res.status(401).json({ error: 'Invalid token' });
}
}
-236
View File
@@ -1,236 +0,0 @@
const crypto = require('crypto');
const { db } = require('../database/db');
const logger = require('../utils/logger');
/**
* Generate a unique identifier for the guest
*/
function generateGuestIdentifier(req) {
const ip = req.ip || req.connection.remoteAddress || 'unknown';
const userAgent = req.headers['user-agent'] || 'unknown';
return crypto
.createHash('sha256')
.update(`${ip}:${userAgent}`)
.digest('hex');
}
/**
* Get rate limit settings from app_settings
*/
async function getRateLimitSettings() {
try {
const settings = await db('app_settings')
.where('setting_key', 'feedback_rate_limits')
.first();
if (settings && settings.setting_value) {
return JSON.parse(settings.setting_value);
}
// Default settings
return {
rating: { max: 100, window: 3600 }, // 100 ratings per hour
comment: { max: 20, window: 3600 }, // 20 comments per hour
like: { max: 200, window: 3600 }, // 200 likes per hour
favorite: { max: 100, window: 3600 } // 100 favorites per hour
};
} catch (error) {
logger.error('Error getting rate limit settings:', error);
// Return defaults on error
return {
rating: { max: 100, window: 3600 },
comment: { max: 20, window: 3600 },
like: { max: 200, window: 3600 },
favorite: { max: 100, window: 3600 }
};
}
}
/**
* Check if action is rate limited
*/
async function checkRateLimit(identifier, eventId, actionType) {
try {
const settings = await getRateLimitSettings();
const limit = settings[actionType] || { max: 100, window: 3600 };
// Clean old entries (older than window)
const cutoff = new Date(Date.now() - limit.window * 1000);
await db('feedback_rate_limits')
.where('window_start', '<', cutoff)
.delete();
// Count recent actions
const recentActions = await db('feedback_rate_limits')
.where({
identifier,
event_id: eventId,
action_type: actionType
})
.where('window_start', '>', cutoff)
.sum('action_count as total')
.first();
const currentCount = recentActions?.total || 0;
if (currentCount >= limit.max) {
return {
limited: true,
limit: limit.max,
window: limit.window,
current: currentCount,
resetAt: new Date(Date.now() + limit.window * 1000)
};
}
return {
limited: false,
limit: limit.max,
window: limit.window,
current: currentCount,
remaining: limit.max - currentCount
};
} catch (error) {
logger.error('Error checking rate limit:', error);
// Allow action on error to avoid blocking legitimate users
return { limited: false };
}
}
/**
* Record an action for rate limiting
*/
async function recordAction(identifier, eventId, actionType) {
try {
await db('feedback_rate_limits').insert({
identifier,
event_id: eventId,
action_type: actionType,
action_count: 1,
window_start: new Date()
});
} catch (error) {
logger.error('Error recording rate limit action:', error);
}
}
/**
* Middleware factory for feedback rate limiting
*/
function feedbackRateLimit(actionType) {
return async (req, res, next) => {
try {
// Extract event ID from params or body
const eventId = req.params.eventId || req.body?.event_id;
if (!eventId) {
return res.status(400).json({ error: 'Event ID required' });
}
// Generate guest identifier
const identifier = generateGuestIdentifier(req);
req.guestIdentifier = identifier;
// Check rate limit
const rateLimitStatus = await checkRateLimit(identifier, eventId, actionType);
// Set rate limit headers
res.set({
'X-RateLimit-Limit': rateLimitStatus.limit,
'X-RateLimit-Remaining': rateLimitStatus.remaining || 0,
'X-RateLimit-Reset': rateLimitStatus.resetAt ? rateLimitStatus.resetAt.toISOString() : new Date().toISOString()
});
if (rateLimitStatus.limited) {
logger.warn(`Rate limit exceeded for ${actionType}`, {
identifier: identifier.substring(0, 16) + '...',
eventId,
actionType
});
return res.status(429).json({
error: 'Too many requests',
message: `Rate limit exceeded. Please try again later.`,
retryAfter: rateLimitStatus.window
});
}
// Record the action after successful processing
res.on('finish', async () => {
if (res.statusCode >= 200 && res.statusCode < 300) {
await recordAction(identifier, eventId, actionType);
}
});
next();
} catch (error) {
logger.error('Error in rate limit middleware:', error);
// Allow request to proceed on error
next();
}
};
}
/**
* IP-based rate limiting for more strict control
*/
function strictRateLimit(options = {}) {
const {
windowMs = 15 * 60 * 1000, // 15 minutes
max = 100, // limit each IP to 100 requests per windowMs
message = 'Too many requests from this IP, please try again later.',
skipSuccessfulRequests = false
} = options;
const store = new Map();
// Clean up old entries periodically
setInterval(() => {
const now = Date.now();
for (const [key, data] of store.entries()) {
if (data.resetTime < now) {
store.delete(key);
}
}
}, windowMs);
return (req, res, next) => {
const ip = req.ip || req.connection.remoteAddress;
const now = Date.now();
const resetTime = now + windowMs;
let data = store.get(ip);
if (!data || data.resetTime < now) {
data = {
count: 0,
resetTime
};
store.set(ip, data);
}
if (data.count >= max) {
return res.status(429).json({
error: 'Too many requests',
message,
retryAfter: Math.ceil((data.resetTime - now) / 1000)
});
}
if (!skipSuccessfulRequests || res.statusCode >= 400) {
data.count++;
}
res.setHeader('X-RateLimit-Limit', max);
res.setHeader('X-RateLimit-Remaining', Math.max(0, max - data.count));
res.setHeader('X-RateLimit-Reset', new Date(data.resetTime).toISOString());
next();
};
}
module.exports = {
feedbackRateLimit,
strictRateLimit,
generateGuestIdentifier,
checkRateLimit,
recordAction
};
+8 -10
View File
@@ -1,5 +1,5 @@
const jwt = require('jsonwebtoken');
const { db, withRetry } = require('../database/db');
const { db } = require('../database/db');
const { formatBoolean } = require('../utils/dbCompat');
// Middleware to verify gallery access
@@ -11,15 +11,13 @@ async function verifyGalleryAccess(req, res, next) {
}
const decoded = jwt.verify(token, process.env.JWT_SECRET);
const event = await withRetry(async () => {
return await db('events')
.where({
id: decoded.eventId,
is_active: formatBoolean(true),
is_archived: formatBoolean(false)
})
.first();
});
const event = await db('events')
.where({
id: decoded.eventId,
is_active: formatBoolean(true),
is_archived: formatBoolean(false)
})
.first();
if (!event) {
return res.status(404).json({ error: 'Gallery not found or expired' });
+1 -1
View File
@@ -10,7 +10,7 @@ const DEFAULT_SESSION_TIMEOUT = 60 * 60 * 1000;
// Cache for session timeout setting
let cachedTimeout = null;
let cacheExpiry = 0;
const CACHE_DURATION = 30 * 60 * 1000; // 30 minutes - reduced DB queries
const CACHE_DURATION = 5 * 60 * 1000; // 5 minutes
// Clean up expired sessions every 5 minutes
setInterval(() => {
-111
View File
@@ -1,111 +0,0 @@
const fs = require('fs').promises;
const path = require('path');
const sharp = require('sharp');
const logger = require('../utils/logger');
/**
* Validate uploaded file is complete and not corrupted
*/
async function validateUploadedFile(filePath) {
try {
// Check file exists and has size
const stats = await fs.stat(filePath);
if (stats.size === 0) {
throw new Error('File is empty');
}
// For image files, verify they can be read by Sharp
const ext = path.extname(filePath).toLowerCase();
const imageExtensions = ['.jpg', '.jpeg', '.png', '.gif', '.webp'];
if (imageExtensions.includes(ext)) {
// Try to read metadata - this will fail if image is corrupted
let metadata;
try {
metadata = await sharp(filePath, {
failOnError: false, // Don't fail on recoverable errors
limitInputPixels: 268402689 // ~16k x 16k max
}).metadata();
} catch (metadataError) {
// If metadata reading fails, the file is likely incomplete
throw new Error(`Invalid image file: ${metadataError.message}`);
}
if (!metadata || !metadata.width || !metadata.height) {
throw new Error('Invalid image dimensions - file may be incomplete');
}
// Check for reasonable dimensions
if (metadata.width < 10 || metadata.height < 10) {
throw new Error('Image dimensions too small');
}
// Additional check: verify we can actually decode a small portion of the image
try {
await sharp(filePath, {
failOnError: false,
limitInputPixels: 268402689
})
.resize(10, 10) // Try to resize to very small size
.toBuffer();
} catch (decodeError) {
throw new Error(`Image decode failed - file may be corrupted: ${decodeError.message}`);
}
return true;
}
return true;
} catch (error) {
logger.error(`File validation failed for ${filePath}:`, error.message);
throw error;
}
}
/**
* Middleware to validate uploaded files after multer processing
*/
async function validateUploadedFiles(req, res, next) {
if (!req.files || req.files.length === 0) {
return next();
}
const validFiles = [];
const invalidFiles = [];
// Validate each file
for (const file of req.files) {
try {
await validateUploadedFile(file.path);
validFiles.push(file);
} catch (error) {
logger.warn(`Removing invalid upload ${file.originalname}: ${error.message}`);
invalidFiles.push({
filename: file.originalname,
error: error.message
});
// Delete the invalid file
try {
await fs.unlink(file.path);
} catch (unlinkErr) {
logger.error(`Failed to delete invalid file ${file.path}:`, unlinkErr.message);
}
}
}
// Update req.files to only include valid files
req.files = validFiles;
// Store invalid files info for response
if (invalidFiles.length > 0) {
req.invalidFiles = invalidFiles;
}
next();
}
module.exports = {
validateUploadedFile,
validateUploadedFiles
};
-4
View File
@@ -11,8 +11,6 @@ const photosRoutes = require('./adminPhotos');
const categoriesRoutes = require('./adminCategories');
const cmsRoutes = require('./adminCMS');
const notificationsRoutes = require('./adminNotifications');
const backupRoutes = require('./adminBackup');
const restoreRoutes = require('./adminRestore');
// Mount sub-routers
router.use('/dashboard', dashboardRoutes);
@@ -24,7 +22,5 @@ router.use('/events', photosRoutes);
router.use('/categories', categoriesRoutes);
router.use('/cms', cmsRoutes);
router.use('/notifications', notificationsRoutes);
router.use('/backup', backupRoutes);
router.use('/restore', restoreRoutes);
module.exports = router;
+1 -1
View File
@@ -251,7 +251,7 @@ router.post('/:id/restore', adminAuth, async (req, res) => {
} catch (statError) {
console.error(`Failed to stat file: ${actualFilePath}`);
console.error(`Entry name was: ${entry.entryName}`);
console.error('Error:', statError.message);
console.error(`Error:`, statError.message);
// Skip this file if we can't stat it
continue;
}
-956
View File
@@ -1,956 +0,0 @@
const express = require('express');
const { db } = require('../database/db');
const { adminAuth } = require('../middleware/auth');
const { triggerManualBackup, getBackupStatus, cleanupOldBackupRuns, getBackupManifest, validateBackupManifest } = require('../services/backupService');
const logger = require('../utils/logger');
const fs = require('fs').promises;
const path = require('path');
const crypto = require('crypto');
const archiver = require('archiver');
const S3StorageAdapter = require('../services/storage/s3Storage');
const router = express.Router();
// Get backup configuration
router.get('/config', adminAuth, async (req, res) => {
try {
const settings = await db('app_settings')
.where('setting_type', 'backup')
.select('setting_key', 'setting_value');
const config = {};
settings.forEach(setting => {
try {
config[setting.setting_key] = JSON.parse(setting.setting_value);
} catch (e) {
config[setting.setting_key] = setting.setting_value;
}
});
res.json(config);
} catch (error) {
logger.error('Failed to get backup configuration:', error);
res.status(500).json({ error: 'Failed to get backup configuration' });
}
});
// Update backup configuration
router.put('/config', adminAuth, async (req, res) => {
try {
const updates = req.body;
// Validate required fields based on destination type
if (updates.backup_destination_type) {
switch (updates.backup_destination_type) {
case 'local':
if (!updates.backup_destination_path) {
return res.status(400).json({ error: 'Local backup requires destination path' });
}
break;
case 'rsync':
if (!updates.backup_rsync_host || !updates.backup_rsync_path) {
return res.status(400).json({ error: 'Rsync backup requires host and path' });
}
break;
case 's3':
if (!updates.backup_s3_endpoint || !updates.backup_s3_bucket ||
!updates.backup_s3_access_key || !updates.backup_s3_secret_key) {
return res.status(400).json({ error: 'S3 backup requires endpoint, bucket, and credentials' });
}
break;
}
}
// Update settings
for (const [key, value] of Object.entries(updates)) {
if (key.startsWith('backup_')) {
await db('app_settings')
.insert({
setting_key: key,
setting_value: JSON.stringify(value),
setting_type: 'backup',
updated_at: new Date()
})
.onConflict('setting_key')
.merge({
setting_value: JSON.stringify(value),
updated_at: new Date()
});
}
}
// Restart backup service if enabled status changed
if ('backup_enabled' in updates) {
const { startBackupService, stopBackupService } = require('../services/backupService');
if (updates.backup_enabled) {
await startBackupService();
} else {
stopBackupService();
}
}
res.json({ success: true, message: 'Backup configuration updated' });
} catch (error) {
logger.error('Failed to update backup configuration:', error);
res.status(500).json({ error: 'Failed to update backup configuration' });
}
});
// Get backup status and history
router.get('/status', adminAuth, async (req, res) => {
try {
const limit = parseInt(req.query.limit) || 10;
const status = await getBackupStatus(limit);
res.json(status);
} catch (error) {
logger.error('Failed to get backup status:', error);
res.status(500).json({ error: 'Failed to get backup status' });
}
});
// Trigger manual backup
router.post('/run', adminAuth, async (req, res) => {
try {
// Check if backup is already running
const status = await getBackupStatus();
if (status.isRunning) {
return res.status(409).json({ error: 'Backup is already running' });
}
// Start backup in background
triggerManualBackup().catch(error => {
logger.error('Manual backup failed:', error);
});
res.json({ success: true, message: 'Backup started' });
} catch (error) {
logger.error('Failed to trigger manual backup:', error);
res.status(500).json({ error: 'Failed to trigger backup' });
}
});
// Get backup run details
router.get('/runs/:id', adminAuth, async (req, res) => {
try {
const { id } = req.params;
const run = await db('backup_runs')
.where('id', id)
.first();
if (!run) {
return res.status(404).json({ error: 'Backup run not found' });
}
// Parse JSON fields
if (run.statistics) {
try {
run.statistics = JSON.parse(run.statistics);
} catch (e) {
// Keep as string if parsing fails
}
}
res.json(run);
} catch (error) {
logger.error('Failed to get backup run details:', error);
res.status(500).json({ error: 'Failed to get backup run details' });
}
});
// Get file states (for debugging/monitoring)
router.get('/files', adminAuth, async (req, res) => {
try {
const { page = 1, limit = 50, search = '' } = req.query;
const offset = (page - 1) * limit;
let query = db('backup_file_states');
if (search) {
query = query.where('file_path', 'like', `%${search}%`);
}
const [files, totalCount] = await Promise.all([
query
.orderBy('last_backed_up', 'desc')
.limit(limit)
.offset(offset),
db('backup_file_states').count('* as count').first()
]);
res.json({
files,
pagination: {
page: parseInt(page),
limit: parseInt(limit),
total: totalCount.count,
pages: Math.ceil(totalCount.count / limit)
}
});
} catch (error) {
logger.error('Failed to get backup file states:', error);
res.status(500).json({ error: 'Failed to get file states' });
}
});
// Clean up old backup runs
router.delete('/cleanup', adminAuth, async (req, res) => {
try {
const { days = 30 } = req.body;
await cleanupOldBackupRuns(days);
res.json({ success: true, message: `Cleaned up backup runs older than ${days} days` });
} catch (error) {
logger.error('Failed to cleanup old backup runs:', error);
res.status(500).json({ error: 'Failed to cleanup backup runs' });
}
});
// Test backup destination connectivity
router.post('/test-connection', adminAuth, async (req, res) => {
try {
const { destination_type, ...config } = req.body;
switch (destination_type) {
case 'local':
// Test local path access
const fs = require('fs').promises;
try {
await fs.access(config.path, fs.constants.W_OK);
res.json({ success: true, message: 'Local path is writable' });
} catch (error) {
res.json({ success: false, message: 'Cannot write to local path: ' + error.message });
}
break;
case 'rsync':
// Test rsync connection
const { exec } = require('child_process');
const { promisify } = require('util');
const execAsync = promisify(exec);
const sshCommand = config.ssh_key
? `ssh -i ${config.ssh_key} -o StrictHostKeyChecking=no -o ConnectTimeout=10`
: 'ssh -o StrictHostKeyChecking=no -o ConnectTimeout=10';
const testCommand = config.user
? `${sshCommand} ${config.user}@${config.host} "echo 'Connection successful'"`
: `${sshCommand} ${config.host} "echo 'Connection successful'"`;
try {
const { stdout } = await execAsync(testCommand);
res.json({ success: true, message: 'Rsync connection successful' });
} catch (error) {
res.json({ success: false, message: 'Rsync connection failed: ' + error.message });
}
break;
case 's3':
// Test S3 connection (would need AWS SDK)
res.json({ success: false, message: 'S3 testing not implemented yet' });
break;
default:
res.status(400).json({ error: 'Invalid destination type' });
}
} catch (error) {
logger.error('Failed to test backup connection:', error);
res.status(500).json({ error: 'Failed to test connection' });
}
});
// Get backup manifest for a specific backup run
router.get('/manifest/:backupRunId', adminAuth, async (req, res) => {
try {
const { backupRunId } = req.params;
const result = await getBackupManifest(backupRunId);
res.json({
backupRunId,
manifest: result.manifest,
summary: result.summary
});
} catch (error) {
logger.error('Failed to get backup manifest:', error);
res.status(404).json({ error: error.message || 'Backup manifest not found' });
}
});
// Validate a backup manifest
router.post('/manifest/validate', adminAuth, async (req, res) => {
try {
const { manifestPath } = req.body;
if (!manifestPath) {
return res.status(400).json({ error: 'manifestPath is required' });
}
const result = await validateBackupManifest(manifestPath);
res.json({
valid: result.valid,
error: result.error,
manifestPath
});
} catch (error) {
logger.error('Failed to validate manifest:', error);
res.status(500).json({ error: 'Failed to validate manifest' });
}
});
// Download backup manifest
router.get('/manifest/:backupRunId/download', adminAuth, async (req, res) => {
try {
const { backupRunId } = req.params;
const { format = 'json' } = req.query;
const result = await getBackupManifest(backupRunId);
// Set appropriate headers
const filename = `backup-manifest-${backupRunId}.${format}`;
res.setHeader('Content-Type', format === 'yaml' ? 'text/yaml' : 'application/json');
res.setHeader('Content-Disposition', `attachment; filename="${filename}"`);
// Send the manifest in requested format
if (format === 'yaml') {
const yaml = require('js-yaml');
res.send(yaml.dump(result.manifest, {
indent: 2,
lineWidth: -1,
noRefs: true,
sortKeys: true
}));
} else {
res.json(result.manifest);
}
} catch (error) {
logger.error('Failed to download backup manifest:', error);
res.status(404).json({ error: error.message || 'Backup manifest not found' });
}
});
// Get manifest for specific backup
router.get('/manifests/:backupId', adminAuth, async (req, res) => {
try {
const { backupId } = req.params;
const result = await getBackupManifest(backupId);
res.json({
backupId,
manifest: result.manifest,
summary: result.summary
});
} catch (error) {
logger.error('Failed to get backup manifest:', error);
res.status(404).json({ error: error.message || 'Backup manifest not found' });
}
});
// Download manifest file
router.get('/manifests/:backupId/download', adminAuth, async (req, res) => {
try {
const { backupId } = req.params;
const { format = 'json' } = req.query;
const result = await getBackupManifest(backupId);
// Set appropriate headers
const filename = `backup-manifest-${backupId}.${format}`;
res.setHeader('Content-Type', format === 'yaml' ? 'text/yaml' : 'application/json');
res.setHeader('Content-Disposition', `attachment; filename="${filename}"`);
// Send the manifest in requested format
if (format === 'yaml') {
const yaml = require('js-yaml');
res.send(yaml.dump(result.manifest, {
indent: 2,
lineWidth: -1,
noRefs: true,
sortKeys: true
}));
} else {
res.json(result.manifest);
}
} catch (error) {
logger.error('Failed to download backup manifest:', error);
res.status(404).json({ error: error.message || 'Backup manifest not found' });
}
});
// Validate a manifest
router.post('/manifests/validate', adminAuth, async (req, res) => {
try {
const { manifestPath, manifestData } = req.body;
if (!manifestPath && !manifestData) {
return res.status(400).json({ error: 'Either manifestPath or manifestData is required' });
}
if (manifestData) {
// Validate provided manifest data directly
const validationResult = await validateManifestData(manifestData);
return res.json(validationResult);
}
// Use existing validation function for path
const result = await validateBackupManifest(manifestPath);
res.json({
valid: result.valid,
error: result.error,
manifestPath
});
} catch (error) {
logger.error('Failed to validate manifest:', error);
res.status(500).json({ error: 'Failed to validate manifest' });
}
});
// List S3 buckets
router.get('/s3/buckets', adminAuth, async (req, res) => {
try {
const config = await getBackupConfig();
if (config.backup_destination_type !== 's3') {
return res.status(400).json({ error: 'S3 backup not configured' });
}
const s3Adapter = new S3StorageAdapter({
endpoint: config.backup_s3_endpoint,
bucket: config.backup_s3_bucket,
accessKeyId: config.backup_s3_access_key,
secretAccessKey: config.backup_s3_secret_key,
region: config.backup_s3_region || 'us-east-1',
forcePathStyle: config.backup_s3_force_path_style || false
});
// List buckets using the S3 client
const { ListBucketsCommand } = require('@aws-sdk/client-s3');
const result = await s3Adapter.s3Client.send(new ListBucketsCommand({}));
res.json({
buckets: result.Buckets || [],
owner: result.Owner || null
});
} catch (error) {
logger.error('Failed to list S3 buckets:', error);
res.status(500).json({ error: 'Failed to list S3 buckets: ' + error.message });
}
});
// List files in S3 backup location
router.get('/s3/files', adminAuth, async (req, res) => {
try {
const { prefix = '', maxKeys = 100, continuationToken } = req.query;
const config = await getBackupConfig();
if (config.backup_destination_type !== 's3') {
return res.status(400).json({ error: 'S3 backup not configured' });
}
const s3Adapter = new S3StorageAdapter({
endpoint: config.backup_s3_endpoint,
bucket: config.backup_s3_bucket,
accessKeyId: config.backup_s3_access_key,
secretAccessKey: config.backup_s3_secret_key,
region: config.backup_s3_region || 'us-east-1',
forcePathStyle: config.backup_s3_force_path_style || false
});
const result = await s3Adapter.list(prefix, {
maxKeys: parseInt(maxKeys),
continuationToken
});
res.json({
files: result.objects || [],
directories: result.directories || [],
isTruncated: result.isTruncated,
nextContinuationToken: result.nextContinuationToken,
prefix: prefix
});
} catch (error) {
logger.error('Failed to list S3 files:', error);
res.status(500).json({ error: 'Failed to list S3 files: ' + error.message });
}
});
// Clean up old S3 backups
router.delete('/s3/cleanup', adminAuth, async (req, res) => {
try {
const { retentionDays = 30, dryRun = false } = req.body;
const config = await getBackupConfig();
if (config.backup_destination_type !== 's3') {
return res.status(400).json({ error: 'S3 backup not configured' });
}
const s3Adapter = new S3StorageAdapter({
endpoint: config.backup_s3_endpoint,
bucket: config.backup_s3_bucket,
accessKeyId: config.backup_s3_access_key,
secretAccessKey: config.backup_s3_secret_key,
region: config.backup_s3_region || 'us-east-1',
forcePathStyle: config.backup_s3_force_path_style || false
});
const cutoffDate = new Date();
cutoffDate.setDate(cutoffDate.getDate() - retentionDays);
// List all backup files
const backupFiles = await s3Adapter.list('backups/', { maxKeys: 1000 });
const filesToDelete = [];
let totalSize = 0;
for (const file of backupFiles.objects || []) {
if (file.lastModified && new Date(file.lastModified) < cutoffDate) {
filesToDelete.push(file.key);
totalSize += file.size || 0;
}
}
if (dryRun) {
return res.json({
wouldDelete: filesToDelete.length,
totalSize: totalSize,
files: filesToDelete.slice(0, 100), // Limit preview
message: 'Dry run completed - no files were deleted'
});
}
// Delete files in batches
const deleteResult = await s3Adapter.deleteMany(filesToDelete);
const deletedCount = deleteResult.Deleted ? deleteResult.Deleted.length : 0;
// Also clean up database records
await cleanupOldBackupRuns(retentionDays);
res.json({
success: true,
deletedCount: deletedCount,
totalSize: totalSize,
message: `Cleaned up ${deletedCount} S3 backup files older than ${retentionDays} days`
});
} catch (error) {
logger.error('Failed to cleanup S3 backups:', error);
res.status(500).json({ error: 'Failed to cleanup S3 backups: ' + error.message });
}
});
// Test S3 upload functionality
router.post('/s3/test-upload', adminAuth, async (req, res) => {
try {
const config = await getBackupConfig();
if (config.backup_destination_type !== 's3') {
return res.status(400).json({ error: 'S3 backup not configured' });
}
const s3Adapter = new S3StorageAdapter({
endpoint: config.backup_s3_endpoint,
bucket: config.backup_s3_bucket,
accessKeyId: config.backup_s3_access_key,
secretAccessKey: config.backup_s3_secret_key,
region: config.backup_s3_region || 'us-east-1',
forcePathStyle: config.backup_s3_force_path_style || false
});
// Create test content
const testKey = `test/backup-test-${Date.now()}.txt`;
const testContent = `PicPeak S3 backup test\nTimestamp: ${new Date().toISOString()}\nEndpoint: ${config.backup_s3_endpoint || 'AWS'}\nBucket: ${config.backup_s3_bucket}`;
// Test upload
const uploadStart = Date.now();
await s3Adapter.upload(testKey, Buffer.from(testContent));
const uploadTime = Date.now() - uploadStart;
// Test download
const downloadStart = Date.now();
const downloadedContent = await s3Adapter.download(testKey);
const downloadTime = Date.now() - downloadStart;
// Verify content
const contentMatch = downloadedContent.toString() === testContent;
// Test deletion
await s3Adapter.delete(testKey);
res.json({
success: true,
testKey: testKey,
uploadTime: uploadTime,
downloadTime: downloadTime,
contentMatch: contentMatch,
message: 'S3 upload test completed successfully'
});
} catch (error) {
logger.error('S3 upload test failed:', error);
res.status(500).json({ error: 'S3 upload test failed: ' + error.message });
}
});
// Download entire backup
router.get('/download/:backupId', adminAuth, async (req, res) => {
try {
const { backupId } = req.params;
// Get backup run details
const backupRun = await db('backup_runs')
.where('id', backupId)
.first();
if (!backupRun) {
return res.status(404).json({ error: 'Backup not found' });
}
if (backupRun.status !== 'completed') {
return res.status(400).json({ error: 'Backup is not completed' });
}
const config = await getBackupConfig();
// Handle different backup types
switch (config.backup_destination_type) {
case 'local':
// Stream local backup as zip
const backupPath = path.join(config.backup_destination_path, `backup-${backupRun.id}`);
const archive = archiver('zip', { zlib: { level: 9 } });
res.attachment(`picpeak-backup-${backupRun.id}.zip`);
archive.pipe(res);
// Add backup directory contents
archive.directory(backupPath, false);
// Add manifest if exists
if (backupRun.manifest_path && await fs.access(backupRun.manifest_path).then(() => true).catch(() => false)) {
archive.file(backupRun.manifest_path, { name: 'manifest.json' });
}
await archive.finalize();
break;
case 's3':
// For S3, provide pre-signed URLs or stream files
const s3Adapter = new S3StorageAdapter({
endpoint: config.backup_s3_endpoint,
bucket: config.backup_s3_bucket,
accessKeyId: config.backup_s3_access_key,
secretAccessKey: config.backup_s3_secret_key,
region: config.backup_s3_region || 'us-east-1',
forcePathStyle: config.backup_s3_force_path_style || false
});
// List all files for this backup
const prefix = `backups/${backupRun.id}/`;
const files = await s3Adapter.list(prefix, { maxKeys: 1000 });
// Generate pre-signed URLs
const urls = [];
for (const file of files.objects || []) {
const url = await s3Adapter.getSignedUrl('getObject', file.key, { expiresIn: 3600 }); // 1 hour
urls.push({
key: file.key,
size: file.size,
url: url
});
}
res.json({
backupId: backupRun.id,
type: 's3',
files: urls,
expiresIn: 3600,
message: 'Use the provided URLs to download individual files'
});
break;
case 'rsync':
return res.status(400).json({ error: 'Direct download not available for rsync backups' });
default:
return res.status(400).json({ error: 'Unknown backup type' });
}
} catch (error) {
logger.error('Failed to download backup:', error);
res.status(500).json({ error: 'Failed to download backup: ' + error.message });
}
});
// Get current file checksums
router.get('/checksums', adminAuth, async (req, res) => {
try {
const { path: targetPath = '', recursive = true } = req.query;
const checksums = {};
// Get storage path
const storagePath = process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
const basePath = targetPath ? path.join(storagePath, targetPath) : storagePath;
// Calculate checksums for files
async function calculateDirChecksums(dirPath, relative = '') {
try {
const entries = await fs.readdir(dirPath, { withFileTypes: true });
for (const entry of entries) {
const fullPath = path.join(dirPath, entry.name);
const relativePath = path.join(relative, entry.name);
if (entry.isDirectory() && recursive) {
await calculateDirChecksums(fullPath, relativePath);
} else if (entry.isFile()) {
const hash = crypto.createHash('sha256');
const stream = require('fs').createReadStream(fullPath);
await new Promise((resolve, reject) => {
stream.on('data', data => hash.update(data));
stream.on('end', () => {
checksums[relativePath] = {
checksum: hash.digest('hex'),
size: entry.size,
modified: entry.mtime
};
resolve();
});
stream.on('error', reject);
});
}
}
} catch (error) {
logger.error(`Failed to calculate checksums for ${dirPath}:`, error);
}
}
await calculateDirChecksums(basePath);
// Also get database checksums from backup_file_states
const dbChecksums = await db('backup_file_states')
.select('file_path', 'checksum', 'size_bytes', 'last_modified');
res.json({
currentChecksums: checksums,
totalFiles: Object.keys(checksums).length,
databaseChecksums: dbChecksums.reduce((acc, row) => {
acc[row.file_path] = {
checksum: row.checksum,
size: row.size_bytes,
modified: row.last_modified
};
return acc;
}, {}),
path: targetPath || '/'
});
} catch (error) {
logger.error('Failed to get file checksums:', error);
res.status(500).json({ error: 'Failed to get file checksums: ' + error.message });
}
});
// Estimate backup size before running
router.post('/estimate', adminAuth, async (req, res) => {
try {
const { includeArchived = true } = req.body;
// Get storage path
const storagePath = process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
let totalSize = 0;
let fileCount = 0;
const breakdown = {};
// Estimate size for each directory
async function estimateDir(dirPath, category) {
let dirSize = 0;
let dirCount = 0;
try {
const entries = await fs.readdir(dirPath, { withFileTypes: true });
for (const entry of entries) {
const fullPath = path.join(dirPath, entry.name);
if (entry.isDirectory()) {
const subResult = await estimateDir(fullPath, category);
dirSize += subResult.size;
dirCount += subResult.count;
} else if (entry.isFile()) {
const stats = await fs.stat(fullPath);
dirSize += stats.size;
dirCount++;
}
}
} catch (error) {
if (error.code !== 'ENOENT') {
logger.error(`Failed to estimate ${dirPath}:`, error);
}
}
return { size: dirSize, count: dirCount };
}
// Estimate each category
const categories = [
{ path: 'events/active', name: 'Active Events' },
{ path: 'thumbnails', name: 'Thumbnails' },
{ path: 'uploads', name: 'Uploads' }
];
if (includeArchived) {
categories.push({ path: 'events/archived', name: 'Archived Events' });
}
for (const category of categories) {
const result = await estimateDir(path.join(storagePath, category.path), category.name);
breakdown[category.name] = {
size: result.size,
sizeFormatted: formatBytes(result.size),
fileCount: result.count
};
totalSize += result.size;
fileCount += result.count;
}
// Estimate database size
const dbPath = process.env.DB_TYPE === 'postgresql'
? null
: path.join(__dirname, '../../database.sqlite');
if (dbPath) {
try {
const dbStats = await fs.stat(dbPath);
breakdown['Database'] = {
size: dbStats.size,
sizeFormatted: formatBytes(dbStats.size),
fileCount: 1
};
totalSize += dbStats.size;
fileCount += 1;
} catch (error) {
logger.error('Failed to get database size:', error);
}
}
// Estimate compression ratio (typically 20-40% for mixed media)
const estimatedCompressedSize = Math.round(totalSize * 0.7);
res.json({
totalSize: totalSize,
totalSizeFormatted: formatBytes(totalSize),
estimatedCompressedSize: estimatedCompressedSize,
estimatedCompressedSizeFormatted: formatBytes(estimatedCompressedSize),
fileCount: fileCount,
breakdown: breakdown,
includeArchived: includeArchived,
estimatedDuration: Math.max(60, Math.round(totalSize / (50 * 1024 * 1024))), // Estimate 50MB/s
warnings: totalSize > 10 * 1024 * 1024 * 1024 ? ['Backup size exceeds 10GB, may take significant time'] : []
});
} catch (error) {
logger.error('Failed to estimate backup size:', error);
res.status(500).json({ error: 'Failed to estimate backup size: ' + error.message });
}
});
// Helper function to format bytes
function formatBytes(bytes, decimals = 2) {
if (bytes === 0) return '0 Bytes';
const k = 1024;
const dm = decimals < 0 ? 0 : decimals;
const sizes = ['Bytes', 'KB', 'MB', 'GB', 'TB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return parseFloat((bytes / Math.pow(k, i)).toFixed(dm)) + ' ' + sizes[i];
}
// Helper function to get backup configuration
async function getBackupConfig() {
try {
const settings = await db('app_settings')
.where('setting_type', 'backup')
.select('setting_key', 'setting_value');
const config = {};
settings.forEach(setting => {
try {
config[setting.setting_key] = JSON.parse(setting.setting_value);
} catch (e) {
config[setting.setting_key] = setting.setting_value;
}
});
return config;
} catch (error) {
logger.error('Failed to get backup configuration:', error);
return {};
}
}
// Helper function to validate manifest data
async function validateManifestData(manifestData) {
try {
// Check required fields
const requiredFields = ['version', 'backupId', 'timestamp', 'files'];
const missingFields = requiredFields.filter(field => !manifestData[field]);
if (missingFields.length > 0) {
return {
valid: false,
error: `Missing required fields: ${missingFields.join(', ')}`,
details: { missingFields }
};
}
// Validate version
if (manifestData.version !== '1.0') {
return {
valid: false,
error: `Unsupported manifest version: ${manifestData.version}`,
details: { version: manifestData.version }
};
}
// Validate files array
if (!Array.isArray(manifestData.files)) {
return {
valid: false,
error: 'Files must be an array',
details: { filesType: typeof manifestData.files }
};
}
// Validate each file entry
const invalidFiles = [];
for (let i = 0; i < manifestData.files.length; i++) {
const file = manifestData.files[i];
if (!file.path || !file.checksum || typeof file.size !== 'number') {
invalidFiles.push({ index: i, file });
}
}
if (invalidFiles.length > 0) {
return {
valid: false,
error: `Invalid file entries: ${invalidFiles.length}`,
details: { invalidFiles: invalidFiles.slice(0, 10) } // Limit to first 10
};
}
return {
valid: true,
details: {
version: manifestData.version,
backupId: manifestData.backupId,
timestamp: manifestData.timestamp,
fileCount: manifestData.files.length,
totalSize: manifestData.files.reduce((sum, f) => sum + (f.size || 0), 0)
}
};
} catch (error) {
return {
valid: false,
error: `Validation error: ${error.message}`,
details: { error: error.message }
};
}
}
module.exports = router;
+11 -35
View File
@@ -48,9 +48,9 @@ router.get('/stats', adminAuth, async (req, res) => {
.count('id as count')
.first();
// Get total downloads (last 30 days) - include both single and bulk downloads
// Get total downloads (last 30 days)
const totalDownloads = await db('access_logs')
.whereIn('action', ['download', 'download_all'])
.where('action', 'download')
.where('timestamp', '>=', thirtyDaysAgo.toISOString())
.count('id as count')
.first();
@@ -73,7 +73,7 @@ router.get('/stats', adminAuth, async (req, res) => {
.first();
const previousDownloads = await db('access_logs')
.whereIn('action', ['download', 'download_all'])
.where('action', 'download')
.where('timestamp', '>=', sixtyDaysAgo.toISOString())
.where('timestamp', '<', thirtyDaysAgo.toISOString())
.count('id as count')
@@ -243,10 +243,10 @@ router.get('/analytics', adminAuth, async (req, res) => {
.where('timestamp', '>=', startDateStr)
.groupByRaw('DATE(timestamp)');
// Get downloads per day - include both single and bulk downloads
// Get downloads per day
const downloadsData = await db('access_logs')
.select(db.raw('DATE(timestamp) as date'), db.raw('COUNT(*) as count'))
.whereIn('action', ['download', 'download_all'])
.where('action', 'download')
.where('timestamp', '>=', startDateStr)
.groupByRaw('DATE(timestamp)');
@@ -272,15 +272,14 @@ router.get('/analytics', adminAuth, async (req, res) => {
if (dateObj) dateObj.uniqueVisitors = row.count;
});
// Get top galleries by views with additional metrics
// Get top galleries by views
const topGalleries = await db('access_logs')
.select('events.id', 'events.event_name', 'events.slug')
.select(db.raw('COUNT(CASE WHEN action = \'view\' THEN 1 END) as views'))
.select(db.raw('COUNT(DISTINCT CASE WHEN action = \'view\' THEN ip_address END) as uniqueVisitors'))
.select(db.raw('COUNT(CASE WHEN action IN (\'download\', \'download_all\') THEN 1 END) as downloads'))
.select('events.event_name', 'events.slug')
.select(db.raw('COUNT(*) as views'))
.join('events', 'access_logs.event_id', 'events.id')
.where('access_logs.action', 'view')
.where('access_logs.timestamp', '>=', startDateStr)
.groupBy('events.id', 'events.event_name', 'events.slug')
.groupBy('events.id')
.orderBy('views', 'desc')
.limit(5);
@@ -310,33 +309,10 @@ router.get('/analytics', adminAuth, async (req, res) => {
devices[d.device_type] = Math.round((d.count / totalDevices) * 100);
});
// Calculate totals for the period (matching /stats logic)
const totalViews = await db('access_logs')
.where('action', 'view')
.where('timestamp', '>=', startDateStr)
.count('id as count')
.first();
const totalDownloadsCount = await db('access_logs')
.whereIn('action', ['download', 'download_all'])
.where('timestamp', '>=', startDateStr)
.count('id as count')
.first();
const totalUniqueVisitors = await db('access_logs')
.where('timestamp', '>=', startDateStr)
.countDistinct('ip_address as count')
.first();
res.json({
chartData: dates,
topGalleries,
devices,
totals: {
views: totalViews?.count || 0,
downloads: totalDownloadsCount?.count || 0,
uniqueVisitors: totalUniqueVisitors?.count || 0
}
devices
});
} catch (error) {
console.error('Analytics error:', error);
-273
View File
@@ -1,273 +0,0 @@
const express = require('express');
const router = express.Router();
const { adminAuth } = require('../middleware/auth');
const { databaseBackupService } = require('../services/databaseBackup');
const { db } = require('../database/db');
const logger = require('../utils/logger');
// All routes require admin authentication
router.use(adminAuth);
/**
* Get database backup status and configuration
*/
router.get('/status', async (req, res) => {
try {
// Get configuration
const config = await databaseBackupService.getBackupConfig();
// Get recent backup history
const history = await databaseBackupService.getBackupHistory(10);
// Get current progress if running
const progress = databaseBackupService.getProgress();
// Calculate health status
const lastBackup = history[0];
const isHealthy = lastBackup && lastBackup.status === 'completed' &&
new Date(lastBackup.completed_at) > new Date(Date.now() - 48 * 60 * 60 * 1000); // Within 48 hours
res.json({
config,
isRunning: databaseBackupService.isRunning,
isHealthy,
currentProgress: progress,
lastBackup,
recentBackups: history,
dbType: databaseBackupService.dbType
});
} catch (error) {
logger.error('Failed to get database backup status:', error);
res.status(500).json({ error: 'Failed to get backup status' });
}
});
/**
* Update database backup configuration
*/
router.put('/config', async (req, res) => {
try {
const allowedSettings = [
'database_backup_enabled',
'database_backup_schedule',
'database_backup_destination_path',
'database_backup_compress',
'database_backup_validate_integrity',
'database_backup_include_checksums',
'database_backup_retention_days',
'database_backup_email_on_failure',
'database_backup_email_on_success'
];
const updates = [];
for (const [key, value] of Object.entries(req.body)) {
if (allowedSettings.includes(key)) {
// Check if setting exists
const existing = await db('app_settings')
.where('setting_key', key)
.first();
if (existing) {
await db('app_settings')
.where('setting_key', key)
.update({
setting_value: JSON.stringify(value),
updated_at: new Date()
});
} else {
await db('app_settings').insert({
setting_key: key,
setting_value: JSON.stringify(value),
setting_type: 'database_backup'
});
}
updates.push(key);
}
}
// Restart scheduled backups if enabled state changed
if (updates.includes('database_backup_enabled') || updates.includes('database_backup_schedule')) {
const { startScheduledBackups, stopScheduledBackups } = require('../services/databaseBackup');
stopScheduledBackups();
await startScheduledBackups();
}
res.json({
success: true,
updatedSettings: updates,
message: 'Database backup configuration updated successfully'
});
} catch (error) {
logger.error('Failed to update database backup config:', error);
res.status(500).json({ error: 'Failed to update configuration' });
}
});
/**
* Trigger manual database backup
*/
router.post('/backup', async (req, res) => {
try {
if (databaseBackupService.isRunning) {
return res.status(409).json({ error: 'Backup already in progress' });
}
// Start backup asynchronously
res.json({
success: true,
message: 'Database backup started',
trackingUrl: '/api/admin/database-backup/progress'
});
// Run backup in background
databaseBackupService.backup(req.body).catch(error => {
logger.error('Manual database backup failed:', error);
});
} catch (error) {
logger.error('Failed to start database backup:', error);
res.status(500).json({ error: 'Failed to start backup' });
}
});
/**
* Get current backup progress
*/
router.get('/progress', async (req, res) => {
try {
const progress = databaseBackupService.getProgress();
res.json({
isRunning: databaseBackupService.isRunning,
progress
});
} catch (error) {
logger.error('Failed to get backup progress:', error);
res.status(500).json({ error: 'Failed to get progress' });
}
});
/**
* Get backup history with pagination
*/
router.get('/history', async (req, res) => {
try {
const page = parseInt(req.query.page) || 1;
const limit = parseInt(req.query.limit) || 20;
const offset = (page - 1) * limit;
const [backups, totalCount] = await Promise.all([
db('database_backup_runs')
.orderBy('started_at', 'desc')
.limit(limit)
.offset(offset),
db('database_backup_runs').count('* as count').first()
]);
res.json({
backups,
pagination: {
page,
limit,
total: totalCount.count,
pages: Math.ceil(totalCount.count / limit)
}
});
} catch (error) {
logger.error('Failed to get backup history:', error);
res.status(500).json({ error: 'Failed to get history' });
}
});
/**
* Delete old backup files
*/
router.delete('/cleanup', async (req, res) => {
try {
const { retentionDays = 30 } = req.body;
await databaseBackupService.cleanupOldBackups(retentionDays);
res.json({
success: true,
message: `Cleaned up backups older than ${retentionDays} days`
});
} catch (error) {
logger.error('Failed to cleanup old backups:', error);
res.status(500).json({ error: 'Failed to cleanup backups' });
}
});
/**
* Test database backup configuration
*/
router.post('/test', async (req, res) => {
try {
const config = await databaseBackupService.getBackupConfig();
// Test database connection
const testResults = {
databaseConnection: false,
destinationWritable: false,
compressionAvailable: true,
estimatedSize: null
};
// Test database connection
try {
await db.raw('SELECT 1');
testResults.databaseConnection = true;
} catch (error) {
testResults.databaseConnectionError = error.message;
}
// Test destination path
if (config.destinationPath) {
try {
const fs = require('fs').promises;
const testFile = `${config.destinationPath}/.test-${Date.now()}`;
await fs.writeFile(testFile, 'test');
await fs.unlink(testFile);
testResults.destinationWritable = true;
} catch (error) {
testResults.destinationError = error.message;
}
}
// Estimate database size
try {
testResults.estimatedSize = await databaseBackupService.getDatabaseSize();
} catch (error) {
testResults.sizeError = error.message;
}
res.json({
success: testResults.databaseConnection && testResults.destinationWritable,
results: testResults
});
} catch (error) {
logger.error('Failed to test backup configuration:', error);
res.status(500).json({ error: 'Failed to test configuration' });
}
});
/**
* Get table checksums
*/
router.get('/checksums', async (req, res) => {
try {
const checksums = await databaseBackupService.getTableChecksums();
res.json({
checksums,
tableCount: Object.keys(checksums).length,
totalRows: Object.values(checksums).reduce((sum, table) => sum + table.rowCount, 0)
});
} catch (error) {
logger.error('Failed to get table checksums:', error);
res.status(500).json({ error: 'Failed to get checksums' });
}
});
module.exports = router;
+9 -9
View File
@@ -210,15 +210,15 @@ router.get('/templates', adminAuth, async (req, res) => {
id: template.id,
template_key: template.template_key,
variables: (() => {
try {
if (!template.variables) return [];
if (typeof template.variables === 'object') return template.variables;
return JSON.parse(template.variables);
} catch (e) {
console.warn('Failed to parse variables for template:', template.template_key, e.message);
return [];
}
})(),
try {
if (!template.variables) return [];
if (typeof template.variables === 'object') return template.variables;
return JSON.parse(template.variables);
} catch (e) {
console.warn('Failed to parse variables for template:', template.template_key, e.message);
return [];
}
})(),
updated_at: template.updated_at
};
+1 -1
View File
@@ -42,7 +42,7 @@ router.post('/', adminAuth, [
} = req.body;
// Validate password strength for gallery
const passwordValidation = await validatePasswordInContext(password, 'gallery', {
const passwordValidation = validatePasswordInContext(password, 'gallery', {
eventName: event_name
});
+14 -18
View File
@@ -10,7 +10,7 @@ const path = require('path');
const { archiveEvent } = require('../services/archiveService');
const { queueEmail } = require('../services/emailProcessor');
const { escapeLikePattern } = require('../utils/sqlSecurity');
// formatDate import removed - dates are formatted by email processor
const { formatDate } = require('../utils/dateFormatter');
const { validatePasswordInContext, getBcryptRounds } = require('../utils/passwordValidation');
const { formatBoolean } = require('../utils/dbCompat');
@@ -53,7 +53,7 @@ router.post('/', adminAuth, [
} = req.body;
// Validate password strength
const passwordValidation = await validatePasswordInContext(password, 'gallery', {
const passwordValidation = validatePasswordInContext(password, 'gallery', {
eventName: event_name
});
@@ -89,14 +89,7 @@ router.post('/', adminAuth, [
const password_hash = await bcrypt.hash(password, getBcryptRounds());
// Calculate expiration date (days after event date)
// Parse YYYY-MM-DD format as local date to avoid timezone issues
let expires_at;
if (event_date.match(/^\d{4}-\d{2}-\d{2}$/)) {
const [year, month, day] = event_date.split('-').map(num => parseInt(num, 10));
expires_at = new Date(year, month - 1, day);
} else {
expires_at = new Date(event_date);
}
const expires_at = new Date(event_date);
expires_at.setDate(expires_at.getDate() + parseInt(expiration_days, 10));
// Create folder structure
@@ -135,7 +128,8 @@ router.post('/', adminAuth, [
);
// Queue creation email
// Language detection is handled by email processor
// Determine language based on email domain
const emailLang = host_email.endsWith('.de') ? 'de' : 'en';
await db('email_queue').insert({
event_id: eventId,
@@ -144,10 +138,10 @@ router.post('/', adminAuth, [
email_data: JSON.stringify({
host_name: host_name,
event_name,
event_date: event_date, // Pass raw date - will be formatted by email processor
event_date: await formatDate(event_date, emailLang),
gallery_link: shareLink,
gallery_password: password,
expiry_date: expires_at.toISOString(), // Pass ISO string - will be formatted by email processor
expiry_date: await formatDate(expires_at, emailLang),
welcome_message: welcome_message || ''
}),
status: 'pending',
@@ -552,10 +546,10 @@ router.post('/:id/reset-password', adminAuth, async (req, res) => {
await queueEmail(id, event.host_email, 'gallery_created', {
host_name: event.host_email.split('@')[0],
event_name: event.event_name,
event_date: event.event_date, // Pass raw date - will be formatted by email processor
event_date: new Date(event.event_date).toLocaleDateString(),
gallery_link: event.share_link,
gallery_password: newPassword,
expiry_date: event.expires_at // Pass raw date - will be formatted by email processor
expiry_date: new Date(event.expires_at).toLocaleDateString()
});
}
@@ -602,16 +596,18 @@ router.post('/:id/resend-email', adminAuth, async (req, res) => {
galleryPassword = '{{password_security_message}}';
}
// Dates will be formatted by the email processor based on recipient language
// Format dates in a neutral format - the email processor will localize them
const eventDate = new Date(event.event_date);
const expiryDate = new Date(event.expires_at);
// Queue the email
await queueEmail(id, event.host_email, 'gallery_created', {
host_name: event.host_name || event.host_email.split('@')[0],
event_name: event.event_name,
event_date: event.event_date, // Pass raw date - will be formatted by email processor
event_date: eventDate.toISOString().split('T')[0], // YYYY-MM-DD format
gallery_link: event.share_link,
gallery_password: galleryPassword,
expiry_date: event.expires_at, // Pass raw date - will be formatted by email processor
expiry_date: expiryDate.toISOString().split('T')[0], // YYYY-MM-DD format
welcome_message: event.welcome_message || '',
eventId: id,
isResend: true // Flag to indicate this is a resend
-411
View File
@@ -1,411 +0,0 @@
const express = require('express');
const router = express.Router();
const { adminAuth } = require('../middleware/auth-enhanced-v2');
const feedbackService = require('../services/feedbackService');
const feedbackModeration = require('../services/feedbackModeration');
const { db, logActivity } = require('../database/db');
const logger = require('../utils/logger');
const {
validateEventId,
validateFeedbackSettings,
validateWordFilter,
checkValidation
} = require('../utils/feedbackValidation');
// Get event feedback settings
router.get('/events/:eventId/feedback-settings',
adminAuth,
validateEventId,
checkValidation,
async (req, res) => {
try {
const { eventId } = req.params;
// Verify event exists and belongs to admin
const event = await db('events').where('id', eventId).first();
if (!event) {
return res.status(404).json({ error: 'Event not found' });
}
const settings = await feedbackService.getEventFeedbackSettings(eventId);
res.json(settings);
} catch (error) {
logger.error('Error getting feedback settings:', error);
res.status(500).json({ error: 'Failed to get feedback settings' });
}
}
);
// Update event feedback settings
router.put('/events/:eventId/feedback-settings',
adminAuth,
validateEventId,
validateFeedbackSettings,
checkValidation,
async (req, res) => {
try {
const { eventId } = req.params;
const settings = req.body;
// Verify event exists
const event = await db('events').where('id', eventId).first();
if (!event) {
return res.status(404).json({ error: 'Event not found' });
}
const updatedSettings = await feedbackService.updateEventFeedbackSettings(eventId, settings);
await logActivity('feedback_settings_updated', {
event_id: eventId,
settings: updatedSettings
}, eventId, {
type: 'admin',
id: req.user.id,
name: req.user.username
});
res.json(updatedSettings);
} catch (error) {
logger.error('Error updating feedback settings:', error);
res.status(500).json({ error: 'Failed to update feedback settings' });
}
}
);
// Get feedback for an event (with filters)
router.get('/events/:eventId/feedback',
adminAuth,
validateEventId,
checkValidation,
async (req, res) => {
try {
const { eventId } = req.params;
const { type, status, photoId, page = 1, limit = 50 } = req.query;
// Build query
let query = db('photo_feedback')
.join('photos', 'photo_feedback.photo_id', 'photos.id')
.where('photo_feedback.event_id', eventId)
.select(
'photo_feedback.*',
'photos.filename',
'photos.path'
);
if (type) {
query = query.where('photo_feedback.feedback_type', type);
}
if (status === 'pending') {
query = query.where('photo_feedback.is_approved', false)
.where('photo_feedback.is_hidden', false);
} else if (status === 'approved') {
query = query.where('photo_feedback.is_approved', true);
} else if (status === 'hidden') {
query = query.where('photo_feedback.is_hidden', true);
}
if (photoId) {
query = query.where('photo_feedback.photo_id', photoId);
}
// Pagination
const offset = (page - 1) * limit;
const totalCount = await query.clone().count('photo_feedback.id as count').first();
const feedback = await query
.orderBy('photo_feedback.created_at', 'desc')
.limit(limit)
.offset(offset);
res.json({
feedback,
pagination: {
page: parseInt(page),
limit: parseInt(limit),
total: totalCount?.count || 0,
pages: Math.ceil((totalCount?.count || 0) / limit)
}
});
} catch (error) {
logger.error('Error getting feedback:', error);
res.status(500).json({ error: 'Failed to get feedback' });
}
}
);
// Moderate feedback (approve/hide/reject)
router.put('/feedback/:feedbackId/:action',
adminAuth,
async (req, res) => {
try {
const { feedbackId, action } = req.params;
if (!['approve', 'hide', 'reject'].includes(action)) {
return res.status(400).json({ error: 'Invalid action' });
}
await feedbackService.moderateFeedback(feedbackId, action, req.user.id);
res.json({ success: true });
} catch (error) {
logger.error('Error moderating feedback:', error);
res.status(500).json({ error: 'Failed to moderate feedback' });
}
}
);
// Delete feedback
router.delete('/feedback/:feedbackId',
adminAuth,
async (req, res) => {
try {
const { feedbackId } = req.params;
await feedbackService.deleteFeedback(feedbackId, req.user.id);
res.json({ success: true });
} catch (error) {
logger.error('Error deleting feedback:', error);
res.status(500).json({ error: 'Failed to delete feedback' });
}
}
);
// Get feedback analytics for an event
router.get('/events/:eventId/feedback-analytics',
adminAuth,
validateEventId,
checkValidation,
async (req, res) => {
try {
const { eventId } = req.params;
// Get summary statistics
const summaryData = await feedbackService.getEventFeedbackSummary(eventId);
// Calculate average rating and other summary stats
const avgRatingResult = await db('photo_feedback')
.where('event_id', eventId)
.where('feedback_type', 'rating')
.avg('rating as average_rating')
.first();
const pendingModeration = await db('photo_feedback')
.where('event_id', eventId)
.where('feedback_type', 'comment')
.where('is_approved', false)
.where('is_hidden', false)
.count('* as count')
.first();
const summary = {
average_rating: parseFloat(avgRatingResult?.average_rating || 0),
total_ratings: summaryData.stats?.total_ratings || 0,
total_likes: summaryData.stats?.total_likes || 0,
total_comments: summaryData.stats?.total_comments || 0,
total_favorites: summaryData.stats?.total_favorites || 0,
pending_moderation: pendingModeration?.count || 0,
total_feedback: (summaryData.stats?.total_ratings || 0) +
(summaryData.stats?.total_likes || 0) +
(summaryData.stats?.total_comments || 0) +
(summaryData.stats?.total_favorites || 0)
};
// Get top-rated photos
const topRated = await db('photos')
.where('event_id', eventId)
.where('average_rating', '>', 0)
.orderBy('average_rating', 'desc')
.orderBy('feedback_count', 'desc')
.limit(10)
.select('id', 'filename', 'average_rating', 'feedback_count', 'like_count');
// Get most liked photos
const mostLiked = await db('photos')
.where('event_id', eventId)
.where('like_count', '>', 0)
.orderBy('like_count', 'desc')
.limit(10)
.select('id', 'filename', 'like_count', 'average_rating');
// Get recent comments
const recentComments = await db('photo_feedback')
.join('photos', 'photo_feedback.photo_id', 'photos.id')
.where('photo_feedback.event_id', eventId)
.where('photo_feedback.feedback_type', 'comment')
.where('photo_feedback.is_approved', true)
.where('photo_feedback.is_hidden', false)
.orderBy('photo_feedback.created_at', 'desc')
.limit(10)
.select(
'photo_feedback.comment_text',
'photo_feedback.guest_name',
'photo_feedback.created_at',
'photos.filename'
);
// Get feedback timeline (last 7 days)
const timeline = await db('photo_feedback')
.where('event_id', eventId)
.where('created_at', '>', new Date(Date.now() - 7 * 24 * 60 * 60 * 1000))
.select(
db.raw('DATE(created_at) as date'),
db.raw('COUNT(*) as count'),
'feedback_type'
)
.groupBy('date', 'feedback_type')
.orderBy('date', 'asc');
res.json({
summary,
topRated,
mostLiked,
recentComments,
timeline
});
} catch (error) {
logger.error('Error getting feedback analytics:', error);
res.status(500).json({ error: 'Failed to get feedback analytics' });
}
}
);
// Export feedback data
router.get('/events/:eventId/feedback/export',
adminAuth,
validateEventId,
checkValidation,
async (req, res) => {
try {
const { eventId } = req.params;
const { format = 'json' } = req.query;
const feedback = await feedbackService.exportEventFeedback(eventId);
if (format === 'csv') {
// Convert to CSV
const csv = convertToCSV(feedback);
res.setHeader('Content-Type', 'text/csv');
res.setHeader('Content-Disposition', `attachment; filename="feedback-${eventId}.csv"`);
res.send(csv);
} else {
res.json(feedback);
}
} catch (error) {
logger.error('Error exporting feedback:', error);
res.status(500).json({ error: 'Failed to export feedback' });
}
}
);
// Get pending moderation items (across all events)
router.get('/feedback/pending-moderation',
adminAuth,
async (req, res) => {
try {
const pending = await feedbackService.getPendingModeration();
res.json(pending);
} catch (error) {
logger.error('Error getting pending moderation:', error);
res.status(500).json({ error: 'Failed to get pending moderation' });
}
}
);
// Word filter management
router.get('/feedback/word-filters',
adminAuth,
async (req, res) => {
try {
const filters = await feedbackModeration.getAllWordFilters();
res.json(filters);
} catch (error) {
logger.error('Error getting word filters:', error);
res.status(500).json({ error: 'Failed to get word filters' });
}
}
);
router.post('/feedback/word-filters',
adminAuth,
validateWordFilter,
checkValidation,
async (req, res) => {
try {
const { word, severity = 'moderate' } = req.body;
await feedbackModeration.addWordFilter(word, severity);
await logActivity('word_filter_added', { word, severity }, null, {
type: 'admin',
id: req.user.id,
name: req.user.username
});
res.json({ success: true });
} catch (error) {
if (error.message === 'Word filter already exists') {
return res.status(409).json({ error: error.message });
}
logger.error('Error adding word filter:', error);
res.status(500).json({ error: 'Failed to add word filter' });
}
}
);
router.put('/feedback/word-filters/:id',
adminAuth,
async (req, res) => {
try {
const { id } = req.params;
const updates = req.body;
await feedbackModeration.updateWordFilter(id, updates);
res.json({ success: true });
} catch (error) {
logger.error('Error updating word filter:', error);
res.status(500).json({ error: 'Failed to update word filter' });
}
}
);
router.delete('/feedback/word-filters/:id',
adminAuth,
async (req, res) => {
try {
const { id } = req.params;
await feedbackModeration.deleteWordFilter(id);
res.json({ success: true });
} catch (error) {
logger.error('Error deleting word filter:', error);
res.status(500).json({ error: 'Failed to delete word filter' });
}
}
);
// Helper function to convert JSON to CSV
function convertToCSV(data) {
if (!data || data.length === 0) return '';
const headers = Object.keys(data[0]);
const csvHeaders = headers.join(',');
const csvRows = data.map(row => {
return headers.map(header => {
const value = row[header];
// Escape quotes and wrap in quotes if contains comma
if (typeof value === 'string' && (value.includes(',') || value.includes('"'))) {
return `"${value.replace(/"/g, '""')}"`;
}
return value || '';
}).join(',');
});
return [csvHeaders, ...csvRows].join('\n');
}
module.exports = router;
+112 -248
View File
@@ -4,41 +4,55 @@ const path = require('path');
const fs = require('fs').promises;
const { db, logActivity } = require('../database/db');
const { adminAuth } = require('../middleware/auth');
const { generateThumbnail, ensureThumbnail } = require('../services/imageProcessor');
const { generateThumbnail } = require('../services/imageProcessor');
const { generatePhotoFilename } = require('../utils/filenameSanitizer');
const { escapeLikePattern } = require('../utils/sqlSecurity');
const { validateUploadedFiles } = require('../middleware/uploadValidation');
const router = express.Router();
// Get storage path from environment or default
const getStoragePath = () => process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
// Configure multer for file uploads
// IMPORTANT: Using synchronous functions to prevent file corruption
const storage = multer.diskStorage({
destination: (req, file, cb) => {
destination: async (req, file, cb) => {
console.log('Multer destination called for file:', file.originalname);
const { eventId } = req.params;
// We'll validate the event exists in the route handler
// For now, just create a temp destination
const tempPath = path.join(getStoragePath(), 'temp', `upload_${Date.now()}_${Math.random().toString(36).substring(7)}`);
// Create directory synchronously
require('fs').mkdirSync(tempPath, { recursive: true });
console.log('Temp destination path:', tempPath);
// Store temp path for cleanup
req.tempUploadPath = tempPath;
cb(null, tempPath);
try {
// Get event details
const event = await db('events').where({ id: eventId }).first();
if (!event) {
console.error('Event not found in multer destination:', eventId);
return cb(new Error('Event not found'));
}
// Store event in request for use in filename generation
req.eventData = event;
// Create destination path - now just event folder, no type subfolder
const destPath = path.join(getStoragePath(), 'events/active', event.slug);
console.log('Destination path:', destPath);
// Ensure directory exists
await fs.mkdir(destPath, { recursive: true });
cb(null, destPath);
} catch (error) {
console.error('Error in multer destination:', error);
cb(error);
}
},
filename: (req, file, cb) => {
filename: async (req, file, cb) => {
console.log('Multer filename called for file:', file.originalname);
// Use a simple temporary filename
const tempName = `temp_${Date.now()}_${Math.round(Math.random() * 1E9)}${path.extname(file.originalname)}`;
console.log('Temp filename:', tempName);
cb(null, tempName);
try {
// Use temporary filename for now, will rename after getting category info
const tempName = `temp_${Date.now()}_${Math.round(Math.random() * 1E9)}${path.extname(file.originalname)}`;
console.log('Temp filename:', tempName);
cb(null, tempName);
} catch (error) {
console.error('Error in multer filename:', error);
cb(error);
}
}
});
@@ -51,9 +65,6 @@ const upload = multer({
files: 500, // Maximum 500 files
// Set a reasonable field size limit to prevent memory issues
fieldSize: 10 * 1024 * 1024, // 10MB for non-file fields
// Add part size limits to prevent incomplete uploads
parts: 10000, // Maximum number of parts (fields + files)
headerPairs: 2000 // Maximum number of header key-value pairs
},
fileFilter: (req, file, cb) => {
// Accept images only with proper validation
@@ -64,9 +75,7 @@ const upload = multer({
} else {
cb(new Error('Only JPEG, PNG and WebP images are allowed'));
}
},
// Add abort on limit to stop processing when limits are exceeded
abortOnLimit: true
}
});
const { createFileUploadValidator } = require('../utils/fileSecurityUtils');
@@ -78,29 +87,9 @@ const validateUploadContent = createFileUploadValidator({
validateContent: true
});
// Request timeout middleware for uploads
const uploadTimeout = (timeout = 300000) => { // 5 minutes default
return (req, res, next) => {
// Set timeout for the request
req.setTimeout(timeout, () => {
console.error('Upload request timed out');
if (!res.headersSent) {
res.status(408).json({ error: 'Upload request timed out' });
}
});
// Set response timeout as well
res.setTimeout(timeout, () => {
console.error('Upload response timed out');
});
next();
};
};
// Upload photos for an event
// Increased limit to 500 files, but recommend chunked uploads for better performance
router.post('/:eventId/upload', adminAuth, uploadTimeout(600000), (req, res, next) => { // 10 minute timeout
router.post('/:eventId/upload', adminAuth, (req, res, next) => {
upload.array('photos', 500)(req, res, (err) => {
if (err) {
console.error('Multer error:', err);
@@ -117,7 +106,7 @@ router.post('/:eventId/upload', adminAuth, uploadTimeout(600000), (req, res, nex
}
next();
});
}, validateUploadContent, validateUploadedFiles, async (req, res) => {
}, validateUploadContent, async (req, res) => {
try {
const { eventId } = req.params;
const { category_id } = req.body;
@@ -132,28 +121,12 @@ router.post('/:eventId/upload', adminAuth, uploadTimeout(600000), (req, res, nex
const event = await db('events').where({ id: eventId }).first();
if (!event) {
console.error('Event not found:', eventId);
// Clean up temp files
if (req.tempUploadPath) {
try {
await fs.rm(req.tempUploadPath, { recursive: true, force: true });
} catch (e) {
console.error('Failed to clean up temp path:', e);
}
}
return res.status(404).json({ error: 'Event not found' });
}
if (!req.files || req.files.length === 0) {
console.error('No files in request. req.files:', req.files);
console.error('Request body keys:', Object.keys(req.body));
// Clean up temp files
if (req.tempUploadPath) {
try {
await fs.rm(req.tempUploadPath, { recursive: true, force: true });
} catch (e) {
console.error('Failed to clean up temp path:', e);
}
}
return res.status(400).json({ error: 'No files uploaded' });
}
@@ -165,27 +138,15 @@ router.post('/:eventId/upload', adminAuth, uploadTimeout(600000), (req, res, nex
if (parsedCategoryId) {
category = await db('photo_categories').where({ id: parsedCategoryId }).first();
if (!category) {
// Clean up temp files
if (req.tempUploadPath) {
try {
await fs.rm(req.tempUploadPath, { recursive: true, force: true });
} catch (e) {
console.error('Failed to clean up temp path:', e);
}
}
return res.status(400).json({ error: 'Invalid category' });
}
}
// Create final destination directory
const finalDestPath = path.join(getStoragePath(), 'events/active', event.slug);
await fs.mkdir(finalDestPath, { recursive: true });
const uploadedPhotos = [];
const errors = [];
// Process files in batches to optimize database operations
const BATCH_SIZE = 25; // Increased batch size for better performance with large uploads
const BATCH_SIZE = 10; // Process 10 files at a time for database operations
for (let i = 0; i < req.files.length; i += BATCH_SIZE) {
const batch = req.files.slice(i, i + BATCH_SIZE);
@@ -208,25 +169,16 @@ router.post('/:eventId/upload', adminAuth, uploadTimeout(600000), (req, res, nex
.whereNull('category_id')
.count('id as count')
.first();
batchCounter = (parseInt(uncategorizedCount.count) || 0) + 1;
batchCounter = (uncategorizedCount.count || 0) + 1;
}
const batchPhotos = [];
const fileRenameOperations = []; // Store rename operations to do after commit
// First pass: prepare data and move files from temp to final location
for (let fileIndex = 0; fileIndex < batch.length; fileIndex++) {
const file = batch[fileIndex];
const counter = batchCounter + fileIndex;
const tempPath = file.path; // Original temp path
try {
// Verify file is complete before processing
const tempStats = await fs.stat(tempPath);
if (tempStats.size === 0) {
throw new Error('File is empty - upload may have been interrupted');
}
// Generate new filename
const extension = path.extname(file.originalname);
const newFilename = generatePhotoFilename(
@@ -236,142 +188,77 @@ router.post('/:eventId/upload', adminAuth, uploadTimeout(600000), (req, res, nex
extension
);
// Calculate final path
const finalPath = path.join(finalDestPath, newFilename);
// Rename the file
const oldPath = file.path;
const newPath = path.join(path.dirname(oldPath), newFilename);
await fs.rename(oldPath, newPath);
// Update file object
file.filename = newFilename;
file.path = newPath;
// Generate thumbnail with new filename
const thumbnailPath = await generateThumbnail(file.path);
// Calculate relative paths
const storagePath = getStoragePath();
const relativePath = path.relative(path.join(storagePath, 'events/active'), finalPath);
const relativePath = path.relative(path.join(storagePath, 'events/active'), file.path);
const relativeThumbPath = thumbnailPath;
// Prepare photo data for batch insert
const photoData = {
event_id: parseInt(eventId),
filename: newFilename,
batchPhotos.push({
event_id: eventId,
filename: file.filename,
path: relativePath,
thumbnail_path: null, // Will generate after successful commit
category_id: parsedCategoryId ? parseInt(parsedCategoryId) : null,
thumbnail_path: relativeThumbPath,
category_id: parsedCategoryId || null,
type: 'individual',
size_bytes: tempStats.size // Use actual file size from stat
};
batchPhotos.push(photoData);
// Store move operation for later
fileRenameOperations.push({
tempPath: tempPath,
finalPath: finalPath,
filename: newFilename,
photoData: photoData
size_bytes: file.size
});
} catch (error) {
console.error(`Error preparing file ${file.originalname}:`, error);
console.error(`Error processing file ${file.originalname}:`, error);
errors.push({ filename: file.originalname, error: error.message });
// Delete the file if it was partially processed
if (file.path) {
try { await fs.unlink(file.path); } catch (e) {}
}
}
}
// Insert all photos in this batch
// Batch insert all photos from this batch
if (batchPhotos.length > 0) {
console.log(`Inserting batch of ${batchPhotos.length} photos with category_id: ${parsedCategoryId}`);
const insertedIds = await trx('photos').insert(batchPhotos).returning('id');
// Update category counter if needed
if (category && parsedCategoryId) {
const newCounter = batchCounter + batchPhotos.length - 1;
if (category) {
await trx('photo_categories')
.where({ id: parsedCategoryId })
.update({ photo_counter: newCounter });
console.log(`Updated category ${parsedCategoryId} counter to ${newCounter}`);
.update({ photo_counter: batchCounter + batchPhotos.length - 1 });
}
// Commit the transaction first
await trx.commit();
console.log(`Successfully committed batch of ${batchPhotos.length} photos`);
// Now move files from temp to final location after successful commit
for (let idx = 0; idx < fileRenameOperations.length; idx++) {
const operation = fileRenameOperations[idx];
try {
// Move the file from temp to final location
await fs.rename(operation.tempPath, operation.finalPath);
console.log(`Moved file from ${operation.tempPath} to ${operation.finalPath}`);
// Verify the file was moved successfully
const finalStats = await fs.stat(operation.finalPath);
if (finalStats.size !== operation.photoData.size_bytes) {
throw new Error(`File size mismatch after move: expected ${operation.photoData.size_bytes}, got ${finalStats.size}`);
}
// Generate thumbnail with final path
let thumbnailPath = null;
try {
thumbnailPath = await generateThumbnail(operation.finalPath);
// Update the database with thumbnail path
if (thumbnailPath && insertedIds[idx]) {
const photoId = insertedIds[idx]?.id || insertedIds[idx];
await db('photos')
.where({ id: photoId })
.update({ thumbnail_path: thumbnailPath });
}
} catch (thumbError) {
console.error(`Thumbnail generation failed for ${operation.filename}:`, thumbError.message);
}
// Add to successful uploads
uploadedPhotos.push({
id: insertedIds[idx]?.id || insertedIds[idx],
filename: operation.filename,
size: operation.photoData.size_bytes,
category_id: operation.photoData.category_id
});
} catch (moveError) {
console.error(`Failed to move file ${operation.tempPath} to ${operation.finalPath}:`, moveError);
errors.push({
filename: operation.filename,
error: `File move failed: ${moveError.message}`
});
// Try to clean up the database entry if file move failed
if (insertedIds[idx]) {
const photoId = insertedIds[idx]?.id || insertedIds[idx];
try {
await db('photos').where({ id: photoId }).delete();
console.log(`Cleaned up database entry for failed photo ${photoId}`);
} catch (cleanupError) {
console.error(`Failed to clean up database entry:`, cleanupError);
}
}
}
}
} else {
// No photos to insert, just rollback
await trx.rollback();
}
} catch (error) {
console.error(`Error processing batch starting at index ${i}:`, error);
console.error('Stack trace:', error.stack);
// Rollback if not already committed
if (!trx.isCompleted()) {
await trx.rollback();
}
// Add all files in this batch to errors
for (const file of batch) {
errors.push({
filename: file.originalname,
error: `Batch processing failed: ${error.message}`
// Add to uploaded photos array
batchPhotos.forEach((photo, index) => {
uploadedPhotos.push({
id: insertedIds[index]?.id || insertedIds[index],
filename: photo.filename,
size: photo.size_bytes,
category_id: photo.category_id
});
});
}
}
}
// Clean up temp upload directory
if (req.tempUploadPath) {
try {
await fs.rm(req.tempUploadPath, { recursive: true, force: true });
console.log(`Cleaned up temp upload directory: ${req.tempUploadPath}`);
} catch (e) {
console.error('Failed to clean up temp upload directory:', e);
// Commit the batch transaction
await trx.commit();
} catch (error) {
console.error(`Error processing batch starting at index ${i}:`, error);
await trx.rollback();
// Try to clean up files from failed batch
for (const file of batch) {
if (file.path) {
try { await fs.unlink(file.path); } catch (e) {}
}
}
}
}
@@ -382,39 +269,24 @@ router.post('/:eventId/upload', adminAuth, uploadTimeout(600000), (req, res, nex
{ type: 'admin', id: req.admin.id, name: req.admin.username }
);
// Include any files that were invalid from the validation middleware
const totalInvalidFiles = (req.invalidFiles || []).concat(errors);
// Prepare response
const totalAttempted = req.files.length + (req.invalidFiles ? req.invalidFiles.length : 0);
const response = {
message: `Successfully uploaded ${uploadedPhotos.length} photos`,
photos: uploadedPhotos,
totalFiles: totalAttempted,
totalFiles: req.files.length,
successCount: uploadedPhotos.length,
failureCount: totalInvalidFiles.length
failureCount: errors.length
};
// Include error details if any files failed
if (totalInvalidFiles.length > 0) {
response.errors = totalInvalidFiles;
response.message = `Uploaded ${uploadedPhotos.length} of ${totalAttempted} photos. ${totalInvalidFiles.length} failed.`;
if (errors.length > 0) {
response.errors = errors;
response.message = `Uploaded ${uploadedPhotos.length} of ${req.files.length} photos. ${errors.length} failed.`;
}
res.json(response);
} catch (error) {
console.error('Error uploading photos:', error);
// Clean up temp upload directory on error
if (req.tempUploadPath) {
try {
await fs.rm(req.tempUploadPath, { recursive: true, force: true });
console.log(`Cleaned up temp upload directory after error: ${req.tempUploadPath}`);
} catch (e) {
console.error('Failed to clean up temp upload directory:', e);
}
}
res.status(500).json({ error: 'Failed to upload photos' });
}
});
@@ -447,14 +319,9 @@ router.delete('/:eventId/photos/:photoId', adminAuth, async (req, res) => {
if (photo.thumbnail_path) {
const thumbPath = path.join(storagePath, 'events/active', photo.thumbnail_path);
try {
// Check if file exists before attempting to delete
await fs.access(thumbPath);
await fs.unlink(thumbPath);
} catch (error) {
// Only log if it's not a "file not found" error
if (error.code !== 'ENOENT') {
console.error('Error deleting thumbnail:', error);
}
console.error('Error deleting thumbnail:', error);
}
}
@@ -539,14 +406,9 @@ router.post('/:eventId/photos/bulk-delete', adminAuth, async (req, res) => {
if (photo.thumbnail_path) {
const thumbPath = path.join(storagePath, photo.thumbnail_path);
try {
// Check if file exists before attempting to delete
await fs.access(thumbPath);
await fs.unlink(thumbPath);
} catch (error) {
// Only log if it's not a "file not found" error
if (error.code !== 'ENOENT') {
console.error('Error deleting thumbnail:', error);
}
console.error('Error deleting thumbnail:', error);
}
}
}
@@ -751,24 +613,26 @@ router.get('/:eventId/thumbnail/:photoId', adminAuth, async (req, res) => {
.where({ id: photoId, event_id: eventId })
.first();
if (!photo) {
console.error(`Photo not found: ${photoId}, event ${eventId}`);
return res.status(404).json({ error: 'Photo not found' });
}
// Ensure thumbnail exists and is valid, regenerate if needed
const thumbnailPath = await ensureThumbnail(photo);
if (!thumbnailPath) {
console.error(`Failed to generate thumbnail for photo ${photoId}`);
return res.status(404).json({ error: 'Thumbnail generation failed' });
if (!photo || !photo.thumbnail_path) {
console.error(`Thumbnail not found for photo ${photoId}, event ${eventId}`);
return res.status(404).json({ error: 'Thumbnail not found' });
}
const storagePath = getStoragePath();
const filePath = path.join(storagePath, thumbnailPath);
const filePath = path.join(storagePath, photo.thumbnail_path);
console.log(`Attempting to serve thumbnail: ${filePath}`);
// Check if file exists
try {
await fs.access(filePath);
} catch (error) {
console.error(`Thumbnail file not found: ${filePath}`, error);
return res.status(404).json({ error: 'Thumbnail file not found' });
}
// Set appropriate headers
res.setHeader('Content-Type', 'image/jpeg'); // Thumbnails are always JPEG
res.setHeader('Content-Type', `image/${path.extname(photo.thumbnail_path).slice(1)}`);
res.setHeader('Cache-Control', 'private, max-age=3600');
res.setHeader('Cross-Origin-Resource-Policy', 'cross-origin');
-461
View File
@@ -1,461 +0,0 @@
const express = require('express');
const router = express.Router();
const { restoreService } = require('../services/restoreService');
const { adminAuth } = require('../middleware/auth');
const { body, query, validationResult } = require('express-validator');
const logger = require('../utils/logger');
const { db } = require('../database/db');
const path = require('path');
const fs = require('fs').promises;
/**
* Admin routes for restore operations
* All routes require admin authentication
*/
// Apply admin authentication to all routes
router.use(adminAuth);
/**
* Get restore service status and history
*/
router.get('/status', async (req, res) => {
try {
const limit = parseInt(req.query.limit) || 10;
const history = await restoreService.getRestoreHistory(limit);
const status = {
isRunning: restoreService.isRunning,
currentProgress: restoreService.getProgress(),
history: history,
settings: await getRestoreSettings()
};
res.json({
success: true,
data: status
});
} catch (error) {
logger.error('Failed to get restore status:', error);
res.status(500).json({
success: false,
error: 'Failed to get restore status'
});
}
});
/**
* Validate restore request
*/
router.post('/validate', [
body('source').notEmpty().withMessage('Backup source is required'),
body('manifestPath').notEmpty().withMessage('Manifest path is required'),
body('restoreType').isIn(['full', 'database', 'files', 'selective']).withMessage('Invalid restore type'),
body('selectedItems').optional().isArray(),
body('s3Config').optional().isObject()
], async (req, res) => {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({
success: false,
errors: errors.array()
});
}
try {
// Perform dry run validation
const result = await restoreService.restore({
...req.body,
dryRun: true,
force: false
});
res.json({
success: true,
data: {
validation: result.validation,
spaceCheck: result.spaceCheck,
logs: result.logs
}
});
} catch (error) {
logger.error('Restore validation failed:', error);
res.status(400).json({
success: false,
error: error.message,
logs: restoreService.restoreLog
});
}
});
/**
* Start restore operation
*/
router.post('/start', [
body('source').notEmpty().withMessage('Backup source is required'),
body('manifestPath').notEmpty().withMessage('Manifest path is required'),
body('restoreType').isIn(['full', 'database', 'files', 'selective']).withMessage('Invalid restore type'),
body('selectedItems').optional().isArray(),
body('skipPreBackup').optional().isBoolean(),
body('force').optional().isBoolean(),
body('s3Config').optional().isObject()
], async (req, res) => {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({
success: false,
errors: errors.array()
});
}
try {
// Check if restore is already running
if (restoreService.isRunning) {
return res.status(409).json({
success: false,
error: 'Restore operation already in progress'
});
}
// Check permissions for dangerous options
const settings = await getRestoreSettings();
if (req.body.force && !settings.restore_allow_force) {
return res.status(403).json({
success: false,
error: 'Force restore is not allowed by system settings'
});
}
if (req.body.skipPreBackup && settings.restore_require_pre_backup) {
return res.status(403).json({
success: false,
error: 'Skipping pre-restore backup is not allowed by system settings'
});
}
// Log restore attempt
logger.warn('Restore operation started', {
user: req.user.email,
ip: req.ip,
restoreType: req.body.restoreType,
source: req.body.source
});
// Start restore in background
restoreService.restore({
...req.body,
dryRun: false,
operator: {
type: 'manual',
userId: req.user.id,
ip: req.ip
}
}).catch(error => {
logger.error('Background restore failed:', error);
});
res.json({
success: true,
message: 'Restore operation started'
});
} catch (error) {
logger.error('Failed to start restore:', error);
res.status(500).json({
success: false,
error: error.message
});
}
});
/**
* Get current restore progress
*/
router.get('/progress', async (req, res) => {
try {
const progress = restoreService.getProgress();
const logs = restoreService.restoreLog.slice(-50); // Last 50 log entries
res.json({
success: true,
data: {
isRunning: restoreService.isRunning,
progress: progress,
logs: logs
}
});
} catch (error) {
logger.error('Failed to get restore progress:', error);
res.status(500).json({
success: false,
error: 'Failed to get restore progress'
});
}
});
/**
* Get restore run details
*/
router.get('/run/:id', async (req, res) => {
try {
const run = await db('restore_runs')
.where('id', req.params.id)
.first();
if (!run) {
return res.status(404).json({
success: false,
error: 'Restore run not found'
});
}
// Parse JSON fields
if (run.statistics) run.statistics = JSON.parse(run.statistics);
if (run.restore_log) run.restore_log = JSON.parse(run.restore_log);
if (run.metadata) run.metadata = JSON.parse(run.metadata);
// Get validation results
const validations = await db('restore_validation_results')
.where('restore_run_id', run.id)
.select('*');
validations.forEach(v => {
if (v.errors) v.errors = JSON.parse(v.errors);
if (v.warnings) v.warnings = JSON.parse(v.warnings);
if (v.checksums) v.checksums = JSON.parse(v.checksums);
});
// Get file operations summary
const fileOps = await db('restore_file_operations')
.where('restore_run_id', run.id)
.select('status', db.raw('COUNT(*) as count'))
.groupBy('status');
res.json({
success: true,
data: {
run: run,
validations: validations,
fileOperations: fileOps
}
});
} catch (error) {
logger.error('Failed to get restore run details:', error);
res.status(500).json({
success: false,
error: 'Failed to get restore run details'
});
}
});
/**
* Get restore run report
*/
router.get('/run/:id/report', async (req, res) => {
try {
const run = await db('restore_runs')
.where('id', req.params.id)
.first();
if (!run) {
return res.status(404).json({
success: false,
error: 'Restore run not found'
});
}
// Parse JSON fields
if (run.statistics) run.statistics = JSON.parse(run.statistics);
if (run.restore_log) run.restore_log = JSON.parse(run.restore_log);
// Generate report
const report = restoreService.generateRestoreReport({
success: run.status === 'completed',
duration: run.duration_seconds,
dryRun: run.is_dry_run,
result: run.statistics,
logs: run.restore_log || []
});
res.type('text/plain').send(report);
} catch (error) {
logger.error('Failed to generate restore report:', error);
res.status(500).json({
success: false,
error: 'Failed to generate restore report'
});
}
});
/**
* List available backups for restore
*/
router.get('/available-backups', async (req, res) => {
try {
const backups = [];
// Get local file backups
const backupConfig = await getBackupConfig();
if (backupConfig.backup_destination_type === 'local' && backupConfig.backup_destination_path) {
try {
const files = await fs.readdir(backupConfig.backup_destination_path);
for (const file of files) {
if (file.endsWith('.json') || file.endsWith('.yaml')) {
const filePath = path.join(backupConfig.backup_destination_path, file);
const stats = await fs.stat(filePath);
backups.push({
type: 'local',
name: file,
path: filePath,
size: stats.size,
modified: stats.mtime
});
}
}
} catch (error) {
logger.warn('Failed to list local backups:', error);
}
}
// Get database backups from backup_runs table
const backupRuns = await db('backup_runs')
.where('status', 'completed')
.whereNotNull('manifest_path')
.orderBy('completed_at', 'desc')
.limit(20);
for (const run of backupRuns) {
backups.push({
type: run.manifest_path.startsWith('s3://') ? 's3' : 'local',
name: `Backup ${run.completed_at}`,
path: run.manifest_path,
manifestId: run.manifest_id,
size: run.total_size_bytes,
filesCount: run.files_backed_up,
duration: run.duration_seconds,
completed: run.completed_at
});
}
res.json({
success: true,
data: backups
});
} catch (error) {
logger.error('Failed to list available backups:', error);
res.status(500).json({
success: false,
error: 'Failed to list available backups'
});
}
});
/**
* Get restore settings
*/
router.get('/settings', async (req, res) => {
try {
const settings = await getRestoreSettings();
res.json({
success: true,
data: settings
});
} catch (error) {
logger.error('Failed to get restore settings:', error);
res.status(500).json({
success: false,
error: 'Failed to get restore settings'
});
}
});
/**
* Update restore settings
*/
router.put('/settings', [
body('restore_allow_force').optional().isBoolean(),
body('restore_require_pre_backup').optional().isBoolean(),
body('restore_max_file_size_mb').optional().isInt({ min: 1 }),
body('restore_verify_checksums').optional().isBoolean(),
body('restore_email_on_completion').optional().isBoolean(),
body('restore_retention_days').optional().isInt({ min: 1 })
], async (req, res) => {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({
success: false,
errors: errors.array()
});
}
try {
// Update settings
for (const [key, value] of Object.entries(req.body)) {
await db('app_settings')
.where('setting_key', key)
.where('setting_type', 'restore')
.update({
setting_value: typeof value === 'boolean' ? (value ? '1' : '0') : value.toString(),
updated_at: db.fn.now()
});
}
logger.info('Restore settings updated', {
user: req.user.email,
settings: req.body
});
res.json({
success: true,
message: 'Settings updated successfully'
});
} catch (error) {
logger.error('Failed to update restore settings:', error);
res.status(500).json({
success: false,
error: 'Failed to update settings'
});
}
});
/**
* Helper function to get restore settings
*/
async function getRestoreSettings() {
const settings = await db('app_settings')
.where('setting_type', 'restore')
.select('setting_key', 'setting_value');
const result = {};
settings.forEach(setting => {
// Convert boolean strings to actual booleans
if (setting.setting_value === '1' || setting.setting_value === '0') {
result[setting.setting_key] = setting.setting_value === '1';
} else {
result[setting.setting_key] = setting.setting_value;
}
});
return result;
}
/**
* Helper function to get backup configuration
*/
async function getBackupConfig() {
const settings = await db('app_settings')
.where('setting_type', 'backup')
.select('setting_key', 'setting_value');
const config = {};
settings.forEach(setting => {
try {
config[setting.setting_key] = JSON.parse(setting.setting_value);
} catch (e) {
config[setting.setting_key] = setting.setting_value;
}
});
return config;
}
module.exports = router;
-102
View File
@@ -7,7 +7,6 @@ const { db, logActivity } = require('../database/db');
const { formatBoolean } = require('../utils/dbCompat');
const { adminAuth } = require('../middleware/auth');
const { clearMaintenanceCache } = require('../middleware/maintenance');
const { clearSettingsCache } = require('../services/rateLimitService');
const router = express.Router();
// Configure multer for logo uploads
@@ -496,43 +495,6 @@ router.put('/security', adminAuth, async (req, res) => {
}
});
// Update analytics settings
router.put('/analytics', adminAuth, async (req, res) => {
try {
const settings = req.body;
// Update or insert each setting
for (const [key, value] of Object.entries(settings)) {
await db('app_settings')
.insert({
setting_key: key,
setting_value: JSON.stringify(value),
setting_type: 'analytics',
updated_at: new Date()
})
.onConflict('setting_key')
.merge({
setting_value: JSON.stringify(value),
updated_at: new Date()
});
}
// Log activity
await db('activity_logs').insert({
activity_type: 'analytics_settings_updated',
actor_type: 'admin',
actor_id: req.admin.id,
actor_name: req.admin.username,
metadata: JSON.stringify({ settings_count: Object.keys(settings).length })
});
res.json({ message: 'Analytics settings updated successfully' });
} catch (error) {
console.error('Analytics settings update error:', error);
res.status(500).json({ error: 'Failed to update analytics settings' });
}
});
// Get storage info
router.get('/storage/info', adminAuth, async (req, res) => {
try {
@@ -620,68 +582,4 @@ router.post('/favicon', adminAuth, faviconUpload.single('favicon'), async (req,
}
});
// Update rate limit settings
router.put('/security/rate-limit', adminAuth, [
body('rate_limit_enabled').isBoolean().withMessage('Enabled must be a boolean'),
body('rate_limit_window_minutes').isInt({ min: 1, max: 60 }).withMessage('Window must be between 1 and 60 minutes'),
body('rate_limit_max_requests').isInt({ min: 10, max: 10000 }).withMessage('Max requests must be between 10 and 10000'),
body('rate_limit_auth_max_requests').isInt({ min: 1, max: 100 }).withMessage('Auth max requests must be between 1 and 100'),
body('rate_limit_skip_authenticated').isBoolean().withMessage('Skip authenticated must be a boolean'),
body('rate_limit_public_endpoints_only').isBoolean().withMessage('Public endpoints only must be a boolean')
], async (req, res) => {
try {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({ errors: errors.array() });
}
const {
rate_limit_enabled,
rate_limit_window_minutes,
rate_limit_max_requests,
rate_limit_auth_max_requests,
rate_limit_skip_authenticated,
rate_limit_public_endpoints_only
} = req.body;
// Update each setting
const settings = [
{ key: 'rate_limit_enabled', value: rate_limit_enabled },
{ key: 'rate_limit_window_minutes', value: rate_limit_window_minutes },
{ key: 'rate_limit_max_requests', value: rate_limit_max_requests },
{ key: 'rate_limit_auth_max_requests', value: rate_limit_auth_max_requests },
{ key: 'rate_limit_skip_authenticated', value: rate_limit_skip_authenticated },
{ key: 'rate_limit_public_endpoints_only', value: rate_limit_public_endpoints_only }
];
for (const { key, value } of settings) {
await db('app_settings')
.where('setting_key', key)
.update({
setting_value: JSON.stringify(value),
updated_at: new Date()
});
}
// Clear the rate limit settings cache to apply changes immediately
clearSettingsCache();
// Log activity
await logActivity('settings_updated',
{
category: 'security',
subcategory: 'rate_limit',
changes: settings.length
},
null,
{ type: 'admin', id: req.admin.id, name: req.admin.username }
);
res.json({ message: 'Rate limit settings updated successfully' });
} catch (error) {
console.error('Rate limit settings update error:', error);
res.status(500).json({ error: 'Failed to update rate limit settings' });
}
});
module.exports = router;
+1 -1
View File
@@ -1,5 +1,5 @@
const express = require('express');
const { db, withRetry } = require('../database/db');
const { db } = require('../database/db');
const { adminAuth } = require('../middleware/auth-enhanced-v2');
const fs = require('fs').promises;
const path = require('path');
+2 -2
View File
@@ -87,10 +87,10 @@ router.post('/', adminAuth, [
await queueEmail(eventId, host_email, 'gallery_created', {
host_name: host_email.split('@')[0], // Extract name from email
event_name,
event_date: event_date, // Pass raw date - will be formatted by email processor
event_date: new Date(event_date).toLocaleDateString(),
gallery_link: shareLink,
gallery_password: password,
expiry_date: expires_at.toISOString(), // Pass ISO string - will be formatted by email processor
expiry_date: expires_at.toLocaleDateString(),
welcome_message: welcome_message || ''
});
-342
View File
@@ -1,342 +0,0 @@
const express = require('express');
const router = express.Router();
const { photoAuth } = require('../middleware/photoAuth');
const { verifyGalleryAccess } = require('../middleware/gallery');
const { feedbackRateLimit, generateGuestIdentifier } = require('../middleware/feedbackRateLimit');
const feedbackService = require('../services/feedbackService');
const feedbackModeration = require('../services/feedbackModeration');
const { db, logActivity } = require('../database/db');
const logger = require('../utils/logger');
const {
validatePhotoId,
validateFeedbackSubmission,
checkValidation,
validateGuestRequirements
} = require('../utils/feedbackValidation');
const { escapeLikePattern } = require('../utils/sqlSecurity');
// Get feedback settings for a gallery
router.get('/:slug/feedback-settings',
verifyGalleryAccess,
async (req, res) => {
try {
const event = req.event;
const settings = await feedbackService.getEventFeedbackSettings(event.id);
// Only send relevant settings to guests
const guestSettings = {
feedback_enabled: settings.feedback_enabled,
allow_ratings: settings.allow_ratings,
allow_likes: settings.allow_likes,
allow_comments: settings.allow_comments,
allow_favorites: settings.allow_favorites,
require_name_email: settings.require_name_email,
show_feedback_to_guests: settings.show_feedback_to_guests
};
res.json(guestSettings);
} catch (error) {
logger.error('Error getting feedback settings:', error);
res.status(500).json({ error: 'Failed to get feedback settings' });
}
}
);
// Get feedback for a specific photo
router.get('/:slug/photos/:photoId/feedback',
verifyGalleryAccess,
validatePhotoId,
checkValidation,
async (req, res) => {
try {
const { photoId } = req.params;
const event = req.event;
const guestIdentifier = generateGuestIdentifier(req);
// Get feedback settings
const settings = await feedbackService.getEventFeedbackSettings(event.id);
if (!settings.feedback_enabled) {
return res.status(403).json({ error: 'Feedback is not enabled for this event' });
}
// Verify photo belongs to event
const photo = await db('photos')
.where({ id: photoId, event_id: event.id })
.first();
if (!photo) {
return res.status(404).json({ error: 'Photo not found' });
}
// Get feedback based on settings
const options = {
approved_only: true,
include_hidden: false
};
// Include guest's own feedback even if not approved
const feedback = await feedbackService.getPhotoFeedback(photoId, options);
// Get guest's own feedback separately
const guestFeedback = await feedbackService.getPhotoFeedback(photoId, {
guest_identifier: guestIdentifier
});
// Combine and deduplicate
const allFeedback = [...feedback];
guestFeedback.forEach(gf => {
if (!feedback.find(f => f.id === gf.id)) {
allFeedback.push({ ...gf, is_mine: true });
} else {
const index = allFeedback.findIndex(f => f.id === gf.id);
allFeedback[index].is_mine = true;
}
});
// Filter based on what guests should see
const visibleFeedback = settings.show_feedback_to_guests ? allFeedback :
allFeedback.filter(f => f.is_mine);
res.json({
feedback: visibleFeedback,
summary: {
average_rating: photo.average_rating || 0,
total_ratings: await db('photo_feedback')
.where({ photo_id: photoId, feedback_type: 'rating', is_hidden: false })
.count('id as count')
.first()
.then(r => r.count),
like_count: photo.like_count || 0,
favorite_count: photo.favorite_count || 0,
comment_count: await db('photo_feedback')
.where({
photo_id: photoId,
feedback_type: 'comment',
is_approved: true,
is_hidden: false
})
.count('id as count')
.first()
.then(r => r.count)
},
my_feedback: {
rating: guestFeedback.find(f => f.feedback_type === 'rating')?.rating,
liked: !!guestFeedback.find(f => f.feedback_type === 'like'),
favorited: !!guestFeedback.find(f => f.feedback_type === 'favorite')
}
});
} catch (error) {
logger.error('Error getting photo feedback:', error);
res.status(500).json({ error: 'Failed to get feedback' });
}
}
);
// Submit feedback for a photo
router.post('/:slug/photos/:photoId/feedback',
verifyGalleryAccess,
validatePhotoId,
validateFeedbackSubmission,
checkValidation,
async (req, res) => {
try {
const { photoId } = req.params;
const event = req.event;
const guestIdentifier = generateGuestIdentifier(req);
// Get feedback settings
const settings = await feedbackService.getEventFeedbackSettings(event.id);
if (!settings.feedback_enabled) {
return res.status(403).json({ error: 'Feedback is not enabled for this event' });
}
// Check if specific feedback type is allowed
const feedbackType = req.body.feedback_type;
const typeAllowed = {
rating: settings.allow_ratings,
like: settings.allow_likes,
comment: settings.allow_comments,
favorite: settings.allow_favorites
};
if (!typeAllowed[feedbackType]) {
return res.status(403).json({ error: `${feedbackType} feedback is not enabled` });
}
// Verify photo belongs to event
const photo = await db('photos')
.where({ id: photoId, event_id: event.id })
.first();
if (!photo) {
return res.status(404).json({ error: 'Photo not found' });
}
// Validate guest requirements
const guestValidation = await validateGuestRequirements(settings, req.body);
if (!guestValidation.valid) {
return res.status(400).json({
error: 'Guest information required',
errors: guestValidation.errors
});
}
// Apply rate limiting based on feedback type
const rateLimitMiddleware = feedbackRateLimit(feedbackType);
await new Promise((resolve, reject) => {
rateLimitMiddleware(req, res, (err) => {
if (err) reject(err);
else resolve();
});
});
// If we got here and response was sent (rate limited), return
if (res.headersSent) return;
// Prepare feedback data
const feedbackData = {
feedback_type: feedbackType,
rating: req.body.rating,
comment_text: req.body.comment_text,
guest_name: req.body.guest_name,
guest_email: req.body.guest_email,
ip_address: req.ip || req.connection.remoteAddress,
user_agent: req.headers['user-agent'],
moderate_comments: settings.moderate_comments
};
// For comments, check moderation
if (feedbackType === 'comment') {
// Check user reputation
const reputation = await feedbackModeration.checkUserReputation(guestIdentifier, event.id);
// Moderate the comment
const moderationResult = await feedbackModeration.moderateText(req.body.comment_text);
if (!moderationResult.approved) {
// Still save but mark as not approved
feedbackData.is_approved = false;
logger.warn('Comment flagged for moderation:', {
reason: moderationResult.reason,
violations: moderationResult.violations
});
} else if (reputation.autoApprove) {
// Trusted user, auto-approve
feedbackData.is_approved = true;
} else if (settings.moderate_comments) {
// Default moderation setting
feedbackData.is_approved = false;
}
}
// Submit feedback
const result = await feedbackService.submitFeedback(
photoId,
event.id,
feedbackData,
guestIdentifier
);
// Log activity
await logActivity(`guest_feedback_${feedbackType}`, {
photo_id: photoId,
result
}, event.id, {
type: 'guest',
id: guestIdentifier.substring(0, 16),
name: req.body.guest_name || 'Anonymous'
});
res.json({
success: true,
...result,
message: feedbackType === 'comment' && !feedbackData.is_approved ?
'Your comment has been submitted for moderation' : undefined
});
} catch (error) {
logger.error('Error submitting feedback:', error);
res.status(500).json({ error: 'Failed to submit feedback' });
}
}
);
// Get feedback summary for entire gallery
router.get('/:slug/feedback-summary',
verifyGalleryAccess,
async (req, res) => {
try {
const event = req.event;
// Get feedback settings
const settings = await feedbackService.getEventFeedbackSettings(event.id);
if (!settings.feedback_enabled || !settings.show_feedback_to_guests) {
return res.json({
enabled: false,
summary: null
});
}
const summary = await feedbackService.getEventFeedbackSummary(event.id);
// Filter data based on what guests should see
const guestSummary = {
stats: summary.stats,
top_rated: summary.photos
.filter(p => p.average_rating > 0)
.slice(0, 5)
.map(p => ({
id: p.id,
filename: p.filename,
average_rating: p.average_rating,
like_count: p.like_count
}))
};
res.json({
enabled: true,
settings: {
allow_ratings: settings.allow_ratings,
allow_likes: settings.allow_likes,
allow_comments: settings.allow_comments,
allow_favorites: settings.allow_favorites
},
summary: guestSummary
});
} catch (error) {
logger.error('Error getting feedback summary:', error);
res.status(500).json({ error: 'Failed to get feedback summary' });
}
}
);
// Get user's own feedback for all photos
router.get('/:slug/my-feedback',
verifyGalleryAccess,
async (req, res) => {
try {
const event = req.event;
const guestIdentifier = generateGuestIdentifier(req);
const myFeedback = await db('photo_feedback')
.join('photos', 'photo_feedback.photo_id', 'photos.id')
.where('photo_feedback.event_id', event.id)
.where('photo_feedback.guest_identifier', guestIdentifier)
.select(
'photo_feedback.*',
'photos.filename',
'photos.path'
)
.orderBy('photo_feedback.created_at', 'desc');
res.json(myFeedback);
} catch (error) {
logger.error('Error getting user feedback:', error);
res.status(500).json({ error: 'Failed to get your feedback' });
}
}
);
module.exports = router;
+6 -18
View File
@@ -1,20 +1,14 @@
const express = require('express');
const { db, withRetry } = require('../database/db');
const { db } = require('../database/db');
const router = express.Router();
// Get public settings (branding and theme)
router.get('/', async (req, res) => {
try {
// Fetch branding, theme, general, and security settings
// Note: We include analytics in the query but it might not exist yet
const settings = await withRetry(async () => {
return await db('app_settings')
.where(function() {
this.whereIn('setting_type', ['branding', 'theme', 'general', 'security', 'analytics'])
.orWhere('setting_key', 'like', 'analytics_%');
})
.select('setting_key', 'setting_value');
});
// Fetch branding, theme, general, and select security settings
const settings = await db('app_settings')
.whereIn('setting_type', ['branding', 'theme', 'general', 'security'])
.select('setting_key', 'setting_value');
// Convert to object format
const settingsObject = {};
@@ -45,15 +39,9 @@ router.get('/', async (req, res) => {
theme_config: settingsObject.theme_config || null,
default_language: settingsObject.general_default_language || 'en',
enable_analytics: settingsObject.general_enable_analytics !== false,
general_date_format: settingsObject.general_date_format || 'PPP',
enable_recaptcha: settingsObject.security_enable_recaptcha === true || settingsObject.security_enable_recaptcha === 'true',
recaptcha_site_key: settingsObject.security_recaptcha_site_key || null,
maintenance_mode: settingsObject.general_maintenance_mode === true || settingsObject.general_maintenance_mode === 'true',
// Umami analytics configuration (only if enabled)
umami_enabled: settingsObject.analytics_umami_enabled === true || settingsObject.analytics_umami_enabled === 'true',
umami_url: (settingsObject.analytics_umami_enabled === true || settingsObject.analytics_umami_enabled === 'true') ? (settingsObject.analytics_umami_url || null) : null,
umami_website_id: (settingsObject.analytics_umami_enabled === true || settingsObject.analytics_umami_enabled === 'true') ? (settingsObject.analytics_umami_website_id || null) : null,
umami_share_url: (settingsObject.analytics_umami_enabled === true || settingsObject.analytics_umami_enabled === 'true') ? (settingsObject.analytics_umami_share_url || null) : null
maintenance_mode: settingsObject.general_maintenance_mode === true || settingsObject.general_maintenance_mode === 'true'
};
res.json(publicSettings);
@@ -1,251 +0,0 @@
const { DatabaseBackupService } = require('../databaseBackup');
const { db } = require('../../database/db');
const fs = require('fs').promises;
const path = require('path');
const crypto = require('crypto');
// Mock dependencies
jest.mock('../../database/db');
jest.mock('../../utils/logger');
jest.mock('../emailProcessor');
jest.mock('child_process');
describe('DatabaseBackupService', () => {
let service;
let mockExecAsync;
beforeEach(() => {
service = new DatabaseBackupService();
mockExecAsync = jest.fn();
// Reset mocks
jest.clearAllMocks();
// Mock execAsync
const childProcess = require('child_process');
childProcess.exec = jest.fn((cmd, opts, callback) => {
if (callback) {
callback(null, { stdout: 'ok', stderr: '' });
}
});
});
afterEach(async () => {
// Cleanup test files
try {
await fs.rmdir('/tmp/test-backup', { recursive: true });
} catch (e) {
// Ignore
}
});
describe('calculateChecksum', () => {
it('should calculate SHA256 checksum of a file', async () => {
const testFile = '/tmp/test-checksum.txt';
const testContent = 'Hello, World!';
await fs.writeFile(testFile, testContent);
const checksum = await service.calculateChecksum(testFile);
// Expected checksum for "Hello, World!"
const expectedChecksum = crypto
.createHash('sha256')
.update(testContent)
.digest('hex');
expect(checksum).toBe(expectedChecksum);
await fs.unlink(testFile);
});
});
describe('getTableChecksums', () => {
it('should get checksums for all tables', async () => {
// Mock getTables
service.getTables = jest.fn().mockResolvedValue(['events', 'photos']);
// Mock SQLite response
db.raw = jest.fn()
.mockResolvedValueOnce([{ row_count: 10, data_sum: 1000 }])
.mockResolvedValueOnce([{ row_count: 20, data_sum: 2000 }]);
const checksums = await service.getTableChecksums();
expect(checksums).toHaveProperty('events');
expect(checksums).toHaveProperty('photos');
expect(checksums.events.rowCount).toBe(10);
expect(checksums.photos.rowCount).toBe(20);
expect(checksums.events.checksum).toBeDefined();
expect(checksums.photos.checksum).toBeDefined();
});
});
describe('getTables', () => {
it('should get list of tables for SQLite', async () => {
service.dbType = 'sqlite';
db.raw = jest.fn().mockResolvedValue([
{ name: 'events' },
{ name: 'photos' },
{ name: 'admin_users' }
]);
const tables = await service.getTables();
expect(tables).toEqual(['events', 'photos', 'admin_users']);
expect(db.raw).toHaveBeenCalledWith(expect.stringContaining('sqlite_master'));
});
it('should get list of tables for PostgreSQL', async () => {
service.dbType = 'postgresql';
db.raw = jest.fn().mockResolvedValue({
rows: [
{ table_name: 'events' },
{ table_name: 'photos' },
{ table_name: 'admin_users' }
]
});
const tables = await service.getTables();
expect(tables).toEqual(['events', 'photos', 'admin_users']);
expect(db.raw).toHaveBeenCalledWith(expect.stringContaining('information_schema.tables'));
});
});
describe('getDatabaseSize', () => {
it('should get database size for SQLite', async () => {
service.dbType = 'sqlite';
const mockSize = 1024 * 1024 * 10; // 10MB
// Mock fs.stat
const originalStat = fs.stat;
fs.stat = jest.fn().mockResolvedValue({ size: mockSize });
const size = await service.getDatabaseSize();
expect(size).toBe(mockSize);
fs.stat = originalStat;
});
it('should get database size for PostgreSQL', async () => {
service.dbType = 'postgresql';
const mockSize = 1024 * 1024 * 100; // 100MB
db.raw = jest.fn().mockResolvedValue({
rows: [{ size: mockSize.toString() }]
});
const size = await service.getDatabaseSize();
expect(size).toBe(mockSize);
expect(db.raw).toHaveBeenCalledWith(expect.stringContaining('pg_database_size'));
});
});
describe('compressFile', () => {
it('should compress file and return stats', async () => {
const testFile = '/tmp/test-compress.txt';
const compressedFile = '/tmp/test-compress.txt.gz';
// Create test file with repetitive content (compresses well)
const testContent = 'Hello, World! '.repeat(1000);
await fs.writeFile(testFile, testContent);
const stats = await service.compressFile(testFile, compressedFile);
expect(stats.originalSize).toBeGreaterThan(0);
expect(stats.compressedSize).toBeGreaterThan(0);
expect(stats.compressedSize).toBeLessThan(stats.originalSize);
expect(parseFloat(stats.compressionRatio)).toBeGreaterThan(0);
// Cleanup
await fs.unlink(testFile);
await fs.unlink(compressedFile);
});
});
describe('backup configuration', () => {
it('should get backup configuration from database', async () => {
const mockConfig = [
{ setting_key: 'database_backup_enabled', setting_value: 'true' },
{ setting_key: 'database_backup_compress', setting_value: 'true' },
{ setting_key: 'database_backup_retention_days', setting_value: '30' }
];
db.mockReturnValue({
where: jest.fn().mockReturnThis(),
select: jest.fn().mockResolvedValue(mockConfig)
});
const config = await service.getBackupConfig();
expect(config.database_backup_enabled).toBe(true);
expect(config.database_backup_compress).toBe(true);
expect(config.database_backup_retention_days).toBe(30);
});
});
describe('cleanupOldBackups', () => {
it('should delete old backup files and records', async () => {
const oldBackups = [
{ id: 1, file_path: '/backup/old1.sql.gz' },
{ id: 2, file_path: '/backup/old2.sql.gz' }
];
db.mockReturnValue({
where: jest.fn().mockReturnThis(),
select: jest.fn().mockResolvedValue(oldBackups),
delete: jest.fn().mockResolvedValue(1)
});
// Mock fs.unlink
fs.unlink = jest.fn().mockResolvedValue(undefined);
await service.cleanupOldBackups(30);
expect(fs.unlink).toHaveBeenCalledTimes(2);
expect(fs.unlink).toHaveBeenCalledWith('/backup/old1.sql.gz');
expect(fs.unlink).toHaveBeenCalledWith('/backup/old2.sql.gz');
});
});
describe('progress tracking', () => {
it('should update and retrieve progress', () => {
expect(service.getProgress()).toBeNull();
service.updateProgress('Testing...', { step: 1 });
const progress = service.getProgress();
expect(progress.message).toBe('Testing...');
expect(progress.details.step).toBe(1);
expect(progress.timestamp).toBeDefined();
});
});
describe('backup history', () => {
it('should retrieve backup history', async () => {
const mockHistory = [
{
id: 1,
started_at: new Date(),
completed_at: new Date(),
status: 'completed',
file_size_bytes: 1024000
}
];
db.mockReturnValue({
orderBy: jest.fn().mockReturnThis(),
limit: jest.fn().mockResolvedValue(mockHistory)
});
const history = await service.getBackupHistory(10);
expect(history).toEqual(mockHistory);
expect(history.length).toBe(1);
});
});
});
-53
View File
@@ -4,7 +4,6 @@ const path = require('path');
const { db } = require('../database/db');
const { queueEmail } = require('./emailProcessor');
const logger = require('../utils/logger');
const feedbackService = require('./feedbackService');
const getStoragePath = () => process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
const ACTIVE_PATH = () => path.join(getStoragePath(), 'events/active');
@@ -29,37 +28,6 @@ async function archiveEvent(event) {
throw err;
});
// Export feedback data before archiving
const feedbackSettings = await feedbackService.getEventFeedbackSettings(event.id);
if (feedbackSettings.feedback_enabled) {
try {
logger.info(`Exporting feedback data for event ${event.slug}`);
const feedbackData = await feedbackService.exportEventFeedback(event.id);
if (feedbackData && feedbackData.length > 0) {
// Create feedback JSON file
const feedbackJson = JSON.stringify(feedbackData, null, 2);
const feedbackJsonPath = path.join(eventPath, 'feedback_data.json');
await fs.writeFile(feedbackJsonPath, feedbackJson, 'utf8');
// Create feedback CSV file
const feedbackCsv = convertToCSV(feedbackData);
const feedbackCsvPath = path.join(eventPath, 'feedback_data.csv');
await fs.writeFile(feedbackCsvPath, feedbackCsv, 'utf8');
// Create feedback summary
const summary = await feedbackService.getEventFeedbackSummary(event.id);
const summaryPath = path.join(eventPath, 'feedback_summary.json');
await fs.writeFile(summaryPath, JSON.stringify(summary, null, 2), 'utf8');
logger.info(`Feedback data exported: ${feedbackData.length} entries`);
}
} catch (error) {
logger.error(`Error exporting feedback for event ${event.slug}:`, error);
// Continue with archiving even if feedback export fails
}
}
output.on('close', async () => {
logger.info(`Archive created: ${archiveName} (${archive.pointer()} bytes)`);
@@ -99,25 +67,4 @@ async function archiveEvent(event) {
}
}
// Helper function to convert JSON to CSV
function convertToCSV(data) {
if (!data || data.length === 0) return '';
const headers = Object.keys(data[0]);
const csvHeaders = headers.join(',');
const csvRows = data.map(row => {
return headers.map(header => {
const value = row[header];
// Escape quotes and wrap in quotes if contains comma
if (typeof value === 'string' && (value.includes(',') || value.includes('"'))) {
return `"${value.replace(/"/g, '""')}"`;
}
return value || '';
}).join(',');
});
return [csvHeaders, ...csvRows].join('\n');
}
module.exports = { archiveEvent };
-501
View File
@@ -1,501 +0,0 @@
const fs = require('fs').promises;
const path = require('path');
const crypto = require('crypto');
const yaml = require('js-yaml');
const os = require('os');
const { db } = require('../database/db');
const logger = require('../utils/logger');
/**
* Backup Manifest Generator
*
* Generates comprehensive manifests for backups including:
* - Version information (app, node, OS)
* - File listings with metadata and checksums
* - Database information
* - System state at backup time
* - Support for both JSON and YAML formats
* - Incremental backup support with parent references
*/
class BackupManifestGenerator {
constructor() {
this.appVersion = require('../../package.json').version;
this.nodeVersion = process.version;
this.platform = process.platform;
this.osRelease = os.release();
this.hostname = os.hostname();
}
/**
* Generate a comprehensive backup manifest
* @param {Object} options - Manifest generation options
* @param {string} options.backupType - 'full' or 'incremental'
* @param {string} options.backupPath - Path to the backup directory
* @param {Array} options.files - Array of backed up files with metadata
* @param {Object} options.databaseInfo - Database backup information
* @param {string} options.parentBackupId - For incremental backups, reference to parent
* @param {string} options.format - 'json' or 'yaml' (default: 'json')
* @param {Object} options.customMetadata - Additional metadata to include
* @returns {Object} Generated manifest object
*/
async generateManifest(options) {
const {
backupType = 'full',
backupPath,
files = [],
databaseInfo = {},
parentBackupId = null,
format = 'json',
customMetadata = {}
} = options;
const manifest = {
// Manifest metadata
manifest: {
version: '2.0',
created: new Date().toISOString(),
generator: 'PicPeak Backup Manifest Generator',
format: format
},
// Backup information
backup: {
id: this.generateBackupId(),
type: backupType,
timestamp: new Date().toISOString(),
path: backupPath,
parent_backup_id: parentBackupId,
retention_days: customMetadata.retentionDays || 30
},
// System information
system: {
hostname: this.hostname,
platform: this.platform,
os_release: this.osRelease,
architecture: os.arch(),
cpu_count: os.cpus().length,
total_memory: os.totalmem(),
free_memory: os.freemem(),
uptime: os.uptime()
},
// Application information
application: {
name: 'PicPeak',
version: this.appVersion,
node_version: this.nodeVersion,
environment: process.env.NODE_ENV || 'production',
storage_path: process.env.STORAGE_PATH || path.join(__dirname, '../../../storage')
},
// Files information
files: {
count: files.length,
total_size: files.reduce((sum, file) => sum + (file.size || 0), 0),
checksums: await this.generateFileChecksums(files),
manifest: files.map(file => ({
path: file.relativePath || file.path,
size: file.size,
modified: file.modified,
checksum: file.checksum,
type: this.getFileType(file.path),
permissions: file.permissions
}))
},
// Database information
database: {
type: databaseInfo.type || this.getDatabaseType(),
backup_file: databaseInfo.backupFile,
size: databaseInfo.size,
checksum: databaseInfo.checksum,
tables: databaseInfo.tables || {},
row_counts: databaseInfo.rowCounts || {},
schema_version: await this.getSchemaVersion()
},
// Verification information
verification: {
total_checksum: null, // Will be calculated after manifest is complete
file_count_check: files.length,
size_check: files.reduce((sum, file) => sum + (file.size || 0), 0),
integrity_timestamp: new Date().toISOString()
},
// Custom metadata
metadata: {
...customMetadata,
backup_settings: await this.getBackupSettings(),
active_events_count: await this.getActiveEventsCount(),
archived_events_count: await this.getArchivedEventsCount(),
total_photos_count: await this.getTotalPhotosCount()
}
};
// Calculate total checksum of the manifest
manifest.verification.total_checksum = this.calculateManifestChecksum(manifest);
return manifest;
}
/**
* Save manifest to file
* @param {Object} manifest - Manifest object to save
* @param {string} filePath - Path to save the manifest
* @param {string} format - 'json' or 'yaml'
*/
async saveManifest(manifest, filePath, format = 'json') {
try {
let content;
if (format === 'yaml') {
content = yaml.dump(manifest, {
indent: 2,
lineWidth: -1,
noRefs: true,
sortKeys: true
});
} else {
content = JSON.stringify(manifest, null, 2);
}
await fs.writeFile(filePath, content, 'utf8');
logger.info(`Manifest saved to ${filePath} (format: ${format})`);
return filePath;
} catch (error) {
logger.error('Failed to save manifest:', error);
throw error;
}
}
/**
* Load and validate an existing manifest
* @param {string} filePath - Path to the manifest file
* @returns {Object} Loaded and validated manifest
*/
async loadManifest(filePath) {
try {
const content = await fs.readFile(filePath, 'utf8');
let manifest;
// Detect format and parse
if (filePath.endsWith('.yaml') || filePath.endsWith('.yml')) {
manifest = yaml.load(content);
} else {
manifest = JSON.parse(content);
}
// Validate manifest structure
this.validateManifest(manifest);
return manifest;
} catch (error) {
logger.error('Failed to load manifest:', error);
throw error;
}
}
/**
* Validate manifest structure and integrity
* @param {Object} manifest - Manifest to validate
* @throws {Error} If validation fails
*/
validateManifest(manifest) {
// Check required sections
const requiredSections = ['manifest', 'backup', 'system', 'application', 'files', 'database', 'verification'];
for (const section of requiredSections) {
if (!manifest[section]) {
throw new Error(`Missing required section: ${section}`);
}
}
// Validate manifest version
if (!manifest.manifest.version) {
throw new Error('Missing manifest version');
}
// Validate file checksums
if (manifest.files.count !== manifest.files.manifest.length) {
throw new Error('File count mismatch');
}
// Validate total checksum
const calculatedChecksum = this.calculateManifestChecksum(manifest);
if (manifest.verification.total_checksum !== calculatedChecksum) {
throw new Error('Manifest checksum verification failed');
}
logger.info('Manifest validation passed');
return true;
}
/**
* Compare two manifests for incremental backup
* @param {Object} currentManifest - Current backup manifest
* @param {Object} parentManifest - Parent backup manifest
* @returns {Object} Comparison results
*/
compareManifests(currentManifest, parentManifest) {
const comparison = {
added_files: [],
modified_files: [],
deleted_files: [],
unchanged_files: [],
size_difference: 0,
database_changes: {}
};
// Create file maps for easy comparison
const currentFiles = new Map(
currentManifest.files.manifest.map(f => [f.path, f])
);
const parentFiles = new Map(
parentManifest.files.manifest.map(f => [f.path, f])
);
// Find added and modified files
for (const [path, file] of currentFiles) {
const parentFile = parentFiles.get(path);
if (!parentFile) {
comparison.added_files.push(file);
comparison.size_difference += file.size;
} else if (file.checksum !== parentFile.checksum) {
comparison.modified_files.push(file);
comparison.size_difference += file.size - parentFile.size;
} else {
comparison.unchanged_files.push(file);
}
}
// Find deleted files
for (const [path, file] of parentFiles) {
if (!currentFiles.has(path)) {
comparison.deleted_files.push(file);
comparison.size_difference -= file.size;
}
}
// Compare database info
comparison.database_changes = {
size_difference: currentManifest.database.size - parentManifest.database.size,
checksum_changed: currentManifest.database.checksum !== parentManifest.database.checksum,
schema_version_changed: currentManifest.database.schema_version !== parentManifest.database.schema_version
};
return comparison;
}
/**
* Generate incremental manifest based on parent
* @param {Object} options - Manifest generation options
* @param {Object} parentManifest - Parent backup manifest
* @returns {Object} Incremental manifest
*/
async generateIncrementalManifest(options, parentManifest) {
const fullManifest = await this.generateManifest({
...options,
backupType: 'incremental'
});
const comparison = this.compareManifests(fullManifest, parentManifest);
// Add incremental-specific information
fullManifest.incremental = {
parent_backup_id: parentManifest.backup.id,
parent_timestamp: parentManifest.backup.timestamp,
changes: {
added_files_count: comparison.added_files.length,
modified_files_count: comparison.modified_files.length,
deleted_files_count: comparison.deleted_files.length,
unchanged_files_count: comparison.unchanged_files.length,
size_difference: comparison.size_difference
},
added_files: comparison.added_files.map(f => f.path),
modified_files: comparison.modified_files.map(f => f.path),
deleted_files: comparison.deleted_files.map(f => f.path)
};
return fullManifest;
}
// Helper methods
generateBackupId() {
const timestamp = new Date().toISOString().replace(/[:-]/g, '').replace('T', '-').split('.')[0];
const random = crypto.randomBytes(4).toString('hex');
return `backup-${timestamp}-${random}`;
}
async generateFileChecksums(files) {
const checksums = {};
for (const file of files) {
if (file.checksum) {
checksums[file.relativePath || file.path] = file.checksum;
}
}
return checksums;
}
getFileType(filePath) {
const ext = path.extname(filePath).toLowerCase();
const typeMap = {
'.jpg': 'image',
'.jpeg': 'image',
'.png': 'image',
'.gif': 'image',
'.webp': 'image',
'.zip': 'archive',
'.sql': 'database',
'.db': 'database',
'.json': 'config',
'.yaml': 'config',
'.yml': 'config'
};
return typeMap[ext] || 'other';
}
getDatabaseType() {
return process.env.DB_TYPE === 'postgresql' ? 'postgresql' : 'sqlite';
}
async getSchemaVersion() {
try {
const result = await db('migrations')
.orderBy('run_at', 'desc')
.first();
return result ? result.migration_name : 'unknown';
} catch (error) {
return 'unknown';
}
}
async getBackupSettings() {
try {
const settings = await db('app_settings')
.where('setting_type', 'backup')
.select('setting_key', 'setting_value');
const config = {};
settings.forEach(setting => {
try {
config[setting.setting_key] = JSON.parse(setting.setting_value);
} catch (e) {
config[setting.setting_key] = setting.setting_value;
}
});
return config;
} catch (error) {
return {};
}
}
async getActiveEventsCount() {
try {
const result = await db('events')
.where('status', 'active')
.count('* as count')
.first();
return result ? parseInt(result.count) : 0;
} catch (error) {
return 0;
}
}
async getArchivedEventsCount() {
try {
const result = await db('events')
.where('status', 'archived')
.count('* as count')
.first();
return result ? parseInt(result.count) : 0;
} catch (error) {
return 0;
}
}
async getTotalPhotosCount() {
try {
const result = await db('photos')
.count('* as count')
.first();
return result ? parseInt(result.count) : 0;
} catch (error) {
return 0;
}
}
calculateManifestChecksum(manifest) {
// Create a copy without the checksum field
const manifestCopy = JSON.parse(JSON.stringify(manifest));
if (manifestCopy.verification) {
delete manifestCopy.verification.total_checksum;
}
// Calculate SHA256 of the sorted JSON
const content = JSON.stringify(manifestCopy, Object.keys(manifestCopy).sort());
return crypto.createHash('sha256').update(content).digest('hex');
}
/**
* Generate a summary report from a manifest
* @param {Object} manifest - Manifest to summarize
* @returns {string} Human-readable summary
*/
generateSummaryReport(manifest) {
const report = [];
report.push('=== BACKUP MANIFEST SUMMARY ===');
report.push(`Backup ID: ${manifest.backup.id}`);
report.push(`Type: ${manifest.backup.type}`);
report.push(`Created: ${manifest.backup.timestamp}`);
if (manifest.backup.parent_backup_id) {
report.push(`Parent Backup: ${manifest.backup.parent_backup_id}`);
}
report.push('\n--- System Information ---');
report.push(`Host: ${manifest.system.hostname}`);
report.push(`Platform: ${manifest.system.platform} ${manifest.system.os_release}`);
report.push(`Architecture: ${manifest.system.architecture}`);
report.push('\n--- Application Information ---');
report.push(`App Version: ${manifest.application.version}`);
report.push(`Node Version: ${manifest.application.node_version}`);
report.push(`Environment: ${manifest.application.environment}`);
report.push('\n--- Files Summary ---');
report.push(`Total Files: ${manifest.files.count}`);
report.push(`Total Size: ${(manifest.files.total_size / 1024 / 1024).toFixed(2)} MB`);
if (manifest.incremental) {
report.push('\n--- Incremental Changes ---');
report.push(`Added Files: ${manifest.incremental.changes.added_files_count}`);
report.push(`Modified Files: ${manifest.incremental.changes.modified_files_count}`);
report.push(`Deleted Files: ${manifest.incremental.changes.deleted_files_count}`);
report.push(`Size Difference: ${(manifest.incremental.changes.size_difference / 1024 / 1024).toFixed(2)} MB`);
}
report.push('\n--- Database Information ---');
report.push(`Type: ${manifest.database.type}`);
report.push(`Size: ${manifest.database.size ? (manifest.database.size / 1024 / 1024).toFixed(2) + ' MB' : 'N/A'}`);
report.push(`Schema Version: ${manifest.database.schema_version}`);
report.push('\n--- Content Statistics ---');
report.push(`Active Events: ${manifest.metadata.active_events_count}`);
report.push(`Archived Events: ${manifest.metadata.archived_events_count}`);
report.push(`Total Photos: ${manifest.metadata.total_photos_count}`);
report.push('\n--- Verification ---');
report.push(`Manifest Checksum: ${manifest.verification.total_checksum}`);
report.push(`Integrity Timestamp: ${manifest.verification.integrity_timestamp}`);
return report.join('\n');
}
}
// Export singleton instance
module.exports = new BackupManifestGenerator();
File diff suppressed because it is too large Load Diff
@@ -1,720 +0,0 @@
const cron = require('node-cron');
const path = require('path');
const fs = require('fs').promises;
const crypto = require('crypto');
const { exec } = require('child_process');
const { promisify } = require('util');
const execAsync = promisify(exec);
const { db } = require('../database/db');
const { queueEmail } = require('./emailProcessor');
const logger = require('../utils/logger');
const { formatBoolean } = require('../utils/dbCompat');
const backupManifest = require('./backupManifest');
// Backup job reference
let backupJob = null;
let backupConfig = null;
let isRunning = false;
// Storage paths
const getStoragePath = () => process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
/**
* Calculate file checksum using SHA256
*/
async function calculateChecksum(filePath) {
const hash = crypto.createHash('sha256');
const stream = require('fs').createReadStream(filePath);
return new Promise((resolve, reject) => {
stream.on('data', data => hash.update(data));
stream.on('end', () => resolve(hash.digest('hex')));
stream.on('error', reject);
});
}
/**
* Get database backup information
*/
async function getDatabaseBackupInfo() {
try {
// Check for recent database backup
const recentDbBackup = await db('database_backup_runs')
.where('status', 'completed')
.orderBy('completed_at', 'desc')
.first();
if (recentDbBackup && recentDbBackup.file_path) {
return {
type: recentDbBackup.backup_type,
backupFile: recentDbBackup.file_path,
size: recentDbBackup.file_size_bytes,
checksum: recentDbBackup.checksum,
tables: recentDbBackup.statistics ? JSON.parse(recentDbBackup.statistics).tables : {},
rowCounts: recentDbBackup.table_checksums ? JSON.parse(recentDbBackup.table_checksums) : {}
};
}
return {
type: process.env.DB_TYPE === 'postgresql' ? 'postgresql' : 'sqlite',
backupFile: null,
size: 0,
checksum: null,
tables: {},
rowCounts: {}
};
} catch (error) {
logger.error('Failed to get database backup info:', error);
return {
type: 'unknown',
backupFile: null,
size: 0,
checksum: null,
tables: {},
rowCounts: {}
};
}
}
/**
* Get backup configuration from database
*/
async function getBackupConfig() {
try {
const settings = await db('app_settings')
.where('setting_type', 'backup')
.select('setting_key', 'setting_value');
const config = {};
settings.forEach(setting => {
try {
config[setting.setting_key] = JSON.parse(setting.setting_value);
} catch (e) {
config[setting.setting_key] = setting.setting_value;
}
});
return config;
} catch (error) {
logger.error('Failed to get backup configuration:', error);
return null;
}
}
/**
* Get list of files to backup
*/
async function getFilesToBackup(includeArchived = true) {
const files = [];
const storagePath = getStoragePath();
try {
// Active events
const activePath = path.join(storagePath, 'events/active');
await scanDirectory(activePath, files, storagePath);
// Archived events (if enabled)
if (includeArchived) {
const archivePath = path.join(storagePath, 'events/archived');
await scanDirectory(archivePath, files, storagePath);
}
// Thumbnails
const thumbsPath = path.join(storagePath, 'thumbnails');
await scanDirectory(thumbsPath, files, storagePath);
// Uploads (logos, favicons, etc.)
const uploadsPath = path.join(storagePath, 'uploads');
await scanDirectory(uploadsPath, files, storagePath);
return files;
} catch (error) {
logger.error('Failed to get files to backup:', error);
throw error;
}
}
/**
* Recursively scan directory for files
*/
async function scanDirectory(dirPath, fileList, basePath, excludePatterns = []) {
try {
const entries = await fs.readdir(dirPath, { withFileTypes: true });
for (const entry of entries) {
const fullPath = path.join(dirPath, entry.name);
const relativePath = path.relative(basePath, fullPath);
// Check exclude patterns
if (excludePatterns.some(pattern => {
if (pattern.includes('*')) {
return new RegExp(pattern.replace(/\*/g, '.*')).test(entry.name);
}
return entry.name === pattern;
})) {
continue;
}
if (entry.isDirectory()) {
await scanDirectory(fullPath, fileList, basePath, excludePatterns);
} else if (entry.isFile()) {
const stats = await fs.stat(fullPath);
fileList.push({
path: fullPath,
relativePath: relativePath,
size: stats.size,
modified: stats.mtime
});
}
}
} catch (error) {
if (error.code !== 'ENOENT') {
logger.error(`Failed to scan directory ${dirPath}:`, error);
}
}
}
/**
* Check if file has changed since last backup
*/
async function hasFileChanged(filePath, checksum) {
try {
const fileState = await db('backup_file_states')
.where('file_path', filePath)
.first();
return !fileState || fileState.checksum !== checksum;
} catch (error) {
logger.error('Failed to check file state:', error);
return true; // Assume changed if we can't check
}
}
/**
* Update file state in database
*/
async function updateFileState(filePath, checksum, size, modified) {
try {
const existing = await db('backup_file_states')
.where('file_path', filePath)
.first();
const data = {
file_path: filePath,
checksum: checksum,
size_bytes: size,
last_modified: modified,
last_backed_up: new Date()
};
if (existing) {
await db('backup_file_states')
.where('id', existing.id)
.update(data);
} else {
await db('backup_file_states').insert(data);
}
} catch (error) {
logger.error('Failed to update file state:', error);
}
}
/**
* Perform local directory backup
*/
async function performLocalBackup(config, files) {
const destPath = config.backup_destination_path;
const storagePath = getStoragePath();
let backedUpCount = 0;
let backedUpSize = 0;
const backedUpFiles = [];
// Ensure destination exists
await fs.mkdir(destPath, { recursive: true });
for (const file of files) {
try {
// Skip large files if configured
const maxSizeMB = config.backup_max_file_size_mb || 5000;
if (file.size > maxSizeMB * 1024 * 1024) {
logger.warn(`Skipping large file: ${file.relativePath} (${(file.size / 1024 / 1024).toFixed(2)} MB)`);
continue;
}
// Calculate checksum
const checksum = await calculateChecksum(file.path);
file.checksum = checksum; // Add checksum to file object
// Check if file has changed
const changed = await hasFileChanged(file.relativePath, checksum);
if (!changed) {
continue;
}
// Copy file
const destFilePath = path.join(destPath, file.relativePath);
const destDir = path.dirname(destFilePath);
await fs.mkdir(destDir, { recursive: true });
await fs.copyFile(file.path, destFilePath);
// Update state
await updateFileState(file.relativePath, checksum, file.size, file.modified);
backedUpCount++;
backedUpSize += file.size;
backedUpFiles.push(file.relativePath);
} catch (error) {
logger.error(`Failed to backup file ${file.relativePath}:`, error);
}
}
return { backedUpCount, backedUpSize, backedUpFiles };
}
/**
* Perform rsync backup
*/
async function performRsyncBackup(config, files) {
const storagePath = getStoragePath();
const host = config.backup_rsync_host;
const user = config.backup_rsync_user;
const remotePath = config.backup_rsync_path;
const sshKey = config.backup_rsync_ssh_key;
if (!host || !remotePath) {
throw new Error('Rsync configuration incomplete');
}
// Build rsync command
const rsyncOptions = [
'-avz', // archive, verbose, compress
'--delete', // remove deleted files
'--stats' // show statistics
];
if (sshKey) {
rsyncOptions.push(`-e "ssh -i ${sshKey} -o StrictHostKeyChecking=no"`);
}
// Add exclude patterns
const excludePatterns = config.backup_exclude_patterns || [];
excludePatterns.forEach(pattern => {
rsyncOptions.push(`--exclude="${pattern}"`);
});
const source = `${storagePath}/`;
const destination = user ? `${user}@${host}:${remotePath}` : `${host}:${remotePath}`;
const rsyncCommand = `rsync ${rsyncOptions.join(' ')} "${source}" "${destination}"`;
try {
const { stdout, stderr } = await execAsync(rsyncCommand);
// Parse rsync stats
const stats = parseRsyncStats(stdout);
// Update file states for successfully synced files
for (const file of files) {
try {
const checksum = await calculateChecksum(file.path);
await updateFileState(file.relativePath, checksum, file.size, file.modified);
} catch (error) {
logger.error(`Failed to update state for ${file.relativePath}:`, error);
}
}
return {
backedUpCount: stats.filesTransferred || files.length,
backedUpSize: stats.totalSize || files.reduce((sum, f) => sum + f.size, 0),
backedUpFiles: files.map(f => f.relativePath)
};
} catch (error) {
logger.error('Rsync backup failed:', error);
throw new Error(`Rsync backup failed: ${error.message}`);
}
}
/**
* Parse rsync statistics from output
*/
function parseRsyncStats(output) {
const stats = {};
// Extract files transferred
const filesMatch = output.match(/Number of files transferred: (\d+)/);
if (filesMatch) {
stats.filesTransferred = parseInt(filesMatch[1]);
}
// Extract total size
const sizeMatch = output.match(/Total file size: ([\d,]+) bytes/);
if (sizeMatch) {
stats.totalSize = parseInt(sizeMatch[1].replace(/,/g, ''));
}
return stats;
}
/**
* Perform S3-compatible backup
*/
async function performS3Backup(config, files) {
// This would require AWS SDK or similar
// For now, return a placeholder
throw new Error('S3 backup not implemented yet');
}
/**
* Run backup process
*/
async function runBackup() {
if (isRunning) {
logger.warn('Backup already running, skipping');
return;
}
isRunning = true;
const startTime = new Date();
let backupRun = null;
try {
// Get current configuration
const config = await getBackupConfig();
if (!config.backup_enabled) {
logger.info('Backup is disabled, skipping');
return;
}
// Create backup run record
const [runId] = await db('backup_runs').insert({
started_at: startTime,
status: 'running',
backup_type: 'scheduled'
});
backupRun = { id: runId };
// Get files to backup
const files = await getFilesToBackup(config.backup_include_archived);
logger.info(`Found ${files.length} files to check for backup`);
// Perform backup based on destination type
let result;
switch (config.backup_destination_type) {
case 'local':
result = await performLocalBackup(config, files);
break;
case 'rsync':
result = await performRsyncBackup(config, files);
break;
case 's3':
result = await performS3Backup(config, files);
break;
default:
throw new Error(`Unknown backup destination type: ${config.backup_destination_type}`);
}
// Calculate duration
const endTime = new Date();
const durationSeconds = Math.round((endTime - startTime) / 1000);
// Generate backup manifest
let manifestPath = null;
try {
logger.info('Generating backup manifest...');
// Get database backup info if available
const databaseInfo = await getDatabaseBackupInfo();
// Determine if this is an incremental backup
const lastSuccessfulBackup = await db('backup_runs')
.where('status', 'completed')
.whereNot('id', runId)
.orderBy('completed_at', 'desc')
.first();
let manifest;
const manifestOptions = {
backupType: lastSuccessfulBackup ? 'incremental' : 'full',
backupPath: config.backup_destination_path || config.backup_destination_type,
files: files.filter(f => result.backedUpFiles && result.backedUpFiles.includes(f.relativePath)),
databaseInfo: databaseInfo,
parentBackupId: lastSuccessfulBackup ? lastSuccessfulBackup.manifest_id : null,
format: config.backup_manifest_format || 'json',
customMetadata: {
backup_run_id: runId,
destination_type: config.backup_destination_type,
operator: 'system',
reason: 'scheduled',
retentionDays: config.backup_retention_days || 30
}
};
if (lastSuccessfulBackup && lastSuccessfulBackup.manifest_path) {
try {
const parentManifest = await backupManifest.loadManifest(lastSuccessfulBackup.manifest_path);
manifest = await backupManifest.generateIncrementalManifest(manifestOptions, parentManifest);
} catch (error) {
logger.warn('Failed to load parent manifest, generating full manifest:', error);
manifest = await backupManifest.generateManifest(manifestOptions);
}
} else {
manifest = await backupManifest.generateManifest(manifestOptions);
}
// Save manifest
const manifestDir = config.backup_manifest_path || path.join(config.backup_destination_path || '/backup', 'manifests');
await fs.mkdir(manifestDir, { recursive: true });
const manifestFileName = `backup-manifest-${manifest.backup.id}.${config.backup_manifest_format || 'json'}`;
manifestPath = path.join(manifestDir, manifestFileName);
await backupManifest.saveManifest(manifest, manifestPath, config.backup_manifest_format || 'json');
logger.info(`Backup manifest saved to ${manifestPath}`);
} catch (error) {
logger.error('Failed to generate backup manifest:', error);
// Don't fail the entire backup for manifest generation failure
}
// Update backup run record
await db('backup_runs')
.where('id', runId)
.update({
completed_at: endTime,
status: 'completed',
files_backed_up: result.backedUpCount,
total_size_bytes: result.backedUpSize,
duration_seconds: durationSeconds,
manifest_path: manifestPath,
manifest_id: manifestPath ? path.basename(manifestPath, path.extname(manifestPath)) : null,
statistics: JSON.stringify({
totalFilesChecked: files.length,
filesBackedUp: result.backedUpCount,
totalSize: result.backedUpSize,
averageFileSize: result.backedUpCount > 0 ? Math.round(result.backedUpSize / result.backedUpCount) : 0,
manifestGenerated: !!manifestPath
})
});
logger.info(`Backup completed: ${result.backedUpCount} files, ${(result.backedUpSize / 1024 / 1024).toFixed(2)} MB in ${durationSeconds}s`);
// Send success email if configured
if (config.backup_email_on_success) {
// Get admin emails
const admins = await db('admin_users').where('is_active', formatBoolean(true));
for (const admin of admins) {
await queueEmail(null, admin.email, 'backup_completed', {
start_time: startTime.toISOString(),
duration: `${durationSeconds} seconds`,
files_count: result.backedUpCount.toString(),
total_size: `${(result.backedUpSize / 1024 / 1024).toFixed(2)} MB`,
backup_type: config.backup_destination_type
});
}
}
} catch (error) {
logger.error('Backup failed:', error);
// Update backup run record
if (backupRun) {
await db('backup_runs')
.where('id', backupRun.id)
.update({
completed_at: new Date(),
status: 'failed',
error_message: error.message
});
}
// Send failure email
const config = await getBackupConfig();
if (config && config.backup_email_on_failure) {
const admins = await db('admin_users').where('is_active', formatBoolean(true));
for (const admin of admins) {
await queueEmail(null, admin.email, 'backup_failed', {
start_time: startTime.toISOString(),
backup_type: config.backup_destination_type || 'unknown',
error_message: error.message
});
}
}
} finally {
isRunning = false;
}
}
/**
* Start backup service
*/
async function startBackupService() {
try {
// Get configuration
backupConfig = await getBackupConfig();
if (!backupConfig || !backupConfig.backup_enabled) {
logger.info('Backup service is disabled');
return;
}
// Cancel existing job if any
if (backupJob) {
backupJob.stop();
}
// Schedule backup job
const schedule = backupConfig.backup_schedule || '0 2 * * *'; // Default: 2 AM daily
backupJob = cron.schedule(schedule, async () => {
logger.info('Starting scheduled backup');
await runBackup();
});
logger.info(`Backup service started with schedule: ${schedule}`);
} catch (error) {
logger.error('Failed to start backup service:', error);
}
}
/**
* Stop backup service
*/
function stopBackupService() {
if (backupJob) {
backupJob.stop();
backupJob = null;
logger.info('Backup service stopped');
}
}
/**
* Trigger manual backup
*/
async function triggerManualBackup() {
logger.info('Starting manual backup');
await runBackup();
}
/**
* Get backup status and history
*/
async function getBackupStatus(limit = 10) {
try {
const runs = await db('backup_runs')
.orderBy('started_at', 'desc')
.limit(limit);
const lastRun = runs[0];
const isHealthy = lastRun && lastRun.status === 'completed';
// Validate manifest if exists
let manifestValid = false;
if (lastRun && lastRun.manifest_path) {
try {
const manifest = await backupManifest.loadManifest(lastRun.manifest_path);
backupManifest.validateManifest(manifest);
manifestValid = true;
} catch (error) {
logger.warn('Manifest validation failed:', error);
}
}
return {
isRunning,
isHealthy,
lastRun: lastRun ? {
...lastRun,
manifestValid
} : null,
recentRuns: runs,
nextScheduledRun: backupJob ? getNextScheduledRun() : null
};
} catch (error) {
logger.error('Failed to get backup status:', error);
return {
isRunning,
isHealthy: false,
error: error.message
};
}
}
/**
* Get next scheduled run time
*/
function getNextScheduledRun() {
// This is a simplified version - would need proper cron parsing
const now = new Date();
const tomorrow = new Date(now);
tomorrow.setDate(tomorrow.getDate() + 1);
tomorrow.setHours(2, 0, 0, 0); // Assuming default 2 AM schedule
return tomorrow.toISOString();
}
/**
* Clean up old backup runs
*/
async function cleanupOldBackupRuns(retentionDays = 30) {
try {
const cutoffDate = new Date();
cutoffDate.setDate(cutoffDate.getDate() - retentionDays);
const deleted = await db('backup_runs')
.where('started_at', '<', cutoffDate)
.delete();
if (deleted > 0) {
logger.info(`Cleaned up ${deleted} old backup runs`);
}
} catch (error) {
logger.error('Failed to cleanup old backup runs:', error);
}
}
/**
* Get backup manifest for a specific backup run
*/
async function getBackupManifest(backupRunId) {
try {
const run = await db('backup_runs')
.where('id', backupRunId)
.first();
if (!run || !run.manifest_path) {
throw new Error('Backup manifest not found');
}
const manifest = await backupManifest.loadManifest(run.manifest_path);
return {
manifest,
summary: backupManifest.generateSummaryReport(manifest)
};
} catch (error) {
logger.error('Failed to get backup manifest:', error);
throw error;
}
}
/**
* Validate a backup manifest file
*/
async function validateBackupManifest(manifestPath) {
try {
const manifest = await backupManifest.loadManifest(manifestPath);
backupManifest.validateManifest(manifest);
return { valid: true, manifest };
} catch (error) {
return { valid: false, error: error.message };
}
}
module.exports = {
startBackupService,
stopBackupService,
triggerManualBackup,
getBackupStatus,
runBackup,
cleanupOldBackupRuns,
getBackupManifest,
validateBackupManifest
};
@@ -1,147 +0,0 @@
/**
* Database Backup Service Usage Examples
*
* This service provides comprehensive database backup functionality
* with support for both SQLite and PostgreSQL databases.
*/
const { databaseBackupService } = require('./databaseBackup');
// Example 1: Manual backup with default settings
async function manualBackup() {
try {
const result = await databaseBackupService.backup();
console.log('Backup completed:', result);
// Result includes: path, size, duration, checksum, compressionRatio
} catch (error) {
console.error('Backup failed:', error);
}
}
// Example 2: Backup with custom options
async function customBackup() {
try {
const result = await databaseBackupService.backup({
destinationPath: '/custom/backup/path',
compress: true, // Enable gzip compression
validateIntegrity: true, // Validate backup after creation
includeChecksums: true, // Calculate table checksums
noTransaction: false // Use transaction for consistency (PostgreSQL)
});
console.log('Custom backup completed:', result);
} catch (error) {
console.error('Backup failed:', error);
}
}
// Example 3: Check backup progress (useful for long-running backups)
async function backupWithProgress() {
// Start backup asynchronously
const backupPromise = databaseBackupService.backup();
// Poll for progress
const progressInterval = setInterval(() => {
const progress = databaseBackupService.getProgress();
if (progress) {
console.log(`Progress: ${progress.message}`, progress.details);
}
}, 1000);
try {
const result = await backupPromise;
clearInterval(progressInterval);
console.log('Backup completed:', result);
} catch (error) {
clearInterval(progressInterval);
console.error('Backup failed:', error);
}
}
// Example 4: Get backup history
async function getBackupHistory() {
const history = await databaseBackupService.getBackupHistory(10);
history.forEach(backup => {
console.log(`Backup ${backup.id}:`);
console.log(` Started: ${backup.started_at}`);
console.log(` Status: ${backup.status}`);
console.log(` Size: ${(backup.file_size_bytes / 1024 / 1024).toFixed(2)} MB`);
console.log(` Duration: ${backup.duration_seconds}s`);
});
}
// Example 5: Clean up old backups
async function cleanupBackups() {
// Delete backups older than 30 days
await databaseBackupService.cleanupOldBackups(30);
console.log('Old backups cleaned up');
}
// Example 6: Get table checksums (useful for monitoring changes)
async function getTableChecksums() {
const checksums = await databaseBackupService.getTableChecksums();
console.log('Table Checksums:');
Object.entries(checksums).forEach(([table, info]) => {
console.log(` ${table}: ${info.rowCount} rows, checksum: ${info.checksum}`);
});
}
// Example 7: Using the scheduled backup service
const { startScheduledBackups, stopScheduledBackups } = require('./databaseBackup');
async function setupScheduledBackups() {
// Start scheduled backups (reads schedule from database config)
await startScheduledBackups();
console.log('Scheduled backups started');
// Later, if needed, stop scheduled backups
// stopScheduledBackups();
}
// Example 8: Admin API endpoints available
/*
GET /api/admin/database-backup/status - Get backup status and config
PUT /api/admin/database-backup/config - Update backup configuration
POST /api/admin/database-backup/backup - Trigger manual backup
GET /api/admin/database-backup/progress - Get current backup progress
GET /api/admin/database-backup/history - Get backup history with pagination
DELETE /api/admin/database-backup/cleanup - Delete old backup files
POST /api/admin/database-backup/test - Test backup configuration
GET /api/admin/database-backup/checksums - Get current table checksums
*/
// Example 9: Configuration options stored in database
/*
database_backup_enabled: boolean - Enable/disable scheduled backups
database_backup_schedule: string - Cron schedule (default: '0 3 * * *')
database_backup_destination_path: string - Where to store backups
database_backup_compress: boolean - Enable gzip compression
database_backup_validate_integrity: boolean - Validate after backup
database_backup_include_checksums: boolean - Calculate table checksums
database_backup_retention_days: number - Days to keep old backups
database_backup_email_on_failure: boolean - Send email on failure
database_backup_email_on_success: boolean - Send email on success
*/
// Example 10: Production considerations
/*
1. Ensure destination path has sufficient space
2. For large databases, backups may take significant time
3. PostgreSQL backups use single-transaction mode by default
4. Compression typically reduces size by 70-90%
5. Schedule backups during low-traffic periods
6. Monitor backup history for failures
7. Test restore procedures regularly
8. Consider replication for real-time redundancy
*/
module.exports = {
manualBackup,
customBackup,
backupWithProgress,
getBackupHistory,
cleanupBackups,
getTableChecksums,
setupScheduledBackups
};
-718
View File
@@ -1,718 +0,0 @@
const fs = require('fs').promises;
const path = require('path');
const { exec } = require('child_process');
const { promisify } = require('util');
const execAsync = promisify(exec);
const crypto = require('crypto');
const zlib = require('zlib');
const { pipeline } = require('stream/promises');
const { createReadStream, createWriteStream } = require('fs');
const { db } = require('../database/db');
const knexConfig = require('../../knexfile');
const logger = require('../utils/logger');
const { queueEmail } = require('./emailProcessor');
const { formatBoolean } = require('../utils/dbCompat');
const packageJson = require('../../package.json');
// Constants
const CHUNK_SIZE = 1024 * 1024; // 1MB chunks for streaming
const PROGRESS_INTERVAL = 100; // Report progress every 100 rows
/**
* Database Backup Service
* Supports both SQLite and PostgreSQL with proper escaping,
* compression, checksums, and validation
*/
class DatabaseBackupService {
constructor() {
this.isRunning = false;
this.currentProgress = null;
this.dbType = knexConfig.client === 'pg' ? 'postgresql' : 'sqlite';
}
/**
* Calculate checksum for a file
*/
async calculateChecksum(filePath) {
const hash = crypto.createHash('sha256');
const stream = createReadStream(filePath);
return new Promise((resolve, reject) => {
stream.on('data', data => hash.update(data));
stream.on('end', () => resolve(hash.digest('hex')));
stream.on('error', reject);
});
}
/**
* Compress a file using gzip
*/
async compressFile(inputPath, outputPath) {
const gzip = zlib.createGzip({ level: 6 }); // Balanced compression
const source = createReadStream(inputPath);
const destination = createWriteStream(outputPath);
await pipeline(source, gzip, destination);
// Get compression ratio
const inputStats = await fs.stat(inputPath);
const outputStats = await fs.stat(outputPath);
const ratio = (1 - outputStats.size / inputStats.size) * 100;
return {
originalSize: inputStats.size,
compressedSize: outputStats.size,
compressionRatio: ratio.toFixed(2)
};
}
/**
* Get database size
*/
async getDatabaseSize() {
if (this.dbType === 'sqlite') {
const dbPath = knexConfig.connection.filename;
const stats = await fs.stat(dbPath);
return stats.size;
} else {
// PostgreSQL
const result = await db.raw(`
SELECT pg_database_size(current_database()) as size
`);
return parseInt(result.rows[0].size);
}
}
/**
* Get table checksums for change detection
*/
async getTableChecksums() {
const checksums = {};
const tables = await this.getTables();
for (const table of tables) {
if (this.dbType === 'sqlite') {
// SQLite: Use aggregate of all row data
const result = await db.raw(`
SELECT
COUNT(*) as row_count,
COALESCE(SUM(LENGTH(CAST(t.* AS TEXT))), 0) as data_sum
FROM "${table}" t
`);
checksums[table] = {
rowCount: result[0].row_count,
checksum: crypto
.createHash('md5')
.update(`${result[0].row_count}-${result[0].data_sum}`)
.digest('hex')
};
} else {
// PostgreSQL: Use built-in functions
const result = await db.raw(`
SELECT
COUNT(*) as row_count,
MD5(COALESCE(STRING_AGG(MD5(t::text), ''), '')) as checksum
FROM "${table}" t
`);
checksums[table] = {
rowCount: parseInt(result.rows[0].row_count),
checksum: result.rows[0].checksum || 'empty'
};
}
}
return checksums;
}
/**
* Get list of tables
*/
async getTables() {
if (this.dbType === 'sqlite') {
const result = await db.raw(`
SELECT name FROM sqlite_master
WHERE type='table'
AND name NOT LIKE 'sqlite_%'
AND name != 'knex_migrations'
AND name != 'knex_migrations_lock'
ORDER BY name
`);
return result.map(row => row.name);
} else {
// PostgreSQL
const result = await db.raw(`
SELECT table_name
FROM information_schema.tables
WHERE table_schema = 'public'
AND table_type = 'BASE TABLE'
AND table_name NOT IN ('knex_migrations', 'knex_migrations_lock')
ORDER BY table_name
`);
return result.rows.map(row => row.table_name);
}
}
/**
* Create SQLite backup
*/
async createSQLiteBackup(outputPath, options = {}) {
const dbPath = knexConfig.connection.filename;
const tempPath = `${outputPath}.tmp`;
try {
// Use SQLite's backup API for consistency
await execAsync(`sqlite3 "${dbPath}" ".backup '${tempPath}'"`);
// Verify the backup
const verifyResult = await execAsync(`sqlite3 "${tempPath}" "PRAGMA integrity_check"`);
if (!verifyResult.stdout.includes('ok')) {
throw new Error('Backup integrity check failed');
}
// Move temp file to final location
await fs.rename(tempPath, outputPath);
return { success: true };
} catch (error) {
// Cleanup temp file if exists
try {
await fs.unlink(tempPath);
} catch (e) {
// Ignore
}
throw error;
}
}
/**
* Create PostgreSQL backup with proper escaping
*/
async createPostgreSQLBackup(outputPath, options = {}) {
const { host, port, user, password, database } = knexConfig.connection;
// Build connection string with proper escaping
const connectionParts = [
`host=${host}`,
`port=${port}`,
`dbname=${database}`,
`user=${user}`
];
// Set PGPASSWORD environment variable for security
const env = { ...process.env };
if (password) {
env.PGPASSWORD = password;
}
// Build pg_dump command with options
const pgDumpOptions = [
'--verbose',
'--no-owner',
'--no-privileges',
'--clean',
'--if-exists',
'--format=plain',
'--encoding=UTF8'
];
// Add transaction support for consistency
if (!options.noTransaction) {
pgDumpOptions.push('--single-transaction');
}
// Add compression if not doing it separately
if (options.compress && !options.separateCompression) {
pgDumpOptions.push('--compress=6');
}
const command = `pg_dump "${connectionParts.join(' ')}" ${pgDumpOptions.join(' ')} > "${outputPath}"`;
try {
const { stderr } = await execAsync(command, {
env,
maxBuffer: 1024 * 1024 * 100 // 100MB buffer
});
// pg_dump writes progress to stderr, not an error
if (stderr && !stderr.includes('dump complete')) {
logger.warn('pg_dump warnings:', stderr);
}
// Verify the dump file is not empty
const stats = await fs.stat(outputPath);
if (stats.size === 0) {
throw new Error('Backup file is empty');
}
return { success: true, warnings: stderr };
} catch (error) {
throw new Error(`PostgreSQL backup failed: ${error.message}`);
}
}
/**
* Validate backup integrity
*/
async validateBackup(backupPath, originalChecksums) {
const tempDbPath = `${backupPath}.validate`;
try {
if (this.dbType === 'sqlite') {
// For SQLite, we can directly check integrity
const result = await execAsync(`sqlite3 "${backupPath}" "PRAGMA integrity_check"`);
if (!result.stdout.includes('ok')) {
throw new Error('Backup integrity check failed');
}
} else {
// For PostgreSQL, we'd need to restore to a temp database
// This is more complex and might not be feasible in production
logger.info('PostgreSQL backup validation would require restore test');
}
return { valid: true };
} finally {
// Cleanup
try {
await fs.unlink(tempDbPath);
} catch (e) {
// Ignore
}
}
}
/**
* Main backup method
*/
async backup(options = {}) {
if (this.isRunning) {
throw new Error('Backup already in progress');
}
this.isRunning = true;
const startTime = new Date();
let backupRun = null;
try {
// Get configuration
const config = await this.getBackupConfig();
const {
destinationPath = '/backup/database',
compress = true,
validateIntegrity = true,
includeChecksums = true
} = { ...config, ...options };
// Create backup directory
await fs.mkdir(destinationPath, { recursive: true });
// Generate backup filename
const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
const baseName = `picpeak-db-${this.dbType}-${timestamp}`;
const sqlFile = path.join(destinationPath, `${baseName}.sql`);
const finalFile = compress ? path.join(destinationPath, `${baseName}.sql.gz`) : sqlFile;
// Get current schema version
const schemaVersion = await this.getCurrentSchemaVersion();
// Create backup run record with version info
const [runId] = await db('database_backup_runs').insert({
started_at: startTime,
status: 'running',
backup_type: this.dbType,
destination_path: finalFile,
app_version: packageJson.version,
node_version: process.version,
db_schema_version: schemaVersion,
environment_info: JSON.stringify({
platform: process.platform,
arch: process.arch,
node_env: process.env.NODE_ENV || 'production',
db_type: this.dbType
})
});
backupRun = { id: runId };
// Get initial checksums
let tableChecksums = null;
if (includeChecksums) {
this.updateProgress('Calculating table checksums...');
tableChecksums = await this.getTableChecksums();
}
// Get database size
const dbSize = await this.getDatabaseSize();
// Create the backup
this.updateProgress('Creating database backup...');
if (this.dbType === 'sqlite') {
await this.createSQLiteBackup(sqlFile, options);
} else {
await this.createPostgreSQLBackup(sqlFile, options);
}
// Compress if requested
let compressionStats = null;
if (compress) {
this.updateProgress('Compressing backup...');
compressionStats = await this.compressFile(sqlFile, finalFile);
await fs.unlink(sqlFile); // Remove uncompressed file
}
// Calculate checksum
this.updateProgress('Calculating backup checksum...');
const backupChecksum = await this.calculateChecksum(finalFile);
// Validate if requested
if (validateIntegrity && !compress) {
this.updateProgress('Validating backup integrity...');
await this.validateBackup(finalFile, tableChecksums);
}
// Get final file size
const finalStats = await fs.stat(finalFile);
// Calculate duration
const endTime = new Date();
const durationSeconds = Math.round((endTime - startTime) / 1000);
// Update backup run record
await db('database_backup_runs')
.where('id', runId)
.update({
completed_at: endTime,
status: 'completed',
file_path: finalFile,
file_size_bytes: finalStats.size,
original_size_bytes: dbSize,
duration_seconds: durationSeconds,
checksum: backupChecksum,
compression_ratio: compressionStats?.compressionRatio || null,
table_checksums: tableChecksums ? JSON.stringify(tableChecksums) : null,
statistics: JSON.stringify({
dbType: this.dbType,
compressed: compress,
validated: validateIntegrity,
compressionStats,
tableCount: tableChecksums ? Object.keys(tableChecksums).length : null,
app_version: packageJson.version,
node_version: process.version,
db_schema_version: await this.getCurrentSchemaVersion()
})
});
logger.info(`Database backup completed: ${finalFile} (${(finalStats.size / 1024 / 1024).toFixed(2)} MB) in ${durationSeconds}s`);
// Send success notification if configured
if (config.emailOnSuccess) {
await this.sendBackupNotification('success', {
duration: durationSeconds,
size: finalStats.size,
compressionRatio: compressionStats?.compressionRatio,
path: finalFile
});
}
return {
success: true,
path: finalFile,
size: finalStats.size,
duration: durationSeconds,
checksum: backupChecksum,
compressionRatio: compressionStats?.compressionRatio
};
} catch (error) {
logger.error('Database backup failed:', error);
// Update backup run record
if (backupRun) {
await db('database_backup_runs')
.where('id', backupRun.id)
.update({
completed_at: new Date(),
status: 'failed',
error_message: error.message
});
}
// Send failure notification
const config = await this.getBackupConfig();
if (config.emailOnFailure) {
await this.sendBackupNotification('failure', {
error: error.message
});
}
throw error;
} finally {
this.isRunning = false;
this.currentProgress = null;
}
}
/**
* Update progress
*/
updateProgress(message, details = {}) {
this.currentProgress = {
message,
details,
timestamp: new Date()
};
logger.info(`Backup progress: ${message}`, details);
}
/**
* Get current progress
*/
getProgress() {
return this.currentProgress;
}
/**
* Get backup configuration
*/
async getBackupConfig() {
const settings = await db('app_settings')
.where('setting_type', 'database_backup')
.select('setting_key', 'setting_value');
const config = {};
settings.forEach(setting => {
try {
config[setting.setting_key] = JSON.parse(setting.setting_value);
} catch (e) {
config[setting.setting_key] = setting.setting_value;
}
});
return config;
}
/**
* Send backup notification email
*/
async sendBackupNotification(type, details) {
const admins = await db('admin_users').where('is_active', formatBoolean(true));
for (const admin of admins) {
if (type === 'success') {
await queueEmail(null, admin.email, 'database_backup_completed', {
backup_type: this.dbType,
duration: `${details.duration} seconds`,
file_size: `${(details.size / 1024 / 1024).toFixed(2)} MB`,
compression_ratio: details.compressionRatio ? `${details.compressionRatio}%` : 'N/A',
file_path: details.path
});
} else {
await queueEmail(null, admin.email, 'database_backup_failed', {
backup_type: this.dbType,
error_message: details.error,
timestamp: new Date().toISOString()
});
}
}
}
/**
* Clean up old backups
*/
async cleanupOldBackups(retentionDays = 30) {
try {
const cutoffDate = new Date();
cutoffDate.setDate(cutoffDate.getDate() - retentionDays);
// Get old backup records
const oldBackups = await db('database_backup_runs')
.where('completed_at', '<', cutoffDate)
.where('status', 'completed')
.select('id', 'file_path');
let deletedCount = 0;
for (const backup of oldBackups) {
try {
// Delete the file
if (backup.file_path) {
await fs.unlink(backup.file_path);
}
// Delete the record
await db('database_backup_runs')
.where('id', backup.id)
.delete();
deletedCount++;
} catch (error) {
logger.error(`Failed to delete old backup ${backup.file_path}:`, error);
}
}
if (deletedCount > 0) {
logger.info(`Cleaned up ${deletedCount} old database backups`);
}
// Also clean up old failed runs
await db('database_backup_runs')
.where('started_at', '<', cutoffDate)
.where('status', 'failed')
.delete();
} catch (error) {
logger.error('Failed to cleanup old database backups:', error);
}
}
/**
* Get backup history
*/
async getBackupHistory(limit = 10) {
return await db('database_backup_runs')
.orderBy('started_at', 'desc')
.limit(limit);
}
/**
* Get current database schema version
*/
async getCurrentSchemaVersion() {
try {
const result = await db('knex_migrations')
.orderBy('id', 'desc')
.first();
return result ? result.name : 'unknown';
} catch (error) {
logger.error('Failed to get schema version:', error);
return 'unknown';
}
}
/**
* Check version compatibility for restore
*/
async checkVersionCompatibility(backupInfo) {
const currentAppVersion = packageJson.version;
const currentNodeVersion = process.version;
const currentSchemaVersion = await this.getCurrentSchemaVersion();
const compatibility = {
compatible: true,
warnings: [],
errors: []
};
// Check app version
if (backupInfo.app_version !== currentAppVersion) {
const backupMajor = backupInfo.app_version?.split('.')[0];
const currentMajor = currentAppVersion.split('.')[0];
if (backupMajor !== currentMajor) {
compatibility.errors.push(
`Major version mismatch: backup v${backupInfo.app_version}, current v${currentAppVersion}`
);
compatibility.compatible = false;
} else {
compatibility.warnings.push(
`Minor version difference: backup v${backupInfo.app_version}, current v${currentAppVersion}`
);
}
}
// Check Node.js version
if (backupInfo.node_version !== currentNodeVersion) {
const backupNodeMajor = backupInfo.node_version?.split('.')[0];
const currentNodeMajor = currentNodeVersion.split('.')[0];
if (backupNodeMajor !== currentNodeMajor) {
compatibility.warnings.push(
`Node.js major version difference: backup ${backupInfo.node_version}, current ${currentNodeVersion}`
);
}
}
// Check schema version
if (backupInfo.db_schema_version && backupInfo.db_schema_version !== currentSchemaVersion) {
compatibility.warnings.push(
`Database schema difference: backup migration '${backupInfo.db_schema_version}', current '${currentSchemaVersion}'`
);
compatibility.warnings.push(
'You may need to run migrations after restore'
);
}
return compatibility;
}
/**
* Restore from backup (with version checking)
*/
async restore(backupPath, options = {}) {
// This is a dangerous operation and should be used with extreme caution
throw new Error('Restore functionality not implemented for safety. Please use restore service or restore manually.');
}
}
// Create singleton instance
const databaseBackupService = new DatabaseBackupService();
// Scheduled backup runner
let backupSchedule = null;
/**
* Start scheduled database backups
*/
async function startScheduledBackups() {
const cron = require('node-cron');
try {
const config = await databaseBackupService.getBackupConfig();
if (!config.enabled) {
logger.info('Database backup service is disabled');
return;
}
// Stop existing schedule
if (backupSchedule) {
backupSchedule.stop();
}
// Default schedule: 3 AM daily (offset from file backups at 2 AM)
const schedule = config.schedule || '0 3 * * *';
backupSchedule = cron.schedule(schedule, async () => {
logger.info('Starting scheduled database backup');
try {
await databaseBackupService.backup();
await databaseBackupService.cleanupOldBackups(config.retentionDays || 30);
} catch (error) {
logger.error('Scheduled database backup failed:', error);
}
});
logger.info(`Database backup service started with schedule: ${schedule}`);
} catch (error) {
logger.error('Failed to start database backup service:', error);
}
}
/**
* Stop scheduled database backups
*/
function stopScheduledBackups() {
if (backupSchedule) {
backupSchedule.stop();
backupSchedule = null;
logger.info('Database backup service stopped');
}
}
module.exports = {
databaseBackupService,
startScheduledBackups,
stopScheduledBackups,
DatabaseBackupService // Export class for testing
};
+1 -7
View File
@@ -109,9 +109,8 @@ async function getRecipientLanguage(email, eventId = null) {
// Process email template with variables
async function processTemplate(template, variables, language = 'en') {
// Import date formatter and text formatters
// Import date formatter
const { formatDate } = require('../utils/dateFormatter');
const { formatWelcomeMessage } = require('../utils/formatters');
// Get the appropriate language fields
const subjectField = language === 'de' ? 'subject_de' : 'subject_en';
@@ -143,11 +142,6 @@ async function processTemplate(template, variables, language = 'en') {
if (processedVariables.archive_date) {
processedVariables.archive_date = await formatDate(processedVariables.archive_date, language);
}
// Format welcome message for HTML display (preserve line breaks)
if (processedVariables.welcome_message) {
processedVariables.welcome_message = formatWelcomeMessage(processedVariables.welcome_message);
}
// Get branding settings for logo
let logoUrl = '';
+1 -2
View File
@@ -60,7 +60,6 @@ async function processEmailQueue() {
}
// Start email queue processor
// DISABLED: Using emailProcessor.js instead to prevent duplicate connections
// setInterval(processEmailQueue, 60000); // Process every minute
setInterval(processEmailQueue, 60000); // Process every minute
module.exports = { sendEmail, processEmailQueue };
-312
View File
@@ -1,312 +0,0 @@
const { db } = require('../database/db');
const logger = require('../utils/logger');
class FeedbackModerationService {
constructor() {
this.wordFiltersCache = null;
this.cacheExpiry = null;
this.CACHE_DURATION = 5 * 60 * 1000; // 5 minutes
}
/**
* Get word filters (with caching)
*/
async getWordFilters() {
try {
// Check cache
if (this.wordFiltersCache && this.cacheExpiry && Date.now() < this.cacheExpiry) {
return this.wordFiltersCache;
}
// Fetch from database
const filters = await db('feedback_word_filters')
.where('is_active', true)
.select('word', 'severity');
// Update cache
this.wordFiltersCache = filters;
this.cacheExpiry = Date.now() + this.CACHE_DURATION;
return filters;
} catch (error) {
logger.error('Error getting word filters:', error);
return [];
}
}
/**
* Clear word filters cache
*/
clearCache() {
this.wordFiltersCache = null;
this.cacheExpiry = null;
}
/**
* Check if text contains inappropriate content
*/
async moderateText(text) {
try {
if (!text || typeof text !== 'string') {
return { approved: true };
}
const filters = await this.getWordFilters();
const violations = [];
const lowerText = text.toLowerCase();
for (const filter of filters) {
// Create regex for whole word matching
const regex = new RegExp(`\\b${this.escapeRegex(filter.word.toLowerCase())}\\b`, 'gi');
if (regex.test(lowerText)) {
violations.push({
word: filter.word,
severity: filter.severity
});
}
}
// Check for severe violations
if (violations.some(v => v.severity === 'severe')) {
return {
approved: false,
reason: 'Content contains prohibited words',
violations: violations.filter(v => v.severity === 'severe')
};
}
// Check for moderate violations
if (violations.some(v => v.severity === 'moderate')) {
return {
approved: false,
reason: 'Content requires moderation',
violations
};
}
// Check for mild violations (may just flag for review)
if (violations.length > 0) {
return {
approved: true,
flagged: true,
reason: 'Content contains potentially inappropriate words',
violations
};
}
// Additional checks
const additionalChecks = this.performAdditionalChecks(text);
if (!additionalChecks.passed) {
return {
approved: false,
reason: additionalChecks.reason
};
}
return { approved: true };
} catch (error) {
logger.error('Error moderating text:', error);
// In case of error, err on the side of caution
return {
approved: false,
reason: 'Moderation system error'
};
}
}
/**
* Perform additional content checks
*/
performAdditionalChecks(text) {
// Check for excessive caps
const capsRatio = (text.match(/[A-Z]/g) || []).length / text.length;
if (text.length > 10 && capsRatio > 0.7) {
return {
passed: false,
reason: 'Excessive use of capital letters'
};
}
// Check for spam patterns
if (this.detectSpamPatterns(text)) {
return {
passed: false,
reason: 'Content appears to be spam'
};
}
// Check for excessive special characters
const specialCharRatio = (text.match(/[!@#$%^&*()]/g) || []).length / text.length;
if (text.length > 10 && specialCharRatio > 0.3) {
return {
passed: false,
reason: 'Excessive use of special characters'
};
}
return { passed: true };
}
/**
* Detect common spam patterns
*/
detectSpamPatterns(text) {
const spamPatterns = [
/\b(buy|cheap|discount|offer|sale|deal)\s+(now|today|here)/gi,
/\b(click|visit|check)\s+(here|link|this)/gi,
/\b(viagra|cialis|pills|drugs)\b/gi,
/\b(casino|betting|poker|slots)\b/gi,
/\b(make|earn)\s+\$?\d+/gi,
/https?:\/\/[^\s]+/gi, // URLs (might want to allow in some cases)
/\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b/gi, // Email addresses
/\b\d{3,}\s?\d{3,}\s?\d{4,}\b/g // Phone numbers
];
return spamPatterns.some(pattern => pattern.test(text));
}
/**
* Escape special regex characters
*/
escapeRegex(string) {
return string.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
}
/**
* Add word filter
*/
async addWordFilter(word, severity = 'moderate') {
try {
await db('feedback_word_filters').insert({
word: word.toLowerCase(),
severity,
is_active: true,
created_at: new Date()
});
this.clearCache();
logger.info(`Added word filter: ${word} (${severity})`);
return true;
} catch (error) {
if (error.code === 'SQLITE_CONSTRAINT' || error.code === '23505') {
throw new Error('Word filter already exists');
}
logger.error('Error adding word filter:', error);
throw error;
}
}
/**
* Update word filter
*/
async updateWordFilter(id, updates) {
try {
await db('feedback_word_filters')
.where('id', id)
.update(updates);
this.clearCache();
return true;
} catch (error) {
logger.error('Error updating word filter:', error);
throw error;
}
}
/**
* Delete word filter
*/
async deleteWordFilter(id) {
try {
await db('feedback_word_filters')
.where('id', id)
.delete();
this.clearCache();
return true;
} catch (error) {
logger.error('Error deleting word filter:', error);
throw error;
}
}
/**
* Get all word filters (for admin)
*/
async getAllWordFilters() {
try {
return await db('feedback_word_filters')
.orderBy('severity', 'desc')
.orderBy('word', 'asc');
} catch (error) {
logger.error('Error getting all word filters:', error);
throw error;
}
}
/**
* Sanitize text for display (remove but don't reject)
*/
sanitizeText(text) {
// Remove excessive whitespace
text = text.replace(/\s+/g, ' ').trim();
// Remove zero-width characters
text = text.replace(/[\u200B-\u200D\uFEFF]/g, '');
// Limit consecutive special characters
text = text.replace(/([!?.]){3,}/g, '$1$1');
return text;
}
/**
* Check if user should be rate limited based on previous violations
*/
async checkUserReputation(guestIdentifier, eventId) {
try {
// Count recent violations
const recentViolations = await db('photo_feedback')
.where('guest_identifier', guestIdentifier)
.where('event_id', eventId)
.where('is_hidden', true)
.where('created_at', '>', new Date(Date.now() - 24 * 60 * 60 * 1000)) // Last 24 hours
.count('id as count')
.first();
// If user has multiple violations, they might be problematic
if (recentViolations && recentViolations.count > 3) {
return {
trusted: false,
reason: 'Multiple recent violations'
};
}
// Check total approved comments
const approvedComments = await db('photo_feedback')
.where('guest_identifier', guestIdentifier)
.where('event_id', eventId)
.where('feedback_type', 'comment')
.where('is_approved', true)
.where('is_hidden', false)
.count('id as count')
.first();
// User with many approved comments is trusted
if (approvedComments && approvedComments.count > 10) {
return {
trusted: true,
autoApprove: true
};
}
return { trusted: true };
} catch (error) {
logger.error('Error checking user reputation:', error);
return { trusted: true }; // Default to trusting in case of error
}
}
}
module.exports = new FeedbackModerationService();
-393
View File
@@ -1,393 +0,0 @@
const { db, logActivity } = require('../database/db');
const logger = require('../utils/logger');
const { formatBoolean } = require('../utils/dbCompat');
class FeedbackService {
/**
* Get feedback settings for an event
*/
async getEventFeedbackSettings(eventId) {
try {
const settings = await db('event_feedback_settings')
.where('event_id', eventId)
.first();
if (!settings) {
// Return default settings if none exist
return {
event_id: eventId,
feedback_enabled: false,
allow_ratings: true,
allow_likes: true,
allow_comments: false,
allow_favorites: true,
require_name_email: false,
moderate_comments: true,
show_feedback_to_guests: true
};
}
return settings;
} catch (error) {
logger.error('Error getting feedback settings:', error);
throw error;
}
}
/**
* Update feedback settings for an event
*/
async updateEventFeedbackSettings(eventId, settings) {
try {
const existing = await db('event_feedback_settings')
.where('event_id', eventId)
.first();
if (existing) {
await db('event_feedback_settings')
.where('event_id', eventId)
.update({
...settings,
updated_at: new Date()
});
} else {
await db('event_feedback_settings').insert({
event_id: eventId,
...settings,
created_at: new Date(),
updated_at: new Date()
});
}
await logActivity('feedback_settings_updated', settings, eventId);
return this.getEventFeedbackSettings(eventId);
} catch (error) {
logger.error('Error updating feedback settings:', error);
throw error;
}
}
/**
* Submit feedback for a photo
*/
async submitFeedback(photoId, eventId, feedbackData, guestIdentifier) {
try {
const { feedback_type, rating, comment_text, guest_name, guest_email, ip_address, user_agent } = feedbackData;
// Validate feedback type
if (!['rating', 'like', 'comment', 'favorite'].includes(feedback_type)) {
throw new Error('Invalid feedback type');
}
// Check if similar feedback already exists (prevent duplicates)
if (feedback_type !== 'comment') {
const existing = await db('photo_feedback')
.where({
photo_id: photoId,
event_id: eventId,
feedback_type,
guest_identifier: guestIdentifier
})
.first();
if (existing) {
if (feedback_type === 'rating' && rating !== existing.rating) {
// Update existing rating
await db('photo_feedback')
.where('id', existing.id)
.update({
rating,
updated_at: new Date()
});
await this.updatePhotoFeedbackStats(photoId);
return { id: existing.id, updated: true };
}
// For likes and favorites, toggle off if already exists
if (feedback_type === 'like' || feedback_type === 'favorite') {
await db('photo_feedback')
.where('id', existing.id)
.delete();
await this.updatePhotoFeedbackStats(photoId);
return { removed: true };
}
return { id: existing.id, exists: true };
}
}
// Insert new feedback
const [id] = await db('photo_feedback').insert({
photo_id: photoId,
event_id: eventId,
feedback_type,
rating: feedback_type === 'rating' ? rating : null,
comment_text: feedback_type === 'comment' ? comment_text : null,
guest_name,
guest_email,
guest_identifier: guestIdentifier,
ip_address,
user_agent,
is_approved: feedback_type !== 'comment' || !feedbackData.moderate_comments,
created_at: new Date(),
updated_at: new Date()
});
// Update photo stats
await this.updatePhotoFeedbackStats(photoId);
// Log activity
await logActivity(`photo_${feedback_type}`, { photo_id: photoId }, eventId);
return { id, created: true };
} catch (error) {
logger.error('Error submitting feedback:', error);
throw error;
}
}
/**
* Get feedback for a photo
*/
async getPhotoFeedback(photoId, options = {}) {
try {
const query = db('photo_feedback')
.where('photo_id', photoId);
if (options.feedback_type) {
query.where('feedback_type', options.feedback_type);
}
if (options.approved_only) {
query.where('is_approved', true);
}
if (!options.include_hidden) {
query.where('is_hidden', false);
}
if (options.guest_identifier) {
query.where('guest_identifier', options.guest_identifier);
}
const feedback = await query
.orderBy('created_at', 'desc')
.select('id', 'feedback_type', 'rating', 'comment_text', 'guest_name', 'created_at');
return feedback;
} catch (error) {
logger.error('Error getting photo feedback:', error);
throw error;
}
}
/**
* Get feedback summary for an event
*/
async getEventFeedbackSummary(eventId) {
try {
const photos = await db('photos')
.where('event_id', eventId)
.select('id', 'filename', 'feedback_count', 'like_count', 'average_rating', 'favorite_count')
.orderBy('average_rating', 'desc')
.orderBy('like_count', 'desc');
const totalStats = await db('photo_feedback')
.where('event_id', eventId)
.select(
db.raw('COUNT(DISTINCT CASE WHEN feedback_type = ? THEN guest_identifier END) as unique_raters', ['rating']),
db.raw('COUNT(CASE WHEN feedback_type = ? THEN 1 END) as total_ratings', ['rating']),
db.raw('COUNT(CASE WHEN feedback_type = ? THEN 1 END) as total_likes', ['like']),
db.raw('COUNT(CASE WHEN feedback_type = ? THEN 1 END) as total_comments', ['comment']),
db.raw('COUNT(CASE WHEN feedback_type = ? THEN 1 END) as total_favorites', ['favorite'])
)
.first();
return {
photos,
stats: totalStats
};
} catch (error) {
logger.error('Error getting feedback summary:', error);
throw error;
}
}
/**
* Update photo feedback statistics
*/
async updatePhotoFeedbackStats(photoId) {
try {
// Get aggregated stats
const stats = await db('photo_feedback')
.where('photo_id', photoId)
.where('is_hidden', false)
.select(
db.raw('COUNT(CASE WHEN feedback_type = ? AND is_approved = ? THEN 1 END) as comment_count', ['comment', formatBoolean(true)]),
db.raw('COUNT(CASE WHEN feedback_type = ? THEN 1 END) as like_count', ['like']),
db.raw('COUNT(CASE WHEN feedback_type = ? THEN 1 END) as favorite_count', ['favorite']),
db.raw('AVG(CASE WHEN feedback_type = ? THEN rating END) as average_rating', ['rating']),
db.raw('COUNT(DISTINCT guest_identifier) as feedback_count')
)
.first();
// Update photo table
await db('photos')
.where('id', photoId)
.update({
feedback_count: stats.feedback_count || 0,
like_count: stats.like_count || 0,
average_rating: stats.average_rating || 0,
favorite_count: stats.favorite_count || 0
});
} catch (error) {
logger.error('Error updating photo feedback stats:', error);
throw error;
}
}
/**
* Moderate feedback (approve/hide)
*/
async moderateFeedback(feedbackId, action, adminId) {
try {
const updates = {
updated_at: new Date()
};
if (action === 'approve') {
updates.is_approved = true;
updates.is_hidden = false;
} else if (action === 'hide') {
updates.is_hidden = true;
} else if (action === 'reject') {
updates.is_approved = false;
updates.is_hidden = true;
}
const feedback = await db('photo_feedback')
.where('id', feedbackId)
.first();
if (!feedback) {
throw new Error('Feedback not found');
}
await db('photo_feedback')
.where('id', feedbackId)
.update(updates);
// Update photo stats if visibility changed
await this.updatePhotoFeedbackStats(feedback.photo_id);
// Log moderation action
await logActivity('feedback_moderated', {
feedback_id: feedbackId,
action,
admin_id: adminId
}, feedback.event_id);
return true;
} catch (error) {
logger.error('Error moderating feedback:', error);
throw error;
}
}
/**
* Delete feedback
*/
async deleteFeedback(feedbackId, adminId) {
try {
const feedback = await db('photo_feedback')
.where('id', feedbackId)
.first();
if (!feedback) {
throw new Error('Feedback not found');
}
await db('photo_feedback')
.where('id', feedbackId)
.delete();
// Update photo stats
await this.updatePhotoFeedbackStats(feedback.photo_id);
// Log deletion
await logActivity('feedback_deleted', {
feedback_id: feedbackId,
feedback_type: feedback.feedback_type,
admin_id: adminId
}, feedback.event_id);
return true;
} catch (error) {
logger.error('Error deleting feedback:', error);
throw error;
}
}
/**
* Get feedback requiring moderation
*/
async getPendingModeration(eventId = null) {
try {
let query = db('photo_feedback')
.join('photos', 'photo_feedback.photo_id', 'photos.id')
.join('events', 'photo_feedback.event_id', 'events.id')
.where('photo_feedback.is_approved', false)
.where('photo_feedback.is_hidden', false)
.where('photo_feedback.feedback_type', 'comment');
if (eventId) {
query = query.where('photo_feedback.event_id', eventId);
}
const pending = await query
.select(
'photo_feedback.*',
'photos.filename as photo_filename',
'events.event_name'
)
.orderBy('photo_feedback.created_at', 'desc');
return pending;
} catch (error) {
logger.error('Error getting pending moderation:', error);
throw error;
}
}
/**
* Export feedback data for an event
*/
async exportEventFeedback(eventId) {
try {
const feedback = await db('photo_feedback')
.join('photos', 'photo_feedback.photo_id', 'photos.id')
.where('photo_feedback.event_id', eventId)
.select(
'photos.filename',
'photo_feedback.feedback_type',
'photo_feedback.rating',
'photo_feedback.comment_text',
'photo_feedback.guest_name',
'photo_feedback.guest_email',
'photo_feedback.created_at'
)
.orderBy('photos.filename')
.orderBy('photo_feedback.created_at');
return feedback;
} catch (error) {
logger.error('Error exporting feedback:', error);
throw error;
}
}
}
module.exports = new FeedbackService();
-7
View File
@@ -51,13 +51,6 @@ async function processNewPhoto(filePath) {
const ext = path.extname(filePath).toLowerCase();
if (!['.jpg', '.jpeg', '.png', '.webp'].includes(ext)) return;
// Skip temporary upload files
const filename = path.basename(filePath);
if (filename.startsWith('temp_')) {
logger.debug(`Skipping temporary upload file: ${filename}`);
return;
}
// Find the event
const event = await db('events').where({ slug: eventSlug, is_active: formatBoolean(true) }).first();
if (!event) return;
+5 -92
View File
@@ -1,7 +1,6 @@
const sharp = require('sharp');
const path = require('path');
const fs = require('fs').promises;
const logger = require('../utils/logger');
// Configure sharp for better memory management with large batches
sharp.cache(false); // Disable cache to prevent memory buildup
@@ -11,7 +10,7 @@ const THUMBNAIL_WIDTH = 300;
const getStoragePath = () => process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
const getThumbnailPath = () => path.join(getStoragePath(), 'thumbnails');
async function generateThumbnail(imagePath, options = {}) {
async function generateThumbnail(imagePath) {
const filename = path.basename(imagePath);
const thumbnailFilename = `thumb_${filename}`;
const thumbnailDir = getThumbnailPath();
@@ -20,29 +19,11 @@ async function generateThumbnail(imagePath, options = {}) {
// Ensure thumbnail directory exists
await fs.mkdir(thumbnailDir, { recursive: true });
// Check if we need to regenerate (for broken thumbnails)
if (options.regenerate) {
try {
await fs.unlink(thumbnailPath);
logger.info(`Deleted broken thumbnail: ${thumbnailPath}`);
} catch (err) {
// File might not exist, that's okay
}
}
try {
// First, verify the source image is complete and valid
const metadata = await sharp(imagePath).metadata();
if (!metadata.width || !metadata.height) {
throw new Error('Invalid image metadata - file may be incomplete');
}
// Generate thumbnail with memory-efficient settings and error handling
// Generate thumbnail with memory-efficient settings
await sharp(imagePath, {
limitInputPixels: 268402689, // ~16k x 16k max
sequentialRead: true, // More memory efficient for large images
failOnError: false // Don't fail on minor issues
sequentialRead: true // More memory efficient for large images
})
.resize(THUMBNAIL_WIDTH, null, {
withoutEnlargement: true,
@@ -55,80 +36,12 @@ async function generateThumbnail(imagePath, options = {}) {
})
.toFile(thumbnailPath);
// Verify the thumbnail was created successfully
const stats = await fs.stat(thumbnailPath);
if (stats.size === 0) {
throw new Error('Generated thumbnail is empty');
}
return path.relative(getStoragePath(), thumbnailPath);
} catch (error) {
logger.error(`Failed to generate thumbnail for ${filename}:`, error.message);
// Clean up any partially created file
try {
await fs.unlink(thumbnailPath);
} catch (unlinkErr) {
// Ignore unlink errors
}
console.error(`Failed to generate thumbnail for ${filename}:`, error);
// Return null if thumbnail generation fails, don't fail the whole upload
return null;
}
}
/**
* Check if a thumbnail exists and is valid
*/
async function isThumbnailValid(thumbnailPath) {
try {
const fullPath = path.join(getStoragePath(), thumbnailPath);
const stats = await fs.stat(fullPath);
// Check if file exists and has content
if (stats.size === 0) {
return false;
}
// Try to read metadata to ensure it's a valid image
await sharp(fullPath).metadata();
return true;
} catch (error) {
return false;
}
}
/**
* Regenerate thumbnail if it's broken or missing
*/
async function ensureThumbnail(photo) {
const storagePath = getStoragePath();
const originalPath = path.join(storagePath, 'events/active', photo.path);
// Check if thumbnail exists and is valid
if (photo.thumbnail_path) {
const isValid = await isThumbnailValid(photo.thumbnail_path);
if (isValid) {
return photo.thumbnail_path;
}
logger.warn(`Invalid thumbnail detected for photo ${photo.id}, regenerating...`);
}
// Generate new thumbnail
const newThumbnailPath = await generateThumbnail(originalPath, { regenerate: true });
if (newThumbnailPath) {
// Update database with new thumbnail path
const { db } = require('../database/db');
await db('photos')
.where({ id: photo.id })
.update({ thumbnail_path: newThumbnailPath });
logger.info(`Regenerated thumbnail for photo ${photo.id}`);
return newThumbnailPath;
}
return null;
}
module.exports = { generateThumbnail, isThumbnailValid, ensureThumbnail };
module.exports = { generateThumbnail };
-283
View File
@@ -1,283 +0,0 @@
const rateLimit = require('express-rate-limit');
const jwt = require('jsonwebtoken');
const { db } = require('../database/db');
const logger = require('../utils/logger');
// Cache for rate limit settings
let settingsCache = null;
let cacheExpiry = 0;
const CACHE_DURATION = 60000; // 1 minute cache
/**
* Get rate limit settings from database with caching
*/
async function getRateLimitSettings() {
try {
// Check cache
if (settingsCache && Date.now() < cacheExpiry) {
return settingsCache;
}
// Fetch from database
const settings = await db('app_settings')
.whereIn('setting_key', [
'rate_limit_enabled',
'rate_limit_window_minutes',
'rate_limit_max_requests',
'rate_limit_auth_max_requests',
'rate_limit_skip_authenticated',
'rate_limit_public_endpoints_only'
]);
// Parse settings into object
const config = {
enabled: true,
windowMinutes: 15,
maxRequests: 100,
authMaxRequests: 5,
skipAuthenticated: true,
publicEndpointsOnly: false
};
settings.forEach(setting => {
const value = JSON.parse(setting.setting_value);
switch (setting.setting_key) {
case 'rate_limit_enabled':
config.enabled = value;
break;
case 'rate_limit_window_minutes':
config.windowMinutes = value;
break;
case 'rate_limit_max_requests':
config.maxRequests = value;
break;
case 'rate_limit_auth_max_requests':
config.authMaxRequests = value;
break;
case 'rate_limit_skip_authenticated':
config.skipAuthenticated = value;
break;
case 'rate_limit_public_endpoints_only':
config.publicEndpointsOnly = value;
break;
}
});
// Update cache
settingsCache = config;
cacheExpiry = Date.now() + CACHE_DURATION;
return config;
} catch (error) {
logger.error('Failed to fetch rate limit settings:', error);
// Return defaults on error
return {
enabled: true,
windowMinutes: 15,
maxRequests: 100,
authMaxRequests: 5,
skipAuthenticated: true,
publicEndpointsOnly: false
};
}
}
/**
* Clear settings cache (call when settings are updated)
*/
function clearSettingsCache() {
settingsCache = null;
cacheExpiry = 0;
}
/**
* Check if request has valid authentication
*/
function isAuthenticated(req) {
try {
const authHeader = req.headers.authorization;
if (!authHeader || !authHeader.startsWith('Bearer ')) {
return false;
}
const token = authHeader.substring(7);
const decoded = jwt.verify(token, process.env.JWT_SECRET);
// Check if token is valid
if (!decoded || typeof decoded !== 'object') {
return false;
}
// Valid token found - check type
req.tokenType = decoded.type; // 'admin' or 'gallery'
req.tokenPayload = decoded;
return true;
} catch (error) {
return false;
}
}
/**
* Determine if rate limiting should be applied to this request
*/
function shouldSkipRateLimit(req, config) {
// If rate limiting is disabled globally
if (!config.enabled) {
return true;
}
// Never skip rate limiting for auth endpoints
const isAuthEndpoint = req.path.match(/\/(auth|login|gallery\/[^/]+\/verify)$/);
if (isAuthEndpoint) {
return false;
}
// Check if we should skip authenticated requests
if (config.skipAuthenticated && isAuthenticated(req)) {
return true;
}
// Check if we only rate limit public endpoints
if (config.publicEndpointsOnly) {
const isPublicEndpoint = req.path.startsWith('/api/public/') ||
req.path.startsWith('/api/gallery/') ||
isAuthEndpoint;
return !isPublicEndpoint;
}
return false;
}
/**
* Create dynamic rate limiter
*/
async function createRateLimiter() {
const config = await getRateLimitSettings();
return rateLimit({
windowMs: config.windowMinutes * 60 * 1000,
max: async (req) => {
// Refresh config for each request
const currentConfig = await getRateLimitSettings();
// Different limits for auth endpoints
const isAuthEndpoint = req.path.match(/\/(auth|login|gallery\/[^/]+\/verify)$/);
return isAuthEndpoint ? currentConfig.authMaxRequests : currentConfig.maxRequests;
},
keyGenerator: (req) => {
// Use correct client IP when behind proxy
return req.headers['x-forwarded-for']?.split(',')[0]?.trim() ||
req.headers['x-real-ip'] ||
req.connection.remoteAddress ||
req.ip;
},
skip: async (req) => {
const currentConfig = await getRateLimitSettings();
return shouldSkipRateLimit(req, currentConfig);
},
handler: (req, res) => {
const clientIp = req.headers['x-forwarded-for']?.split(',')[0]?.trim() ||
req.headers['x-real-ip'] ||
req.connection.remoteAddress ||
req.ip;
// Enhanced logging for production analysis
logger.warn('Rate limit exceeded', {
ip: clientIp,
path: req.path,
method: req.method,
authenticated: isAuthenticated(req),
tokenType: req.tokenType,
userAgent: req.headers['user-agent'],
referer: req.headers['referer'],
origin: req.headers['origin'],
timestamp: new Date().toISOString(),
headers: {
'x-forwarded-for': req.headers['x-forwarded-for'],
'x-real-ip': req.headers['x-real-ip']
},
requestUrl: req.originalUrl,
rateLimitInfo: {
limit: req.rateLimit?.limit,
current: req.rateLimit?.current,
remaining: req.rateLimit?.remaining,
resetTime: req.rateLimit?.resetTime ? new Date(req.rateLimit.resetTime).toISOString() : null
}
});
res.status(429).json({
error: 'Too many requests, please try again later.',
retryAfter: res.getHeader('Retry-After')
});
},
standardHeaders: true, // Return rate limit info in headers
legacyHeaders: false, // Disable X-RateLimit headers
});
}
/**
* Create auth-specific rate limiter
*/
async function createAuthRateLimiter() {
const config = await getRateLimitSettings();
return rateLimit({
windowMs: config.windowMinutes * 60 * 1000,
max: config.authMaxRequests,
keyGenerator: (req) => {
// Use correct client IP when behind proxy
return req.headers['x-forwarded-for']?.split(',')[0]?.trim() ||
req.headers['x-real-ip'] ||
req.connection.remoteAddress ||
req.ip;
},
skip: async () => {
const currentConfig = await getRateLimitSettings();
return !currentConfig.enabled;
},
handler: (req, res) => {
const clientIp = req.headers['x-forwarded-for']?.split(',')[0]?.trim() ||
req.headers['x-real-ip'] ||
req.connection.remoteAddress ||
req.ip;
// Enhanced logging for auth failures
logger.warn('Auth rate limit exceeded', {
ip: clientIp,
path: req.path,
method: req.method,
userAgent: req.headers['user-agent'],
timestamp: new Date().toISOString(),
headers: {
'x-forwarded-for': req.headers['x-forwarded-for'],
'x-real-ip': req.headers['x-real-ip']
},
requestUrl: req.originalUrl,
authType: req.path.includes('admin') ? 'admin' : 'gallery',
rateLimitInfo: {
limit: req.rateLimit?.limit,
current: req.rateLimit?.current,
remaining: req.rateLimit?.remaining,
resetTime: req.rateLimit?.resetTime ? new Date(req.rateLimit.resetTime).toISOString() : null
}
});
res.status(429).json({
error: 'Too many authentication attempts, please try again later.',
retryAfter: res.getHeader('Retry-After')
});
},
standardHeaders: true,
legacyHeaders: false,
});
}
module.exports = {
getRateLimitSettings,
clearSettingsCache,
createRateLimiter,
createAuthRateLimiter,
isAuthenticated,
shouldSkipRateLimit
};
File diff suppressed because it is too large Load Diff
@@ -1,311 +0,0 @@
const S3StorageAdapter = require('../s3Storage');
const { S3Client } = require('@aws-sdk/client-s3');
const { Upload } = require('@aws-sdk/lib-storage');
const fs = require('fs');
const stream = require('stream');
// Mock AWS SDK
jest.mock('@aws-sdk/client-s3');
jest.mock('@aws-sdk/lib-storage');
jest.mock('@aws-sdk/s3-request-presigner');
describe('S3StorageAdapter', () => {
let mockS3Client;
let mockSend;
let s3Storage;
beforeEach(() => {
// Reset mocks
jest.clearAllMocks();
// Mock S3Client
mockSend = jest.fn();
mockS3Client = {
send: mockSend
};
S3Client.mockImplementation(() => mockS3Client);
// Create adapter instance
s3Storage = new S3StorageAdapter({
bucket: 'test-bucket',
region: 'us-east-1',
accessKeyId: 'test-key',
secretAccessKey: 'test-secret'
});
});
describe('constructor', () => {
it('should initialize with required config', () => {
expect(s3Storage.bucket).toBe('test-bucket');
expect(s3Storage.config.region).toBe('us-east-1');
});
it('should throw error if bucket is not provided', () => {
expect(() => {
new S3StorageAdapter({ region: 'us-east-1' });
}).toThrow('S3 bucket name is required');
});
it('should configure for MinIO with path style', () => {
const minioStorage = new S3StorageAdapter({
bucket: 'test-bucket',
endpoint: 'http://localhost:9000',
forcePathStyle: true,
sslEnabled: false
});
expect(S3Client).toHaveBeenCalledWith(
expect.objectContaining({
endpoint: 'http://localhost:9000',
forcePathStyle: true
})
);
});
});
describe('testConnection', () => {
it('should successfully test connection', async () => {
mockSend.mockResolvedValueOnce({});
const result = await s3Storage.testConnection();
expect(result).toBe(true);
expect(mockSend).toHaveBeenCalledWith(
expect.objectContaining({
input: { Bucket: 'test-bucket' }
})
);
});
it('should throw error on connection failure', async () => {
mockSend.mockRejectedValueOnce(new Error('Access Denied'));
await expect(s3Storage.testConnection()).rejects.toThrow('S3 connection test failed');
});
});
describe('upload', () => {
let mockUpload;
let mockDone;
beforeEach(() => {
mockDone = jest.fn().mockResolvedValue({
Location: 'https://test-bucket.s3.amazonaws.com/test-key',
ETag: '"test-etag"'
});
mockUpload = {
on: jest.fn().mockReturnThis(),
done: mockDone
};
Upload.mockImplementation(() => mockUpload);
// Mock fs.stat
jest.spyOn(fs.promises, 'stat').mockResolvedValue({
size: 1024
});
// Mock fs.createReadStream
jest.spyOn(fs, 'createReadStream').mockReturnValue(new stream.Readable());
});
afterEach(() => {
jest.restoreAllMocks();
});
it('should upload file successfully', async () => {
const result = await s3Storage.upload('/path/to/file.jpg', 'test-key');
expect(result.Location).toBe('https://test-bucket.s3.amazonaws.com/test-key');
expect(Upload).toHaveBeenCalledWith(
expect.objectContaining({
client: mockS3Client,
params: expect.objectContaining({
Bucket: 'test-bucket',
Key: 'test-key',
ContentType: 'application/octet-stream'
})
})
);
});
it('should track upload progress', async () => {
const onProgress = jest.fn();
let progressCallback;
mockUpload.on.mockImplementation((event, callback) => {
if (event === 'httpUploadProgress') {
progressCallback = callback;
}
return mockUpload;
});
const uploadPromise = s3Storage.upload('/path/to/file.jpg', 'test-key', {
onProgress
});
// Simulate progress
progressCallback({ loaded: 512, total: 1024 });
await uploadPromise;
expect(onProgress).toHaveBeenCalledWith(512, 1024);
});
it('should emit upload events', async () => {
const uploadStartSpy = jest.fn();
const uploadCompleteSpy = jest.fn();
s3Storage.on('uploadStart', uploadStartSpy);
s3Storage.on('uploadComplete', uploadCompleteSpy);
await s3Storage.upload('/path/to/file.jpg', 'test-key');
expect(uploadStartSpy).toHaveBeenCalledWith({ key: 'test-key', size: 1024 });
expect(uploadCompleteSpy).toHaveBeenCalledWith({
key: 'test-key',
location: 'https://test-bucket.s3.amazonaws.com/test-key'
});
});
});
describe('exists', () => {
it('should return true if object exists', async () => {
mockSend.mockResolvedValueOnce({});
const result = await s3Storage.exists('test-key');
expect(result).toBe(true);
expect(mockSend).toHaveBeenCalledWith(
expect.objectContaining({
input: { Bucket: 'test-bucket', Key: 'test-key' }
})
);
});
it('should return false if object does not exist', async () => {
const error = new Error('Not Found');
error.name = 'NotFound';
error.$metadata = { httpStatusCode: 404 };
mockSend.mockRejectedValueOnce(error);
const result = await s3Storage.exists('test-key');
expect(result).toBe(false);
});
});
describe('generateKey', () => {
it('should generate unique key with timestamp and random string', () => {
const key = s3Storage.generateKey('photo.jpg');
expect(key).toMatch(/^\d+_[a-f0-9]{16}_photo\.jpg$/);
});
it('should add prefix if provided', () => {
const key = s3Storage.generateKey('photo.jpg', 'events/wedding');
expect(key).toMatch(/^events\/wedding\/\d+_[a-f0-9]{16}_photo\.jpg$/);
});
it('should sanitize filename', () => {
const key = s3Storage.generateKey('my photo (1).jpg');
expect(key).toMatch(/^\d+_[a-f0-9]{16}_my_photo__1_\.jpg$/);
});
});
describe('retry logic', () => {
it('should retry on retryable errors', async () => {
const retryableError = new Error('Connection reset');
retryableError.code = 'ECONNRESET';
// First attempt fails, second succeeds
mockSend
.mockRejectedValueOnce(retryableError)
.mockResolvedValueOnce({});
// Mock setTimeout to speed up test
jest.useFakeTimers();
const promise = s3Storage.exists('test-key');
// Advance timers
jest.runAllTimers();
const result = await promise;
expect(result).toBe(true);
expect(mockSend).toHaveBeenCalledTimes(2);
jest.useRealTimers();
});
it('should not retry on non-retryable errors', async () => {
const nonRetryableError = new Error('Invalid credentials');
nonRetryableError.code = 'InvalidCredentials';
mockSend.mockRejectedValueOnce(nonRetryableError);
await expect(s3Storage.exists('test-key')).rejects.toThrow('Invalid credentials');
expect(mockSend).toHaveBeenCalledTimes(1);
});
it('should stop retrying after max attempts', async () => {
const retryableError = new Error('Service unavailable');
retryableError.code = 'ServiceUnavailable';
mockSend.mockRejectedValue(retryableError);
// Mock setTimeout to speed up test
jest.useFakeTimers();
const promise = s3Storage.exists('test-key');
// Advance timers for all retries
for (let i = 0; i < 4; i++) {
jest.runAllTimers();
}
await expect(promise).rejects.toThrow('Service unavailable');
expect(mockSend).toHaveBeenCalledTimes(4); // Initial + 3 retries
jest.useRealTimers();
});
});
describe('getStats', () => {
it('should calculate storage statistics', async () => {
mockSend.mockResolvedValueOnce({
Contents: [
{ Key: 'file1.jpg', Size: 1024 },
{ Key: 'file2.jpg', Size: 2048 }
],
NextContinuationToken: 'token123'
}).mockResolvedValueOnce({
Contents: [
{ Key: 'file3.jpg', Size: 3072 }
]
});
const stats = await s3Storage.getStats('events/');
expect(stats).toEqual({
totalSize: 6144,
totalCount: 3,
totalSizeFormatted: '6 KB'
});
});
});
describe('_formatBytes', () => {
it('should format bytes correctly', () => {
expect(s3Storage._formatBytes(0)).toBe('0 Bytes');
expect(s3Storage._formatBytes(1024)).toBe('1 KB');
expect(s3Storage._formatBytes(1048576)).toBe('1 MB');
expect(s3Storage._formatBytes(1073741824)).toBe('1 GB');
expect(s3Storage._formatBytes(1536, 1)).toBe('1.5 KB');
});
});
});
@@ -1,221 +0,0 @@
/**
* Example usage of the S3StorageAdapter
*
* This file demonstrates how to use the S3 storage adapter for various operations
*/
const S3StorageAdapter = require('./s3Storage');
// Example 1: Basic AWS S3 Configuration
const s3Storage = new S3StorageAdapter({
bucket: 'my-photo-bucket',
region: 'us-east-1',
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
});
// Example 2: MinIO Configuration (S3-compatible)
const minioStorage = new S3StorageAdapter({
bucket: 'photo-storage',
endpoint: 'http://localhost:9000', // MinIO endpoint
accessKeyId: 'minioadmin',
secretAccessKey: 'minioadmin',
forcePathStyle: true, // Required for MinIO
sslEnabled: false // For local development
});
// Example 3: DigitalOcean Spaces Configuration
const spacesStorage = new S3StorageAdapter({
bucket: 'my-space-name',
endpoint: 'https://nyc3.digitaloceanspaces.com',
region: 'nyc3',
accessKeyId: process.env.DO_SPACES_KEY,
secretAccessKey: process.env.DO_SPACES_SECRET
});
// Usage Examples
async function examples() {
try {
// Test connection
await s3Storage.testConnection();
console.log('Connection successful!');
// Upload a file with progress tracking
const uploadResult = await s3Storage.upload(
'/path/to/local/photo.jpg',
'events/wedding-2024/photo.jpg',
{
contentType: 'image/jpeg',
metadata: {
event: 'wedding-2024',
photographer: 'John Doe'
},
onProgress: (loaded, total) => {
const percentage = Math.round((loaded / total) * 100);
console.log(`Upload progress: ${percentage}%`);
}
}
);
console.log('Upload complete:', uploadResult.Location);
// Upload from stream
const readStream = fs.createReadStream('/path/to/large-video.mp4');
await s3Storage.uploadStream(readStream, 'events/wedding-2024/video.mp4', {
contentType: 'video/mp4',
onProgress: (loaded, total) => {
console.log(`Streamed ${loaded} of ${total} bytes`);
}
});
// Download a file
await s3Storage.download(
'events/wedding-2024/photo.jpg',
'/path/to/downloaded/photo.jpg',
{
onProgress: (loaded, total) => {
const percentage = Math.round((loaded / total) * 100);
console.log(`Download progress: ${percentage}%`);
}
}
);
// Get a download stream
const downloadStream = await s3Storage.downloadStream('events/wedding-2024/photo.jpg');
downloadStream.pipe(fs.createWriteStream('/path/to/output.jpg'));
// List files
const listing = await s3Storage.list('events/wedding-2024/');
console.log(`Found ${listing.Contents.length} files`);
listing.Contents.forEach(file => {
console.log(`- ${file.Key} (${file.Size} bytes)`);
});
// Generate pre-signed URL for temporary access
const downloadUrl = await s3Storage.getSignedUrl('getObject', 'events/wedding-2024/photo.jpg', {
expiresIn: 3600 // 1 hour
});
console.log('Pre-signed download URL:', downloadUrl);
// Generate pre-signed upload URL
const uploadUrl = await s3Storage.getSignedUrl('putObject', 'events/wedding-2024/new-photo.jpg', {
expiresIn: 1800, // 30 minutes
params: {
ContentType: 'image/jpeg'
}
});
console.log('Pre-signed upload URL:', uploadUrl);
// Check if file exists
const exists = await s3Storage.exists('events/wedding-2024/photo.jpg');
console.log('File exists:', exists);
// Get metadata
const metadata = await s3Storage.getMetadata('events/wedding-2024/photo.jpg');
console.log('File metadata:', metadata);
// Copy file
await s3Storage.copy(
'events/wedding-2024/photo.jpg',
'events/wedding-2024/photo-copy.jpg'
);
// Move file
await s3Storage.move(
'events/wedding-2024/photo-copy.jpg',
'events/wedding-2024/archived/photo.jpg'
);
// Delete file
await s3Storage.delete('events/wedding-2024/temp-photo.jpg');
// Delete multiple files
const deleteResult = await s3Storage.deleteMany([
'events/wedding-2024/temp1.jpg',
'events/wedding-2024/temp2.jpg',
'events/wedding-2024/temp3.jpg'
]);
console.log(`Deleted ${deleteResult.Deleted.length} files`);
// Get storage statistics
const stats = await s3Storage.getStats('events/');
console.log(`Total files: ${stats.totalCount}`);
console.log(`Total size: ${stats.totalSizeFormatted}`);
// Listen to events
s3Storage.on('uploadProgress', (data) => {
console.log(`Uploading ${data.key}: ${data.loaded}/${data.total}`);
});
s3Storage.on('uploadComplete', (data) => {
console.log(`Upload completed: ${data.key}`);
});
s3Storage.on('uploadError', (data) => {
console.error(`Upload failed for ${data.key}:`, data.error);
});
} catch (error) {
console.error('Error:', error);
}
}
// Integration with existing photo upload workflow
async function integrateWithPhotoUpload(eventId, files) {
const storage = new S3StorageAdapter({
bucket: process.env.S3_BUCKET,
region: process.env.AWS_REGION,
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
});
const uploadedPhotos = [];
for (const file of files) {
try {
// Generate unique S3 key
const s3Key = storage.generateKey(file.originalname, `events/${eventId}`);
// Upload to S3
const result = await storage.upload(file.path, s3Key, {
contentType: file.mimetype,
metadata: {
eventId: eventId,
originalName: file.originalname,
uploadedAt: new Date().toISOString()
}
});
uploadedPhotos.push({
filename: s3Key,
originalName: file.originalname,
size: file.size,
mimeType: file.mimetype,
s3Location: result.Location,
s3Key: s3Key
});
// Clean up local temp file
await fs.promises.unlink(file.path);
} catch (error) {
console.error(`Failed to upload ${file.originalname}:`, error);
throw error;
}
}
return uploadedPhotos;
}
// Environment variables needed:
// AWS_ACCESS_KEY_ID=your-access-key
// AWS_SECRET_ACCESS_KEY=your-secret-key
// AWS_REGION=us-east-1
// S3_BUCKET=your-bucket-name
// For MinIO:
// MINIO_ENDPOINT=http://localhost:9000
// MINIO_ACCESS_KEY=minioadmin
// MINIO_SECRET_KEY=minioadmin
// MINIO_BUCKET=photo-storage
module.exports = { examples, integrateWithPhotoUpload };
-733
View File
@@ -1,733 +0,0 @@
const { S3Client, HeadBucketCommand, HeadObjectCommand, GetObjectCommand, PutObjectCommand, DeleteObjectCommand, DeleteObjectsCommand, ListObjectsV2Command, CopyObjectCommand, CreateMultipartUploadCommand, UploadPartCommand, CompleteMultipartUploadCommand, AbortMultipartUploadCommand } = require('@aws-sdk/client-s3');
const { Upload } = require('@aws-sdk/lib-storage');
const { getSignedUrl } = require('@aws-sdk/s3-request-presigner');
const fs = require('fs');
const fsPromises = require('fs').promises;
const path = require('path');
const stream = require('stream');
const crypto = require('crypto');
const logger = require('../../utils/logger');
/**
* S3 Storage Adapter for handling file uploads to S3 and S3-compatible services
*
* Features:
* - Support for S3 and S3-compatible services (MinIO, DigitalOcean Spaces, etc.)
* - Multipart upload for large files (>100MB)
* - Progress tracking with event emitter
* - Retry logic with exponential backoff
* - Stream support for memory efficiency
* - Path-style URL support for MinIO
* - Connection testing
* - Comprehensive error handling
*
* @class S3StorageAdapter
*/
class S3StorageAdapter extends stream.EventEmitter {
/**
* Creates an instance of S3StorageAdapter
*
* @param {Object} config - Configuration object
* @param {string} config.bucket - S3 bucket name
* @param {string} [config.region='us-east-1'] - AWS region
* @param {string} [config.endpoint] - Custom endpoint URL for S3-compatible services
* @param {string} [config.accessKeyId] - AWS access key ID
* @param {string} [config.secretAccessKey] - AWS secret access key
* @param {boolean} [config.forcePathStyle=false] - Force path-style URLs (required for MinIO)
* @param {boolean} [config.sslEnabled=true] - Enable SSL for connections
* @param {number} [config.multipartThreshold=104857600] - Threshold for multipart upload (default 100MB)
* @param {number} [config.partSize=10485760] - Part size for multipart upload (default 10MB)
* @param {number} [config.maxRetries=3] - Maximum number of retry attempts
* @param {number} [config.retryDelay=1000] - Initial retry delay in milliseconds
*/
constructor(config) {
super();
// Validate required config
if (!config.bucket) {
throw new Error('S3 bucket name is required');
}
// Set defaults
this.config = {
region: 'us-east-1',
forcePathStyle: false,
sslEnabled: true,
multipartThreshold: 100 * 1024 * 1024, // 100MB
partSize: 10 * 1024 * 1024, // 10MB
maxRetries: 3,
retryDelay: 1000,
...config
};
// Initialize S3 client
const s3Config = {
region: this.config.region,
forcePathStyle: this.config.forcePathStyle
};
// Add credentials if provided
if (this.config.accessKeyId && this.config.secretAccessKey) {
s3Config.credentials = {
accessKeyId: this.config.accessKeyId,
secretAccessKey: this.config.secretAccessKey
};
}
// Add custom endpoint if provided (for S3-compatible services)
if (this.config.endpoint) {
s3Config.endpoint = this.config.endpoint;
// For MinIO and other S3-compatible services
if (!this.config.endpoint.startsWith('https://') && this.config.sslEnabled) {
s3Config.endpoint = `https://${this.config.endpoint}`;
} else if (!this.config.endpoint.startsWith('http://') && !this.config.sslEnabled) {
s3Config.endpoint = `http://${this.config.endpoint}`;
}
}
this.s3Client = new S3Client(s3Config);
this.bucket = this.config.bucket;
// Bind methods to preserve context
this.upload = this.upload.bind(this);
this.uploadStream = this.uploadStream.bind(this);
this.download = this.download.bind(this);
this.downloadStream = this.downloadStream.bind(this);
this.delete = this.delete.bind(this);
this.exists = this.exists.bind(this);
this.list = this.list.bind(this);
this.copy = this.copy.bind(this);
this.move = this.move.bind(this);
this.getSignedUrl = this.getSignedUrl.bind(this);
this.testConnection = this.testConnection.bind(this);
}
/**
* Test connection to S3 bucket
*
* @returns {Promise<boolean>} - True if connection successful
* @throws {Error} - If connection fails
*/
async testConnection() {
try {
await this.s3Client.send(new HeadBucketCommand({ Bucket: this.bucket }));
logger.info(`Successfully connected to S3 bucket: ${this.bucket}`);
return true;
} catch (error) {
logger.error(`Failed to connect to S3 bucket ${this.bucket}:`, error);
throw new Error(`S3 connection test failed: ${error.message}`);
}
}
/**
* Upload a file to S3 with automatic multipart for large files
*
* @param {string} localPath - Local file path to upload
* @param {string} s3Key - S3 object key (path in bucket)
* @param {Object} [options={}] - Additional options
* @param {Object} [options.metadata] - Object metadata
* @param {string} [options.contentType] - Content type
* @param {string} [options.cacheControl] - Cache control header
* @param {Function} [options.onProgress] - Progress callback function(loaded, total)
* @returns {Promise<Object>} - Upload result with Location, ETag, etc.
*/
async upload(localPath, s3Key, options = {}) {
try {
const stats = await fsPromises.stat(localPath);
const fileSize = stats.size;
// Emit upload start event
this.emit('uploadStart', { key: s3Key, size: fileSize });
// Create file stream
const fileStream = fs.createReadStream(localPath);
// Prepare upload parameters
const uploadParams = {
Bucket: this.bucket,
Key: s3Key,
Body: fileStream,
ContentType: options.contentType || 'application/octet-stream',
Metadata: options.metadata || {},
CacheControl: options.cacheControl
};
// Remove undefined values
Object.keys(uploadParams).forEach(key => uploadParams[key] === undefined && delete uploadParams[key]);
// Use AWS SDK v3 Upload for automatic multipart handling
const parallelUploads3 = new Upload({
client: this.s3Client,
params: uploadParams,
queueSize: 4, // Optional: concurrent uploads
partSize: this.config.partSize,
leavePartsOnError: false
});
// Track upload progress
parallelUploads3.on('httpUploadProgress', (progress) => {
if (options.onProgress) {
options.onProgress(progress.loaded, progress.total);
}
this.emit('uploadProgress', { key: s3Key, loaded: progress.loaded, total: progress.total });
});
// Perform upload with retry
const result = await this._retryOperation(() => parallelUploads3.done());
this.emit('uploadComplete', { key: s3Key, location: result.Location });
return result;
} catch (error) {
logger.error(`Failed to upload file ${localPath} to S3:`, error);
this.emit('uploadError', { key: s3Key, error });
throw error;
}
}
/**
* Upload a stream to S3
*
* @param {stream.Readable} readStream - Readable stream to upload
* @param {string} s3Key - S3 object key
* @param {Object} [options={}] - Additional options
* @returns {Promise<Object>} - Upload result
*/
async uploadStream(readStream, s3Key, options = {}) {
const uploadParams = {
Bucket: this.bucket,
Key: s3Key,
Body: readStream,
ContentType: options.contentType || 'application/octet-stream',
Metadata: options.metadata || {},
CacheControl: options.cacheControl
};
// Remove undefined values
Object.keys(uploadParams).forEach(key => uploadParams[key] === undefined && delete uploadParams[key]);
const parallelUploads3 = new Upload({
client: this.s3Client,
params: uploadParams,
queueSize: 4,
partSize: this.config.partSize,
leavePartsOnError: false
});
// Track progress if callback provided
if (options.onProgress) {
parallelUploads3.on('httpUploadProgress', (progress) => {
options.onProgress(progress.loaded, progress.total);
this.emit('uploadProgress', { key: s3Key, ...progress });
});
}
return await this._retryOperation(() => parallelUploads3.done());
}
/**
* Download a file from S3
*
* @param {string} s3Key - S3 object key
* @param {string} localPath - Local file path to save to
* @param {Object} [options={}] - Additional options
* @param {Function} [options.onProgress] - Progress callback
* @returns {Promise<void>}
*/
async download(s3Key, localPath, options = {}) {
try {
// Ensure directory exists
await fsPromises.mkdir(path.dirname(localPath), { recursive: true });
// Get object metadata first for progress tracking
const headResult = await this._retryOperation(() =>
this.s3Client.send(new HeadObjectCommand({
Bucket: this.bucket,
Key: s3Key
}))
);
const fileSize = headResult.ContentLength;
this.emit('downloadStart', { key: s3Key, size: fileSize });
// Get object
const getObjectResult = await this._retryOperation(() =>
this.s3Client.send(new GetObjectCommand({
Bucket: this.bucket,
Key: s3Key
}))
);
// Create write stream
const writeStream = fs.createWriteStream(localPath);
// Download with progress tracking
await new Promise((resolve, reject) => {
let downloaded = 0;
const bodyStream = getObjectResult.Body;
bodyStream.on('data', (chunk) => {
downloaded += chunk.length;
if (options.onProgress) {
options.onProgress(downloaded, fileSize);
}
this.emit('downloadProgress', { key: s3Key, loaded: downloaded, total: fileSize });
});
bodyStream.on('error', reject);
writeStream.on('error', reject);
writeStream.on('finish', resolve);
bodyStream.pipe(writeStream);
});
this.emit('downloadComplete', { key: s3Key });
} catch (error) {
logger.error(`Failed to download file ${s3Key} from S3:`, error);
this.emit('downloadError', { key: s3Key, error });
throw error;
}
}
/**
* Get a download stream from S3
*
* @param {string} s3Key - S3 object key
* @param {Object} [options={}] - Additional options
* @param {string} [options.range] - Byte range to download (e.g., 'bytes=0-1023')
* @returns {Promise<stream.Readable>} - Readable stream
*/
async downloadStream(s3Key, options = {}) {
const params = {
Bucket: this.bucket,
Key: s3Key,
Range: options.range
};
// Remove undefined values
Object.keys(params).forEach(key => params[key] === undefined && delete params[key]);
const result = await this.s3Client.send(new GetObjectCommand(params));
return result.Body;
}
/**
* Delete a file from S3
*
* @param {string} s3Key - S3 object key
* @returns {Promise<void>}
*/
async delete(s3Key) {
return await this._retryOperation(() =>
this.s3Client.send(new DeleteObjectCommand({
Bucket: this.bucket,
Key: s3Key
}))
);
}
/**
* Delete multiple files from S3
*
* @param {string[]} s3Keys - Array of S3 object keys
* @returns {Promise<Object>} - Delete result
*/
async deleteMany(s3Keys) {
if (!s3Keys || s3Keys.length === 0) {
return { Deleted: [], Errors: [] };
}
// S3 deleteObjects has a limit of 1000 keys per request
const chunks = [];
for (let i = 0; i < s3Keys.length; i += 1000) {
chunks.push(s3Keys.slice(i, i + 1000));
}
const results = await Promise.all(
chunks.map(chunk =>
this._retryOperation(() =>
this.s3Client.send(new DeleteObjectsCommand({
Bucket: this.bucket,
Delete: {
Objects: chunk.map(key => ({ Key: key })),
Quiet: false
}
}))
)
)
);
// Combine results
return {
Deleted: results.flatMap(r => r.Deleted || []),
Errors: results.flatMap(r => r.Errors || [])
};
}
/**
* Check if a file exists in S3
*
* @param {string} s3Key - S3 object key
* @returns {Promise<boolean>} - True if exists
*/
async exists(s3Key) {
try {
await this.s3Client.send(new HeadObjectCommand({
Bucket: this.bucket,
Key: s3Key
}));
return true;
} catch (error) {
if (error.name === 'NotFound' || error.$metadata?.httpStatusCode === 404) {
return false;
}
throw error;
}
}
/**
* List files in S3
*
* @param {string} prefix - S3 prefix to list
* @param {Object} [options={}] - Additional options
* @param {number} [options.maxKeys=1000] - Maximum number of keys to return
* @param {string} [options.continuationToken] - Continuation token for pagination
* @returns {Promise<Object>} - List result with Contents array and NextContinuationToken
*/
async list(prefix, options = {}) {
const params = {
Bucket: this.bucket,
Prefix: prefix,
MaxKeys: options.maxKeys || 1000,
ContinuationToken: options.continuationToken
};
return await this._retryOperation(() =>
this.s3Client.send(new ListObjectsV2Command(params))
);
}
/**
* Copy a file within S3
*
* @param {string} sourceKey - Source S3 object key
* @param {string} targetKey - Target S3 object key
* @param {Object} [options={}] - Additional options
* @returns {Promise<Object>} - Copy result
*/
async copy(sourceKey, targetKey, options = {}) {
const copySource = `${this.bucket}/${sourceKey}`;
const params = {
Bucket: this.bucket,
CopySource: copySource,
Key: targetKey,
MetadataDirective: options.metadata ? 'REPLACE' : 'COPY',
Metadata: options.metadata,
ContentType: options.contentType,
CacheControl: options.cacheControl
};
// Remove undefined values
Object.keys(params).forEach(key => params[key] === undefined && delete params[key]);
return await this._retryOperation(() =>
this.s3Client.send(new CopyObjectCommand(params))
);
}
/**
* Move a file within S3 (copy then delete)
*
* @param {string} sourceKey - Source S3 object key
* @param {string} targetKey - Target S3 object key
* @param {Object} [options={}] - Additional options
* @returns {Promise<Object>} - Move result
*/
async move(sourceKey, targetKey, options = {}) {
// First copy the object
const copyResult = await this.copy(sourceKey, targetKey, options);
// Then delete the original
await this.delete(sourceKey);
return copyResult;
}
/**
* Get a pre-signed URL for downloading or uploading
*
* @param {string} operation - Operation type ('getObject' or 'putObject')
* @param {string} s3Key - S3 object key
* @param {Object} [options={}] - Additional options
* @param {number} [options.expiresIn=3600] - URL expiration in seconds
* @param {Object} [options.params] - Additional parameters for the operation
* @returns {Promise<string>} - Pre-signed URL
*/
async getSignedUrl(operation, s3Key, options = {}) {
const params = {
Bucket: this.bucket,
Key: s3Key,
...options.params
};
let command;
switch (operation.toLowerCase()) {
case 'getobject':
command = new GetObjectCommand(params);
break;
case 'putobject':
command = new PutObjectCommand(params);
break;
default:
throw new Error(`Unsupported operation: ${operation}`);
}
return await getSignedUrl(this.s3Client, command, {
expiresIn: options.expiresIn || 3600
});
}
/**
* Get object metadata
*
* @param {string} s3Key - S3 object key
* @returns {Promise<Object>} - Object metadata
*/
async getMetadata(s3Key) {
return await this._retryOperation(() =>
this.s3Client.send(new HeadObjectCommand({
Bucket: this.bucket,
Key: s3Key
}))
);
}
/**
* Update object metadata
*
* @param {string} s3Key - S3 object key
* @param {Object} metadata - New metadata
* @returns {Promise<Object>} - Update result
*/
async updateMetadata(s3Key, metadata) {
// S3 requires copying the object to itself to update metadata
return await this.copy(s3Key, s3Key, { metadata });
}
/**
* Manual multipart upload for advanced use cases
*
* @param {string} localPath - Local file path
* @param {string} s3Key - S3 object key
* @param {number} fileSize - File size in bytes
* @param {Object} options - Upload options
* @returns {Promise<Object>} - Upload result
* @private
*/
async _manualMultipartUpload(localPath, s3Key, fileSize, options) {
logger.info(`Starting manual multipart upload for ${s3Key} (${fileSize} bytes)`);
// Initiate multipart upload
const multipartParams = {
Bucket: this.bucket,
Key: s3Key,
ContentType: options.contentType || 'application/octet-stream',
Metadata: options.metadata || {},
CacheControl: options.cacheControl
};
// Remove undefined values
Object.keys(multipartParams).forEach(key => multipartParams[key] === undefined && delete multipartParams[key]);
const multipart = await this._retryOperation(() =>
this.s3Client.send(new CreateMultipartUploadCommand(multipartParams))
);
const uploadId = multipart.UploadId;
const partSize = this.config.partSize;
const numParts = Math.ceil(fileSize / partSize);
let uploaded = 0;
const parts = [];
try {
// Upload parts
for (let partNum = 1; partNum <= numParts; partNum++) {
const start = (partNum - 1) * partSize;
const end = Math.min(start + partSize, fileSize);
const partStream = fs.createReadStream(localPath, {
start,
end: end - 1
});
const partParams = {
Bucket: this.bucket,
Key: s3Key,
PartNumber: partNum,
UploadId: uploadId,
Body: partStream
};
// Upload part with retry
const partResult = await this._retryOperation(() =>
this.s3Client.send(new UploadPartCommand(partParams))
);
parts.push({
ETag: partResult.ETag,
PartNumber: partNum
});
uploaded += (end - start);
if (options.onProgress) {
options.onProgress(uploaded, fileSize);
}
this.emit('uploadProgress', { key: s3Key, loaded: uploaded, total: fileSize });
logger.info(`Uploaded part ${partNum}/${numParts} for ${s3Key}`);
}
// Complete multipart upload
const completeParams = {
Bucket: this.bucket,
Key: s3Key,
UploadId: uploadId,
MultipartUpload: { Parts: parts }
};
const result = await this._retryOperation(() =>
this.s3Client.send(new CompleteMultipartUploadCommand(completeParams))
);
this.emit('uploadComplete', { key: s3Key, location: result.Location });
logger.info(`Completed multipart upload for ${s3Key}`);
return result;
} catch (error) {
// Abort multipart upload on error
logger.error(`Multipart upload failed for ${s3Key}, aborting:`, error);
try {
await this.s3Client.send(new AbortMultipartUploadCommand({
Bucket: this.bucket,
Key: s3Key,
UploadId: uploadId
}));
} catch (abortError) {
logger.error(`Failed to abort multipart upload:`, abortError);
}
throw error;
}
}
/**
* Retry operation with exponential backoff
* @private
*/
async _retryOperation(operation, retryCount = 0) {
try {
return await operation();
} catch (error) {
if (retryCount >= this.config.maxRetries) {
throw error;
}
// Check if error is retryable
const retryableErrors = ['ECONNRESET', 'ETIMEDOUT', 'ENOTFOUND', 'ESOCKETTIMEDOUT', 'RequestTimeout', 'SlowDown', 'ServiceUnavailable', 'InternalError'];
const isRetryable = retryableErrors.some(code =>
error.code === code ||
error.name === code ||
error.$metadata?.httpStatusCode === 503 ||
error.$metadata?.httpStatusCode === 500
);
if (!isRetryable) {
throw error;
}
// Calculate delay with exponential backoff and jitter
const delay = Math.min(
this.config.retryDelay * Math.pow(2, retryCount) + Math.random() * 1000,
30000 // Max 30 seconds
);
logger.warn(`Retrying operation after ${delay}ms (attempt ${retryCount + 1}/${this.config.maxRetries}):`, error.message);
await new Promise(resolve => setTimeout(resolve, delay));
return this._retryOperation(operation, retryCount + 1);
}
}
/**
* Generate a unique S3 key for a file
*
* @param {string} originalName - Original filename
* @param {string} [prefix=''] - Optional prefix for the key
* @returns {string} - Generated S3 key
*/
generateKey(originalName, prefix = '') {
const timestamp = Date.now();
const randomStr = crypto.randomBytes(8).toString('hex');
const ext = path.extname(originalName);
const basename = path.basename(originalName, ext);
// Sanitize basename
const safeName = basename.replace(/[^a-zA-Z0-9-_]/g, '_');
const key = `${timestamp}_${randomStr}_${safeName}${ext}`;
return prefix ? path.posix.join(prefix, key) : key;
}
/**
* Get storage statistics
*
* @param {string} [prefix=''] - Optional prefix to filter
* @returns {Promise<Object>} - Storage statistics
*/
async getStats(prefix = '') {
let totalSize = 0;
let totalCount = 0;
let continuationToken;
do {
const result = await this.list(prefix, { continuationToken });
if (result.Contents) {
totalCount += result.Contents.length;
totalSize += result.Contents.reduce((sum, obj) => sum + (obj.Size || 0), 0);
}
continuationToken = result.NextContinuationToken;
} while (continuationToken);
return {
totalSize,
totalCount,
totalSizeFormatted: this._formatBytes(totalSize)
};
}
/**
* Format bytes to human readable format
* @private
*/
_formatBytes(bytes, decimals = 2) {
if (bytes === 0) return '0 Bytes';
const k = 1024;
const dm = decimals < 0 ? 0 : decimals;
const sizes = ['Bytes', 'KB', 'MB', 'GB', 'TB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return parseFloat((bytes / Math.pow(k, i)).toFixed(dm)) + ' ' + sizes[i];
}
}
module.exports = S3StorageAdapter;
+24 -24
View File
@@ -56,30 +56,30 @@ class WatermarkService {
let left, top;
switch (position) {
case 'top-left':
left = padding;
top = padding;
break;
case 'top-right':
left = imageWidth - watermarkWidth - padding;
top = padding;
break;
case 'bottom-left':
left = padding;
top = imageHeight - watermarkHeight - padding;
break;
case 'bottom-right':
left = imageWidth - watermarkWidth - padding;
top = imageHeight - watermarkHeight - padding;
break;
case 'center':
left = Math.floor((imageWidth - watermarkWidth) / 2);
top = Math.floor((imageHeight - watermarkHeight) / 2);
break;
default:
// Default to bottom-right
left = imageWidth - watermarkWidth - padding;
top = imageHeight - watermarkHeight - padding;
case 'top-left':
left = padding;
top = padding;
break;
case 'top-right':
left = imageWidth - watermarkWidth - padding;
top = padding;
break;
case 'bottom-left':
left = padding;
top = imageHeight - watermarkHeight - padding;
break;
case 'bottom-right':
left = imageWidth - watermarkWidth - padding;
top = imageHeight - watermarkHeight - padding;
break;
case 'center':
left = Math.floor((imageWidth - watermarkWidth) / 2);
top = Math.floor((imageHeight - watermarkHeight) / 2);
break;
default:
// Default to bottom-right
left = imageWidth - watermarkWidth - padding;
top = imageHeight - watermarkHeight - padding;
}
return { left: Math.max(0, left), top: Math.max(0, top) };
-78
View File
@@ -1,78 +0,0 @@
const path = require('path');
const fs = require('fs').promises;
const logger = require('./logger');
const getStoragePath = () => process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
/**
* Clean up old temporary upload directories
* Removes temp directories older than 1 hour
*/
async function cleanupTempUploads() {
const tempPath = path.join(getStoragePath(), 'temp');
try {
// Ensure temp directory exists
await fs.mkdir(tempPath, { recursive: true });
// Read all items in temp directory
const items = await fs.readdir(tempPath);
let cleanedCount = 0;
const oneHourAgo = Date.now() - (60 * 60 * 1000); // 1 hour
for (const item of items) {
const itemPath = path.join(tempPath, item);
try {
const stats = await fs.stat(itemPath);
// Only process directories that match our upload pattern
if (stats.isDirectory() && item.startsWith('upload_')) {
// Extract timestamp from directory name
const parts = item.split('_');
if (parts.length >= 2) {
const timestamp = parseInt(parts[1]);
// Remove if older than 1 hour
if (!isNaN(timestamp) && timestamp < oneHourAgo) {
logger.info(`Cleaning up old temp upload directory: ${item}`);
await fs.rm(itemPath, { recursive: true, force: true });
cleanedCount++;
}
}
}
} catch (error) {
logger.error(`Error processing temp item ${item}:`, error.message);
}
}
if (cleanedCount > 0) {
logger.info(`Cleaned up ${cleanedCount} old temp upload directories`);
}
} catch (error) {
logger.error('Error during temp upload cleanup:', error);
}
}
/**
* Start periodic cleanup of temp uploads
* Runs every hour
*/
function startTempUploadCleanup() {
// Run immediately on startup
cleanupTempUploads();
// Then run every hour
setInterval(() => {
cleanupTempUploads();
}, 60 * 60 * 1000); // 1 hour
logger.info('Temp upload cleanup service started');
}
module.exports = {
cleanupTempUploads,
startTempUploadCleanup
};
+28 -49
View File
@@ -27,28 +27,7 @@ async function formatDate(date, language = 'en') {
}
}
// Ensure proper date parsing
let dateObj;
if (date instanceof Date) {
dateObj = date;
} else if (typeof date === 'string') {
// For date strings like "2025-07-16", parse as local date to avoid timezone issues
if (date.match(/^\d{4}-\d{2}-\d{2}$/)) {
// Parse YYYY-MM-DD format as local date
const [year, month, day] = date.split('-').map(num => parseInt(num, 10));
dateObj = new Date(year, month - 1, day);
} else {
dateObj = new Date(date);
}
} else {
dateObj = new Date(date);
}
// Check if date is valid
if (isNaN(dateObj.getTime())) {
console.error('Invalid date provided to formatDate:', date);
throw new Error('Invalid date');
}
const dateObj = date instanceof Date ? date : new Date(date);
// Use appropriate locale based on language
let locale = dateConfig.locale || 'en-GB';
@@ -60,33 +39,33 @@ async function formatDate(date, language = 'en') {
// Format based on the configured format
switch (dateConfig.format) {
case 'MM/DD/YYYY':
return dateObj.toLocaleDateString(locale, {
month: '2-digit',
day: '2-digit',
year: 'numeric'
});
case 'DD/MM/YYYY':
return dateObj.toLocaleDateString(locale, {
day: '2-digit',
month: '2-digit',
year: 'numeric'
});
case 'YYYY-MM-DD':
return dateObj.toISOString().split('T')[0];
case 'DD.MM.YYYY':
return dateObj.toLocaleDateString('de-DE', {
day: '2-digit',
month: '2-digit',
year: 'numeric'
});
default:
// Use long format as fallback
return dateObj.toLocaleDateString(locale, {
year: 'numeric',
month: 'long',
day: 'numeric'
});
case 'MM/DD/YYYY':
return dateObj.toLocaleDateString(locale, {
month: '2-digit',
day: '2-digit',
year: 'numeric'
});
case 'DD/MM/YYYY':
return dateObj.toLocaleDateString(locale, {
day: '2-digit',
month: '2-digit',
year: 'numeric'
});
case 'YYYY-MM-DD':
return dateObj.toISOString().split('T')[0];
case 'DD.MM.YYYY':
return dateObj.toLocaleDateString('de-DE', {
day: '2-digit',
month: '2-digit',
year: 'numeric'
});
default:
// Use long format as fallback
return dateObj.toLocaleDateString(locale, {
year: 'numeric',
month: 'long',
day: 'numeric'
});
}
} catch (error) {
console.error('Error formatting date:', error);
-253
View File
@@ -1,253 +0,0 @@
const { body, param, validationResult } = require('express-validator');
const validator = require('validator');
/**
* Validation rules for feedback submission
*/
const feedbackValidationRules = {
rating: [
body('feedback_type').equals('rating'),
body('rating')
.isInt({ min: 1, max: 5 })
.withMessage('Rating must be between 1 and 5'),
body('guest_name')
.optional()
.trim()
.isLength({ max: 100 })
.withMessage('Name must be less than 100 characters'),
body('guest_email')
.optional()
.trim()
.isEmail()
.normalizeEmail()
.withMessage('Invalid email address')
],
like: [
body('feedback_type').equals('like'),
body('guest_name')
.optional()
.trim()
.isLength({ max: 100 }),
body('guest_email')
.optional()
.trim()
.isEmail()
.normalizeEmail()
],
favorite: [
body('feedback_type').equals('favorite'),
body('guest_name')
.optional()
.trim()
.isLength({ max: 100 }),
body('guest_email')
.optional()
.trim()
.isEmail()
.normalizeEmail()
],
comment: [
body('feedback_type').equals('comment'),
body('comment_text')
.trim()
.notEmpty()
.withMessage('Comment cannot be empty')
.isLength({ min: 1, max: 1000 })
.withMessage('Comment must be between 1 and 1000 characters')
.customSanitizer(value => sanitizeComment(value)),
body('guest_name')
.optional()
.trim()
.isLength({ max: 100 })
.withMessage('Name must be less than 100 characters'),
body('guest_email')
.optional()
.trim()
.isEmail()
.normalizeEmail()
.withMessage('Invalid email address')
]
};
/**
* Sanitize comment text
*/
function sanitizeComment(text) {
if (!text) return '';
// Remove excessive whitespace
text = text.replace(/\s+/g, ' ').trim();
// Remove zero-width characters
text = text.replace(/[\u200B-\u200D\uFEFF]/g, '');
// Remove control characters
text = text.replace(/[\x00-\x1F\x7F]/g, '');
// Limit consecutive special characters
text = text.replace(/([!?.]){4,}/g, '$1$1$1');
// Remove script tags and other dangerous HTML (basic sanitization)
text = text.replace(/<script[^>]*>[\s\S]*?<\/script>/gi, '');
text = text.replace(/<iframe[^>]*>[\s\S]*?<\/iframe>/gi, '');
text = text.replace(/<object[^>]*>[\s\S]*?<\/object>/gi, '');
text = text.replace(/<embed[^>]*>/gi, '');
return text;
}
/**
* Validate feedback type parameter
*/
const validateFeedbackType = param('feedbackType')
.isIn(['rating', 'like', 'comment', 'favorite'])
.withMessage('Invalid feedback type');
/**
* Validate photo ID parameter
*/
const validatePhotoId = param('photoId')
.isInt({ min: 1 })
.withMessage('Invalid photo ID');
/**
* Validate event ID parameter
*/
const validateEventId = param('eventId')
.isInt({ min: 1 })
.withMessage('Invalid event ID');
/**
* Get validation rules based on feedback type
*/
function getValidationRules(feedbackType) {
return feedbackValidationRules[feedbackType] || [];
}
/**
* Validation middleware for feedback submission
*/
const validateFeedbackSubmission = [
body('feedback_type')
.isIn(['rating', 'like', 'comment', 'favorite'])
.withMessage('Invalid feedback type'),
// Conditional validation based on feedback type
body('rating')
.if(body('feedback_type').equals('rating'))
.isInt({ min: 1, max: 5 })
.withMessage('Rating must be between 1 and 5'),
body('comment_text')
.if(body('feedback_type').equals('comment'))
.trim()
.notEmpty()
.withMessage('Comment cannot be empty')
.isLength({ min: 1, max: 1000 })
.withMessage('Comment must be between 1 and 1000 characters')
.customSanitizer(value => sanitizeComment(value)),
body('guest_name')
.optional()
.trim()
.isLength({ max: 100 })
.withMessage('Name must be less than 100 characters')
.matches(/^[a-zA-Z0-9\s\-'.]+$/)
.withMessage('Name contains invalid characters'),
body('guest_email')
.optional()
.trim()
.isEmail()
.normalizeEmail()
.withMessage('Invalid email address')
];
/**
* Validation for feedback settings
*/
const validateFeedbackSettings = [
body('feedback_enabled').optional().isBoolean(),
body('allow_ratings').optional().isBoolean(),
body('allow_likes').optional().isBoolean(),
body('allow_comments').optional().isBoolean(),
body('allow_favorites').optional().isBoolean(),
body('require_name_email').optional().isBoolean(),
body('moderate_comments').optional().isBoolean(),
body('show_feedback_to_guests').optional().isBoolean()
];
/**
* Validation for word filters
*/
const validateWordFilter = [
body('word')
.trim()
.notEmpty()
.withMessage('Word cannot be empty')
.isLength({ min: 2, max: 100 })
.withMessage('Word must be between 2 and 100 characters'),
body('severity')
.optional()
.isIn(['mild', 'moderate', 'severe'])
.withMessage('Invalid severity level')
];
/**
* Check validation results middleware
*/
const checkValidation = (req, res, next) => {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({
error: 'Validation failed',
errors: errors.array()
});
}
next();
};
/**
* Validate guest identity requirements
*/
async function validateGuestRequirements(settings, guestData) {
if (!settings.require_name_email) {
return { valid: true };
}
const errors = [];
if (!guestData.guest_name || guestData.guest_name.trim().length === 0) {
errors.push('Name is required');
}
if (!guestData.guest_email || !validator.isEmail(guestData.guest_email)) {
errors.push('Valid email is required');
}
if (errors.length > 0) {
return {
valid: false,
errors
};
}
return { valid: true };
}
module.exports = {
feedbackValidationRules,
validateFeedbackType,
validatePhotoId,
validateEventId,
validateFeedbackSubmission,
validateFeedbackSettings,
validateWordFilter,
checkValidation,
getValidationRules,
sanitizeComment,
validateGuestRequirements
};
-40
View File
@@ -1,40 +0,0 @@
/**
* Formatters for email content and other text transformations
*/
/**
* Convert plain text line breaks to HTML line breaks
* @param {string} text - The text to format
* @returns {string} - Text with HTML line breaks
*/
function nl2br(text) {
if (!text) return '';
// Normalize line endings
text = text.replace(/\r\n/g, '\n').replace(/\r/g, '\n');
// Convert newlines to <br> tags
return text
.split('\n')
.map(line => line.trim())
.filter(line => line.length > 0)
.join('<br />');
}
/**
* Format welcome message for email templates
* @param {string} message - The welcome message
* @returns {string} - Formatted message for HTML emails
*/
function formatWelcomeMessage(message) {
if (!message || message.trim() === '') {
return '';
}
return nl2br(message);
}
module.exports = {
nl2br,
formatWelcomeMessage
};
+10 -77
View File
@@ -1,98 +1,31 @@
const winston = require('winston');
const path = require('path');
const fs = require('fs');
// Ensure logs directory exists
const logDir = path.join(__dirname, '../../logs');
if (!fs.existsSync(logDir)) {
fs.mkdirSync(logDir, { recursive: true });
}
// Custom format for production logs
const productionFormat = winston.format.combine(
winston.format.timestamp({ format: 'YYYY-MM-DD HH:mm:ss.SSS' }),
winston.format.errors({ stack: true }),
winston.format.json(),
winston.format.printf(info => {
// Ensure all security events are properly formatted
if (info.level === 'warn' && (info.message.includes('rate limit') ||
info.message.includes('auth') ||
info.message.includes('login') ||
info.message.includes('JWT'))) {
return JSON.stringify({
timestamp: info.timestamp,
level: info.level,
message: info.message,
security: true,
...info
});
}
return JSON.stringify(info);
})
);
const logger = winston.createLogger({
level: process.env.LOG_LEVEL || 'info',
format: productionFormat,
format: winston.format.combine(
winston.format.timestamp(),
winston.format.errors({ stack: true }),
winston.format.json()
),
transports: [
new winston.transports.File({
filename: path.join(logDir, 'error.log'),
level: 'error',
maxsize: 10 * 1024 * 1024, // 10MB
maxFiles: 5,
tailable: true
filename: path.join(__dirname, '../../logs/error.log'),
level: 'error'
}),
new winston.transports.File({
filename: path.join(logDir, 'combined.log'),
maxsize: 50 * 1024 * 1024, // 50MB
maxFiles: 10,
tailable: true
}),
// Separate security log for authentication and rate limiting
new winston.transports.File({
filename: path.join(logDir, 'security.log'),
level: 'warn',
maxsize: 20 * 1024 * 1024, // 20MB
maxFiles: 10,
tailable: true,
format: winston.format.combine(
winston.format.timestamp({ format: 'YYYY-MM-DD HH:mm:ss.SSS' }),
winston.format.json(),
winston.format.printf(info => {
// Only log security-related warnings
if (info.message.includes('rate limit') ||
info.message.includes('auth') ||
info.message.includes('login') ||
info.message.includes('JWT') ||
info.message.includes('lockout') ||
info.message.includes('suspicious')) {
return JSON.stringify(info);
}
return null;
})
)
filename: path.join(__dirname, '../../logs/combined.log')
})
].filter(Boolean)
]
});
// Add console logging for non-production environments
if (process.env.NODE_ENV !== 'production') {
logger.add(new winston.transports.Console({
format: winston.format.combine(
winston.format.colorize(),
winston.format.timestamp({ format: 'HH:mm:ss' }),
winston.format.printf(info => {
return `[${info.timestamp}] ${info.level}: ${info.message} ${info.stack || ''}`;
})
winston.format.simple()
)
}));
} else {
// In production, also log to console for container environments
if (process.env.LOG_TO_CONSOLE === 'true') {
logger.add(new winston.transports.Console({
format: productionFormat
}));
}
}
module.exports = logger;
+11 -88
View File
@@ -113,84 +113,6 @@ function validatePassword(password, options = {}) {
};
}
/**
* Get complexity settings from database
* @returns {Object} - Password complexity configuration
*/
async function getPasswordComplexitySettings() {
try {
const { db, withRetry } = require('../database/db');
// Use retry wrapper to handle connection failures
const settings = await withRetry(async () => {
return await db('app_settings')
.where('setting_key', 'security_password_complexity_level')
.first();
});
if (!settings || !settings.setting_value) {
return 'moderate'; // Default
}
const value = typeof settings.setting_value === 'string'
? JSON.parse(settings.setting_value)
: settings.setting_value;
return value;
} catch (error) {
logger.error('Failed to get password complexity settings:', error);
return 'moderate'; // Default on error - ensures app continues working
}
}
/**
* Get password configuration based on complexity level
* @param {string} complexityLevel - Complexity level (simple, moderate, strong, very_strong)
* @returns {Object} - Password configuration
*/
function getPasswordConfigForComplexity(complexityLevel) {
const configs = {
simple: {
minLength: 6,
requireUppercase: false,
requireLowercase: false,
requireNumbers: false,
requireSpecialChars: false,
preventCommonPasswords: true,
minStrengthScore: 0
},
moderate: {
minLength: 8,
requireUppercase: true,
requireLowercase: true,
requireNumbers: true,
requireSpecialChars: false,
preventCommonPasswords: true,
minStrengthScore: 2
},
strong: {
minLength: 12,
requireUppercase: true,
requireLowercase: true,
requireNumbers: true,
requireSpecialChars: false,
preventCommonPasswords: true,
minStrengthScore: 3
},
very_strong: {
minLength: 12,
requireUppercase: true,
requireLowercase: true,
requireNumbers: true,
requireSpecialChars: true,
preventCommonPasswords: true,
minStrengthScore: 3
}
};
return configs[complexityLevel] || configs.moderate;
}
/**
* Validate password for specific contexts (admin, gallery)
* @param {string} password - Password to validate
@@ -198,16 +120,19 @@ function getPasswordConfigForComplexity(complexityLevel) {
* @param {Object} userData - Additional user data for context-aware validation
* @returns {Object} - Validation result
*/
async function validatePasswordInContext(password, context, userData = {}) {
// For gallery context, use dynamic complexity settings
function validatePasswordInContext(password, context, userData = {}) {
// For gallery context, use more lenient validation
if (context === 'gallery') {
// Get complexity settings from database
const complexityLevel = await getPasswordComplexitySettings();
// Get configuration for the complexity level
// Gallery-specific validation options
const galleryOptions = {
...getPasswordConfigForComplexity(complexityLevel),
skipStrengthCheck: complexityLevel === 'simple' // Skip zxcvbn for simple passwords
minLength: 6, // Reduced minimum length
requireUppercase: false, // Don't require uppercase for galleries
requireLowercase: false, // Don't require lowercase for galleries
requireNumbers: false, // Numbers are optional
requireSpecialChars: false, // Special chars are optional
preventCommonPasswords: true, // Still prevent common passwords
minStrengthScore: 0, // Accept any score for galleries
skipStrengthCheck: true // Skip zxcvbn strength analysis for galleries
};
// Base validation with gallery-specific options
@@ -356,7 +281,5 @@ module.exports = {
generateSecurePassword,
getBcryptRounds,
logPasswordValidationFailure,
getPasswordComplexitySettings,
getPasswordConfigForComplexity,
PASSWORD_CONFIG
};
+1 -1
View File
@@ -47,7 +47,7 @@ function escapeLikePattern(input) {
.replace(/\\/g, '\\\\') // Escape backslashes first
.replace(/%/g, '\\%') // Escape percent signs
.replace(/_/g, '\\_') // Escape underscores
.replace(/'/g, '\'\''); // Escape single quotes for safety
.replace(/'/g, "''"); // Escape single quotes for safety
}
/**
-478
View File
@@ -1,478 +0,0 @@
# Comprehensive Backup & Restore Implementation Plan
## Introduction
This plan extends the existing backup system to include full database backups, S3/MinIO support, intelligent change detection, and complete restore functionality. The system will create versioned, encrypted backups with manifests for easy restoration while minimizing storage usage through incremental backups and smart scheduling.
---
## Phase 1: Enhanced Database Schema & Core Infrastructure
### Task 1.1: Create Enhanced Database Migration
**File:** `backend/migrations/030_enhance_backup_system.js`
**Purpose:** Add tables for database backups, restore operations, and backup manifests
```sql
-- backup_manifests table
- id (primary key)
- backup_run_id (FK to backup_runs)
- manifest_version (e.g., "1.0.0")
- created_at
- database_dump_path
- database_checksum
- files_manifest (JSON with all file paths/checksums)
- metadata (JSON with system info, versions, etc.)
-- restore_operations table
- id (primary key)
- started_at
- completed_at
- status (pending, running, completed, failed)
- restore_type (full, partial, database_only, files_only)
- source_backup_id (FK to backup_runs)
- restored_by (FK to admin_users)
- error_message
- restore_log (detailed log)
-- backup_change_tracking table
- id (primary key)
- table_name
- last_change_timestamp
- row_count
- checksum
- last_backed_up
```
### Task 1.2: Add S3 Configuration Settings
**File:** Update migration `029_add_backup_service_tables.js`
**Add settings:**
- `backup_s3_use_ssl` (boolean)
- `backup_s3_path_style` (for MinIO compatibility)
- `backup_encryption_enabled` (boolean)
- `backup_encryption_key` (encrypted storage)
- `backup_database_included` (boolean)
- `backup_incremental_enabled` (boolean)
- `backup_versioning_enabled` (boolean)
- `backup_versions_to_keep` (number)
---
## Phase 2: S3/MinIO Implementation
### Task 2.1: Install S3 Dependencies
**File:** `backend/package.json`
**Command:** `npm install @aws-sdk/client-s3 @aws-sdk/lib-storage mime-types`
**Purpose:** AWS SDK v3 for S3-compatible storage
### Task 2.2: Create S3 Storage Adapter
**File:** `backend/src/services/storage/s3Storage.js`
**Implementation:**
```javascript
class S3StorageAdapter {
constructor(config)
connect() // Test connection
uploadFile(localPath, remotePath, metadata)
uploadStream(stream, remotePath, metadata)
downloadFile(remotePath, localPath)
listFiles(prefix)
deleteFile(remotePath)
getSignedUrl(remotePath, expiresIn)
createMultipartUpload(remotePath) // For large files
uploadPart(uploadId, partNumber, data)
completeMultipartUpload(uploadId, parts)
}
```
### Task 2.3: Implement MinIO Compatibility Layer
**File:** `backend/src/services/storage/minioCompat.js`
**Features:**
- Path-style URL handling
- Custom endpoint configuration
- SSL/TLS options
- Bucket creation if not exists
---
## Phase 3: Database Backup Integration
### Task 3.1: Create Database Dump Service
**File:** `backend/src/services/databaseBackup.js`
**Implementation:**
```javascript
class DatabaseBackupService {
async createBackup(format = 'sql') // sql or json
async dumpSQLite(outputPath)
async dumpPostgreSQL(outputPath)
async compressBackup(inputPath, outputPath)
async encryptBackup(inputPath, outputPath, key)
async validateBackup(backupPath)
async getTableChecksums() // For change detection
}
```
### Task 3.2: Implement Change Detection for Database
**File:** `backend/src/services/changeDetection.js`
**Features:**
- Track table modifications using triggers
- Calculate table checksums
- Compare with last backup state
- Intelligent backup decision making
---
## Phase 4: Enhanced Backup Service
### Task 4.1: Refactor Backup Service for S3
**File:** `backend/src/services/backupService.js`
**Modifications:**
- Add `performS3Backup()` implementation
- Support multipart uploads for large files
- Add progress tracking callbacks
- Implement retry logic with exponential backoff
### Task 4.2: Create Backup Manifest Generator
**File:** `backend/src/services/backupManifest.js`
**Structure:**
```javascript
{
version: "1.0.0",
created_at: "2024-01-20T10:00:00Z",
system_info: {
app_version: "1.0.74",
node_version: "18.x",
database_type: "sqlite|postgresql"
},
database: {
dump_file: "database/dump.sql.gz",
checksum: "sha256:...",
tables: { /* table info */ }
},
files: {
count: 1234,
total_size: 5678901234,
entries: [
{
path: "events/active/...",
checksum: "sha256:...",
size: 12345,
modified: "2024-01-20T09:00:00Z"
}
]
},
settings: { /* app settings snapshot */ }
}
```
### Task 4.3: Implement Incremental Backup Logic
**File:** `backend/src/services/incrementalBackup.js`
**Features:**
- Track changed files since last full backup
- Create incremental manifest
- Link to parent backup
- Merge incremental backups
---
## Phase 5: Restore Functionality
### Task 5.1: Create Restore Service
**File:** `backend/src/services/restoreService.js`
**Implementation:**
```javascript
class RestoreService {
async validateBackup(backupId)
async prepareRestore(backupId, options)
async restoreDatabase(manifestPath)
async restoreFiles(manifestPath, options)
async performFullRestore(backupId)
async performPartialRestore(backupId, selections)
async rollbackRestore(restoreId)
async verifyRestore(restoreId)
}
```
### Task 5.2: Implement Safe Restore Process
**File:** `backend/src/services/restoreValidation.js`
**Safety Features:**
- Pre-restore backup creation
- Validation checksums
- Atomic operations
- Rollback capability
- Post-restore verification
### Task 5.3: Create Restore CLI Tool
**File:** `backend/scripts/restore-backup.js`
**Purpose:** Emergency restore without running application
**Features:**
- Interactive mode
- Dry-run option
- Progress display
- Validation reports
---
## Phase 6: Admin API Extensions
### Task 6.1: Add Restore Endpoints
**File:** `backend/src/routes/adminBackup.js`
**New Endpoints:**
```javascript
POST /api/admin/backup/restore/validate
POST /api/admin/backup/restore/start
GET /api/admin/backup/restore/:id/status
POST /api/admin/backup/restore/:id/cancel
GET /api/admin/backup/manifests/:backupId
GET /api/admin/backup/download/:backupId
```
### Task 6.2: Add S3 Management Endpoints
**File:** `backend/src/routes/adminBackup.js`
**New Endpoints:**
```javascript
GET /api/admin/backup/s3/buckets
GET /api/admin/backup/s3/files
DELETE /api/admin/backup/s3/cleanup
POST /api/admin/backup/s3/test-upload
```
---
## Phase 7: Frontend Implementation
### Task 7.1: Create Backup Management Page
**File:** `frontend/src/pages/admin/BackupManagement.jsx`
**Components:**
- Backup configuration form
- Backup history table
- Manual backup trigger
- Restore interface
- Progress indicators
### Task 7.2: Create Backup Status Dashboard
**File:** `frontend/src/components/admin/BackupDashboard.jsx`
**Features:**
- Real-time backup status
- Storage usage charts
- Backup success rate
- Next scheduled backup
- Recent backup/restore operations
### Task 7.3: Implement Restore Wizard
**File:** `frontend/src/components/admin/RestoreWizard.jsx`
**Steps:**
1. Select backup to restore
2. Choose restore type (full/partial)
3. Select components (database/files/settings)
4. Review and confirm
5. Monitor progress
6. Verify results
---
## Phase 8: Background Job Enhancements
### Task 8.1: Implement Smart Scheduling
**File:** `backend/src/services/smartScheduler.js`
**Features:**
- Skip backup if no changes detected
- Adaptive scheduling based on activity
- Priority queuing for critical backups
- Resource usage monitoring
### Task 8.2: Create Backup Monitor Service
**File:** `backend/src/services/backupMonitor.js`
**Purpose:** Monitor backup health and alert on issues
**Features:**
- Check last successful backup age
- Verify backup integrity periodically
- Monitor storage usage
- Alert on failures or anomalies
---
## Phase 9: Security & Encryption
### Task 9.1: Implement Backup Encryption
**File:** `backend/src/utils/encryption.js`
**Features:**
- AES-256-GCM encryption
- Key derivation from master key
- Encrypted manifest headers
- Secure key storage
### Task 9.2: Add Access Control
**File:** `backend/src/middleware/backupAuth.js`
**Features:**
- Separate permissions for backup/restore
- Audit logging for all operations
- IP whitelist for restore operations
- Two-factor authentication for restore
---
## Phase 10: Testing & Validation
### Task 10.1: Create Backup Test Suite
**File:** `backend/__tests__/services/backup.test.js`
**Tests:**
- Unit tests for each backup method
- Integration tests with real S3/MinIO
- Database backup/restore cycles
- Encryption/decryption validation
- Manifest generation and parsing
### Task 10.2: Create Restore Test Suite
**File:** `backend/__tests__/services/restore.test.js`
**Tests:**
- Full restore scenarios
- Partial restore validation
- Rollback testing
- Corruption recovery
- Cross-version compatibility
### Task 10.3: Create E2E Backup/Restore Tests
**File:** `backend/__tests__/e2e/backupRestore.test.js`
**Scenarios:**
- Complete backup/restore cycle
- Disaster recovery simulation
- Performance benchmarks
- Storage optimization validation
---
## Phase 11: Documentation & Deployment
### Task 11.1: Create Backup Administrator Guide
**File:** `docs/backup-admin-guide.md`
**Contents:**
- Configuration guide
- Best practices
- Troubleshooting
- Recovery procedures
- Performance tuning
### Task 11.2: Update Docker Configuration
**Files:** `docker-compose.yml`, `Dockerfile`
**Changes:**
- Add S3/MinIO service for development
- Volume mappings for backups
- Environment variable templates
- Health checks for backup service
### Task 11.3: Create Backup Playbook
**File:** `docs/backup-playbook.md`
**Scenarios:**
- Daily backup verification
- Disaster recovery steps
- Migration procedures
- Troubleshooting flowchart
---
## Implementation Order & Priority
### Critical Path (Must Have):
1. S3 Storage Adapter (Task 2.2)
2. Database Backup Service (Task 3.1)
3. Enhanced Backup Service (Task 4.1)
4. Basic Restore Service (Task 5.1)
5. Admin API Extensions (Task 6.1)
6. Backup Test Suite (Task 10.1)
### High Priority (Should Have):
1. Backup Manifest Generator (Task 4.2)
2. Change Detection (Task 3.2)
3. Frontend Backup Page (Task 7.1)
4. Encryption Implementation (Task 9.1)
5. Restore Wizard (Task 7.3)
### Nice to Have:
1. Incremental Backups (Task 4.3)
2. Smart Scheduling (Task 8.1)
3. Advanced Monitoring (Task 8.2)
4. MinIO Compatibility (Task 2.3)
---
## Configuration Examples
### S3 Configuration:
```javascript
{
backup_destination_type: "s3",
backup_s3_endpoint: "https://s3.amazonaws.com",
backup_s3_bucket: "wedding-backups",
backup_s3_access_key: "AKIA...",
backup_s3_secret_key: "secret",
backup_s3_region: "us-east-1",
backup_s3_use_ssl: true,
backup_s3_path_style: false
}
```
### MinIO Configuration:
```javascript
{
backup_destination_type: "s3",
backup_s3_endpoint: "https://minio.example.com:9000",
backup_s3_bucket: "picpeak-backups",
backup_s3_access_key: "minioadmin",
backup_s3_secret_key: "minioadmin",
backup_s3_region: "us-east-1",
backup_s3_use_ssl: true,
backup_s3_path_style: true // Required for MinIO
}
```
---
## Best Practices Implementation
1. **Change Detection**: Use database triggers and file checksums to detect changes
2. **Compression**: Always compress before encryption for better ratios
3. **Chunking**: Split large backups into manageable chunks
4. **Versioning**: Keep multiple backup versions with rotation
5. **Validation**: Verify every backup immediately after creation
6. **Monitoring**: Alert on backup failures within 5 minutes
7. **Testing**: Perform monthly restore drills
8. **Documentation**: Log every backup/restore operation with details
---
## Key Features of This Implementation
### Intelligent Change Detection
- Only backs up when changes are detected
- Tracks database modifications via checksums
- Monitors file system changes
- Reduces unnecessary backup operations
### Comprehensive Backup Scope
- Full database dumps (SQLite/PostgreSQL)
- All event photos and thumbnails
- Application settings and configuration
- Email templates and user data
- Complete system state capture
### Flexible Storage Options
- Local directory backup
- Remote server via rsync
- S3-compatible storage (AWS, MinIO, etc.)
- Encrypted storage for security
- Compression for space efficiency
### Robust Restore Capabilities
- Full system restore
- Partial restore (specific events/data)
- Point-in-time recovery
- Pre-restore validation
- Rollback on failure
### Enterprise-Grade Features
- Backup manifests for verification
- Incremental backup support
- Version retention policies
- Automated cleanup of old backups
- Comprehensive audit logging
This comprehensive plan provides a robust, enterprise-grade backup solution with full disaster recovery capabilities.
-163
View File
@@ -1,163 +0,0 @@
#!/bin/bash
# Script to clean git history - removes all commits before July 17, 2025
# Safe version: Moves current main to main-old and creates new clean main
# WARNING: This will rewrite history!
set -e
echo "⚠️ WARNING: This script will rewrite git history!"
echo "⚠️ All commits before July 17, 2025 will be removed."
echo "⚠️ Current main branch will be preserved as main-old"
echo ""
read -p "Are you sure you want to continue? (type 'yes' to proceed): " confirmation
if [ "$confirmation" != "yes" ]; then
echo "Operation cancelled."
exit 1
fi
# Check current state
CURRENT_BRANCH=$(git branch --show-current)
echo "Current branch: $CURRENT_BRANCH"
# Check if we're in the middle of a cherry-pick or rebase
if [ -d ".git/CHERRY_PICK_HEAD" ] || [ -d ".git/rebase-merge" ] || [ -d ".git/rebase-apply" ]; then
echo "ERROR: You're in the middle of a cherry-pick or rebase. Please resolve or abort it first."
echo "To abort cherry-pick: git cherry-pick --abort"
echo "To abort rebase: git rebase --abort"
exit 1
fi
# Clean any previous attempts
echo "Cleaning up any previous attempts..."
git cherry-pick --abort 2>/dev/null || true
git rebase --abort 2>/dev/null || true
# Check if main-old already exists
if git show-ref --verify --quiet refs/heads/main-old; then
echo ""
echo "⚠️ Branch 'main-old' already exists!"
echo "Options:"
echo "1. Delete it and continue (previous backup will be lost)"
echo "2. Rename it with timestamp and continue"
echo "3. Cancel operation"
read -p "Choose option (1/2/3): " option
case $option in
1)
echo "Deleting existing main-old branch..."
git branch -D main-old
;;
2)
TIMESTAMP=$(date +%Y%m%d-%H%M%S)
NEW_NAME="main-old-${TIMESTAMP}"
echo "Renaming existing main-old to ${NEW_NAME}..."
git branch -m main-old "${NEW_NAME}"
;;
3)
echo "Operation cancelled."
exit 1
;;
*)
echo "Invalid option. Operation cancelled."
exit 1
;;
esac
fi
# Make sure we're on main branch
if [ "$CURRENT_BRANCH" != "main" ]; then
echo "Switching to main branch..."
git checkout main
fi
# Move current main to main-old
echo "Moving current main branch to main-old..."
git branch -m main main-old
# Find the first commit on or after July 17, 2025
echo "Finding first commit after July 17, 2025..."
FIRST_COMMIT=$(git log --since="2025-07-17" --reverse --format="%H" | head -1)
if [ -z "$FIRST_COMMIT" ]; then
echo "ERROR: No commits found after July 17, 2025"
# Restore main branch
git branch -m main-old main
exit 1
fi
echo "First commit to keep: $FIRST_COMMIT"
echo "Commit details: $(git log --oneline -1 $FIRST_COMMIT)"
# Count total commits to process
TOTAL_COMMITS=$(git log --since="2025-07-17" --oneline | wc -l)
echo "Total commits to preserve: $TOTAL_COMMITS"
# Create new orphan branch for clean main
echo "Creating new clean main branch..."
git checkout --orphan main
# Clean the working directory
git rm -rf . || true
# Get the tree from the first commit
git checkout $FIRST_COMMIT -- .
# Create new initial commit with same content
ORIGINAL_MESSAGE=$(git log --format="%B" -1 $FIRST_COMMIT)
ORIGINAL_AUTHOR=$(git log --format="%an <%ae>" -1 $FIRST_COMMIT)
ORIGINAL_DATE=$(git log --format="%ad" -1 $FIRST_COMMIT)
GIT_AUTHOR_NAME=$(echo "$ORIGINAL_AUTHOR" | cut -d'<' -f1 | xargs)
GIT_AUTHOR_EMAIL=$(echo "$ORIGINAL_AUTHOR" | cut -d'<' -f2 | cut -d'>' -f1)
GIT_AUTHOR_DATE="$ORIGINAL_DATE"
export GIT_AUTHOR_NAME GIT_AUTHOR_EMAIL GIT_AUTHOR_DATE
git add -A
git commit -m "Initial commit - Project start (July 17, 2025)
Original: $ORIGINAL_MESSAGE"
# Now rebase the rest of the history onto the new main
echo "Rebasing remaining commits..."
echo "This will linearize the history (merge commits will be flattened)..."
# Use rebase to apply all commits
git rebase --onto main $FIRST_COMMIT main-old || {
echo ""
echo "⚠️ Rebase encountered conflicts!"
echo ""
echo "To resolve:"
echo "1. Fix the conflicts in the listed files"
echo "2. Stage the resolved files: git add <files>"
echo "3. Continue rebase: git rebase --continue"
echo "4. If you want to abort and restore: git rebase --abort && git branch -D main && git branch -m main-old main"
echo ""
echo "After successful rebase, your main branch will have the clean history."
echo "The old history is preserved in main-old branch."
exit 1
}
# If we get here, rebase was successful
echo ""
echo "✅ New clean history created successfully!"
echo "Total commits in new history: $(git rev-list --count HEAD)"
echo ""
echo "Branch status:"
echo " - main: Clean history starting from July 17, 2025"
echo " - main-old: Original history with all commits"
echo ""
echo "To push the new history to remote, run:"
echo " git push origin main --force"
echo ""
echo "To also push the old history backup:"
echo " git push origin main-old"
echo ""
echo "⚠️ WARNING: Force pushing will overwrite the remote repository!"
echo "⚠️ Make sure all team members are aware before pushing!"
echo ""
echo "If you need to restore the original history:"
echo " git checkout main-old"
echo " git branch -D main"
echo " git branch -m main"
-123
View File
@@ -1,123 +0,0 @@
#!/bin/bash
# Script to clean git history - removes all commits before July 17, 2025
# Version 2: Handles merge commits properly
# WARNING: This is destructive and will rewrite history!
set -e
echo "⚠️ WARNING: This script will permanently rewrite git history!"
echo "⚠️ All commits before July 17, 2025 will be removed."
echo "⚠️ This action cannot be undone!"
echo ""
read -p "Are you sure you want to continue? (type 'yes' to proceed): " confirmation
if [ "$confirmation" != "yes" ]; then
echo "Operation cancelled."
exit 1
fi
# Check current state
CURRENT_BRANCH=$(git branch --show-current)
echo "Current branch: $CURRENT_BRANCH"
# Check if we're in the middle of a cherry-pick
if [ -d ".git/CHERRY_PICK_HEAD" ]; then
echo "ERROR: You're in the middle of a cherry-pick. Please resolve or abort it first."
echo "To abort: git cherry-pick --abort"
echo "To continue: git cherry-pick --continue"
exit 1
fi
# Clean any previous attempts
echo "Cleaning up any previous attempts..."
git cherry-pick --abort 2>/dev/null || true
git checkout main 2>/dev/null || true
git branch -D new-main 2>/dev/null || true
# Create backup branch
echo "Creating backup branch..."
BACKUP_BRANCH="backup-before-cleanup-$(date +%Y%m%d-%H%M%S)"
git checkout -b "$BACKUP_BRANCH"
git checkout main
# Find the first commit on or after July 17, 2025
echo "Finding first commit after July 17, 2025..."
FIRST_COMMIT=$(git log --since="2025-07-17" --reverse --format="%H" | head -1)
if [ -z "$FIRST_COMMIT" ]; then
echo "ERROR: No commits found after July 17, 2025"
exit 1
fi
echo "First commit to keep: $FIRST_COMMIT"
echo "Commit details: $(git log --oneline -1 $FIRST_COMMIT)"
# Count total commits to process
TOTAL_COMMITS=$(git log --since="2025-07-17" --oneline | wc -l)
echo "Total commits to preserve: $TOTAL_COMMITS"
# Create new orphan branch
echo "Creating new clean history..."
git checkout --orphan new-main
# Clean the working directory
git rm -rf . || true
# Get the tree from the first commit
git checkout $FIRST_COMMIT -- .
# Create new initial commit with same content
ORIGINAL_MESSAGE=$(git log --format="%B" -1 $FIRST_COMMIT)
ORIGINAL_AUTHOR=$(git log --format="%an <%ae>" -1 $FIRST_COMMIT)
ORIGINAL_DATE=$(git log --format="%ad" -1 $FIRST_COMMIT)
GIT_AUTHOR_NAME=$(echo "$ORIGINAL_AUTHOR" | cut -d'<' -f1 | xargs)
GIT_AUTHOR_EMAIL=$(echo "$ORIGINAL_AUTHOR" | cut -d'<' -f2 | cut -d'>' -f1)
GIT_AUTHOR_DATE="$ORIGINAL_DATE"
export GIT_AUTHOR_NAME GIT_AUTHOR_EMAIL GIT_AUTHOR_DATE
git add -A
git commit -m "Initial commit - Project start (July 17, 2025)
Original: $ORIGINAL_MESSAGE"
# Now we'll use git rebase instead of cherry-pick to handle merges better
echo "Rebasing remaining commits..."
echo "This will linearize the history (merge commits will be flattened)..."
# Get the commit range
LAST_COMMIT=$(git rev-parse main)
# Use rebase to apply all commits
git rebase --onto new-main $FIRST_COMMIT main || {
echo ""
echo "⚠️ Rebase encountered conflicts!"
echo ""
echo "To resolve:"
echo "1. Fix the conflicts in the listed files"
echo "2. Stage the resolved files: git add <files>"
echo "3. Continue rebase: git rebase --continue"
echo "4. If you want to abort: git rebase --abort"
echo ""
echo "After successful rebase, run:"
echo " git branch -D main"
echo " git branch -m main"
echo " git push origin main --force"
exit 1
}
# If we get here, rebase was successful
echo ""
echo "✅ New history created successfully!"
echo "Total commits in new history: $(git rev-list --count HEAD)"
echo ""
echo "Your backup branch is: $BACKUP_BRANCH"
echo ""
echo "To finalize the cleanup, run these commands:"
echo " git branch -D main"
echo " git branch -m main"
echo " git push origin main --force"
echo ""
echo "⚠️ WARNING: Force pushing will overwrite the remote repository!"
echo "⚠️ Make sure you have a backup and all team members are aware!"
-91
View File
@@ -1,91 +0,0 @@
#!/bin/bash
# Script to clean git history - removes all commits before July 17, 2025
# WARNING: This is destructive and will rewrite history!
set -e
echo "⚠️ WARNING: This script will permanently rewrite git history!"
echo "⚠️ All commits before July 17, 2025 will be removed."
echo "⚠️ This action cannot be undone!"
echo ""
read -p "Are you sure you want to continue? (type 'yes' to proceed): " confirmation
if [ "$confirmation" != "yes" ]; then
echo "Operation cancelled."
exit 1
fi
# Create backup branch
echo "Creating backup branch..."
git checkout -b backup-before-cleanup-$(date +%Y%m%d-%H%M%S)
git checkout main
# Find the first commit on or after July 17, 2025
echo "Finding first commit after July 17, 2025..."
FIRST_COMMIT=$(git log --since="2025-07-17" --reverse --format="%H" | head -1)
if [ -z "$FIRST_COMMIT" ]; then
echo "ERROR: No commits found after July 17, 2025"
exit 1
fi
echo "First commit to keep: $FIRST_COMMIT"
echo "Commit details: $(git log --oneline -1 $FIRST_COMMIT)"
# Get all commits we want to keep
COMMITS_TO_KEEP=$(git log --since="2025-07-17" --reverse --format="%H")
COMMIT_COUNT=$(echo "$COMMITS_TO_KEEP" | wc -l)
echo "Total commits to preserve: $COMMIT_COUNT"
# Create new orphan branch
echo "Creating new clean history..."
git checkout --orphan new-main
# Clean the working directory
git rm -rf . || true
# Get the tree from the first commit
git checkout $FIRST_COMMIT -- .
# Create new initial commit with same content but new message
ORIGINAL_MESSAGE=$(git log --format="%B" -1 $FIRST_COMMIT)
ORIGINAL_AUTHOR=$(git log --format="%an <%ae>" -1 $FIRST_COMMIT)
ORIGINAL_DATE=$(git log --format="%ad" -1 $FIRST_COMMIT)
GIT_AUTHOR_NAME=$(echo "$ORIGINAL_AUTHOR" | cut -d'<' -f1 | xargs)
GIT_AUTHOR_EMAIL=$(echo "$ORIGINAL_AUTHOR" | cut -d'<' -f2 | cut -d'>' -f1)
GIT_AUTHOR_DATE="$ORIGINAL_DATE"
export GIT_AUTHOR_NAME GIT_AUTHOR_EMAIL GIT_AUTHOR_DATE
git add -A
git commit -m "Initial commit - Project start (July 17, 2025)
Original: $ORIGINAL_MESSAGE"
# Cherry-pick remaining commits
echo "Applying remaining commits..."
REMAINING_COMMITS=$(git log --since="2025-07-17" --reverse --format="%H" $FIRST_COMMIT..main)
if [ -n "$REMAINING_COMMITS" ]; then
for commit in $REMAINING_COMMITS; do
echo "Applying: $(git log --oneline -1 $commit)"
git cherry-pick $commit || {
echo "ERROR: Failed to cherry-pick $commit"
echo "You may need to resolve conflicts and continue manually"
exit 1
}
done
fi
echo ""
echo "✅ New history created successfully!"
echo "Total commits in new history: $(git rev-list --count HEAD)"
echo ""
echo "To finalize the cleanup, run these commands:"
echo " git branch -D main"
echo " git branch -m main"
echo " git push origin main --force"
echo ""
echo "⚠️ WARNING: Force pushing will overwrite the remote repository!"
echo "⚠️ Make sure you have a backup and all team members are aware!"
+8 -13
View File
@@ -1,16 +1,11 @@
# Backend API URL
# For local development:
VITE_API_URL=http://localhost:3001
VITE_API_URL=http://localhost:3000
# For production behind reverse proxy (Traefik, nginx, etc):
# VITE_API_URL=/api
# Umami Analytics Configuration
# Get these values from your Umami installation
VITE_UMAMI_URL=https://analytics.yourdomain.com
VITE_UMAMI_WEBSITE_ID=your-website-id-from-umami
# Umami Analytics Configuration (OPTIONAL - Fallback only)
# NOTE: Primary Umami configuration should be done through Admin UI > Settings > Analytics
# These environment variables serve as fallbacks when backend settings are not available
# Useful for: development environments, initial setup, or when backend is unavailable
#
# Example values:
# VITE_UMAMI_URL=https://analytics.example.com
# VITE_UMAMI_WEBSITE_ID=abc123def-4567-89ab-cdef-0123456789ab
# VITE_UMAMI_SHARE_URL=https://analytics.example.com/share/xyz789/wedding-photos
# Optional: Umami share URL for embedding full dashboard
# This is the public share URL from Umami's share feature
VITE_UMAMI_SHARE_URL=https://analytics.yourdomain.com/share/your-share-id/photo-sharing
+4 -8
View File
@@ -8,11 +8,7 @@ VITE_API_URL=/api
# For development or if frontend/backend are on different domains:
# VITE_API_URL=https://api.yourdomain.com
# Umami Analytics Configuration (OPTIONAL - Fallback only)
# NOTE: Primary Umami configuration should be done through Admin UI > Settings > Analytics
# These environment variables serve as fallbacks when backend settings are not available
#
# Real-world example values:
# VITE_UMAMI_URL=https://analytics.picpeak.com
# VITE_UMAMI_WEBSITE_ID=b4d3c2a1-5678-90ab-cdef-1234567890ab
# VITE_UMAMI_SHARE_URL=https://analytics.picpeak.com/share/Ab3Cd5Fg/picpeak-gallery
# Umami Analytics Configuration (optional)
# VITE_UMAMI_URL=https://analytics.yourdomain.com
# VITE_UMAMI_WEBSITE_ID=your-website-id-from-umami
# VITE_UMAMI_SHARE_URL=https://analytics.yourdomain.com/share/your-share-id/photo-sharing
+451 -591
View File
File diff suppressed because it is too large Load Diff
+4 -12
View File
@@ -1,7 +1,7 @@
{
"name": "picpeak-frontend",
"private": true,
"version": "1.0.93",
"version": "1.0.57",
"type": "module",
"scripts": {
"dev": "vite",
@@ -12,28 +12,20 @@
},
"dependencies": {
"@tanstack/react-query": "^5.0.0",
"@tiptap/extension-character-count": "^2.26.1",
"@tiptap/extension-code-block-lowlight": "^2.26.1",
"@tiptap/extension-hard-break": "^2.26.1",
"@tiptap/extension-link": "^2.25.0",
"@tiptap/extension-placeholder": "^2.26.1",
"@tiptap/extension-text-align": "^2.26.1",
"@tiptap/react": "^2.25.0",
"@tiptap/starter-kit": "^2.25.0",
"@types/dompurify": "^3.0.5",
"@types/lodash": "^4.17.20",
"@types/react-google-recaptcha": "^2.1.9",
"axios": "^1.3.2",
"clsx": "^2.0.0",
"date-fns": "4.1.0",
"date-fns": "^2.29.3",
"dompurify": "^3.2.6",
"i18next": "^25.3.1",
"i18next-browser-languagedetector": "^8.2.0",
"i18next-http-backend": "^3.0.2",
"js-cookie": "^3.0.5",
"lodash": "^4.17.21",
"lowlight": "^2.9.0",
"lucide-react": "0.525.0",
"lucide-react": "^0.292.0",
"react": "^18.3.1",
"react-countdown": "^2.3.5",
"react-dom": "^18.3.1",
@@ -42,7 +34,7 @@
"react-image-gallery": "^1.2.11",
"react-intersection-observer": "^9.4.3",
"react-router-dom": "^6.8.0",
"react-toastify": "11.0.5",
"react-toastify": "^9.1.1",
"tailwind-merge": "^3.3.1"
},
"devDependencies": {
+15 -34
View File
@@ -16,16 +16,13 @@ import {
EventsListPage,
CreateEventPageEnhanced as CreateEventPage,
EventDetailsPage,
EventFeedbackPage,
EmailConfigPage,
ArchivesPage,
AnalyticsPage,
BrandingPage,
SettingsPage,
CMSPage,
BackupManagement
CMSPage
} from './pages/admin';
import { CMSPageEnhanced } from './pages/admin/CMSPageEnhanced';
import { AdminLayout, AdminAuthWrapper } from './components/admin';
import { PageErrorBoundary, OfflineIndicator, SkipLink, DynamicFavicon } from './components/common';
import { MaintenanceWrapper } from './components/MaintenanceWrapper';
@@ -46,26 +43,17 @@ function App() {
// Initialize Umami Analytics based on settings
useEffect(() => {
const initializeAnalytics = async () => {
try {
// Fetch public settings to get Umami configuration
const response = await fetch(`${getApiBaseUrl()}/public/settings`);
const settings = await response.json();
// Check if Umami is enabled and configured in backend settings
if (settings.umami_enabled && settings.umami_url && settings.umami_website_id) {
// Use backend configuration
analyticsService.initialize({
websiteId: settings.umami_website_id,
hostUrl: settings.umami_url,
autoTrack: true,
doNotTrack: true
});
} else {
// Fall back to environment variables if backend not configured
const umamiUrl = import.meta.env.VITE_UMAMI_URL;
const umamiWebsiteId = import.meta.env.VITE_UMAMI_WEBSITE_ID;
const umamiUrl = import.meta.env.VITE_UMAMI_URL;
const umamiWebsiteId = import.meta.env.VITE_UMAMI_WEBSITE_ID;
if (umamiUrl && umamiWebsiteId) {
try {
// Fetch public settings to check if analytics is enabled
const response = await fetch(`${getApiBaseUrl()}/public/settings`);
const settings = await response.json();
if (umamiUrl && umamiWebsiteId && settings.enable_analytics !== false) {
// Only initialize if analytics is enabled in settings
if (settings.enable_analytics !== false) {
analyticsService.initialize({
websiteId: umamiWebsiteId,
hostUrl: umamiUrl,
@@ -73,14 +61,9 @@ function App() {
doNotTrack: true
});
}
}
} catch (error) {
console.error('Failed to fetch settings for analytics:', error);
// Fall back to environment variables on error
const umamiUrl = import.meta.env.VITE_UMAMI_URL;
const umamiWebsiteId = import.meta.env.VITE_UMAMI_WEBSITE_ID;
if (umamiUrl && umamiWebsiteId) {
} catch (error) {
console.error('Failed to fetch settings for analytics:', error);
// Initialize analytics anyway if settings fetch fails
analyticsService.initialize({
websiteId: umamiWebsiteId,
hostUrl: umamiUrl,
@@ -121,14 +104,12 @@ function App() {
<Route path="events" element={<EventsListPage />} />
<Route path="events/new" element={<CreateEventPage />} />
<Route path="events/:id" element={<EventDetailsPage />} />
<Route path="events/:id/feedback" element={<EventFeedbackPage />} />
<Route path="archives" element={<ArchivesPage />} />
<Route path="email" element={<EmailConfigPage />} />
<Route path="analytics" element={<AnalyticsPage />} />
<Route path="branding" element={<BrandingPage />} />
<Route path="settings" element={<SettingsPage />} />
<Route path="backup" element={<BackupManagement />} />
<Route path="cms" element={<CMSPageEnhanced />} />
<Route path="cms" element={<CMSPage />} />
<Route index element={<Navigate to="/admin/dashboard" replace />} />
</Route>
</Route>
@@ -36,7 +36,23 @@ export const AdminAuthenticatedImage: React.FC<AdminAuthenticatedImageProps> = (
setLoading(false);
}
} catch (err: any) {
// Image loading failed - handled by error state
console.error('Failed to load image:', src, err);
// Log more details about the error
if (err.response) {
console.error('Response status:', err.response.status);
console.error('Response headers:', err.response.headers);
if (err.response.data instanceof Blob) {
// Try to read error message from blob
try {
const text = await err.response.data.text();
console.error('Response data:', text);
} catch (e) {
console.error('Could not read blob data');
}
} else {
console.error('Response data:', err.response.data);
}
}
if (!cancelled) {
setError(true);
setLoading(false);
@@ -23,7 +23,7 @@ export const AdminPhotoGrid: React.FC<AdminPhotoGridProps> = ({
const [selectedPhotos, setSelectedPhotos] = useState<Set<number>>(new Set());
const [isSelectionMode, setIsSelectionMode] = useState(false);
const [isDeleting, setIsDeleting] = useState(false);
const [deletingPhotos, setDeletingPhotos] = useState<Set<number>>(new Set());
const [deletingPhotoId, setDeletingPhotoId] = useState<number | null>(null);
const handlePhotoSelect = (photoId: number, e?: React.MouseEvent) => {
if (e) {
@@ -54,18 +54,15 @@ export const AdminPhotoGrid: React.FC<AdminPhotoGridProps> = ({
return;
}
setDeletingPhotos(prev => new Set(prev).add(photo.id));
setDeletingPhotoId(photo.id);
try {
await photosService.deletePhoto(eventId, photo.id);
toast.success('Photo deleted successfully');
onPhotosDeleted();
} catch (error) {
toast.error('Failed to delete photo');
setDeletingPhotos(prev => {
const newSet = new Set(prev);
newSet.delete(photo.id);
return newSet;
});
} finally {
setDeletingPhotoId(null);
}
};
@@ -78,18 +75,14 @@ export const AdminPhotoGrid: React.FC<AdminPhotoGridProps> = ({
}
setIsDeleting(true);
const selectedIds = Array.from(selectedPhotos);
setDeletingPhotos(new Set(selectedIds));
try {
await photosService.deletePhotos(eventId, selectedIds);
await photosService.deletePhotos(eventId, Array.from(selectedPhotos));
toast.success(`${count} photo${count > 1 ? 's' : ''} deleted successfully`);
setSelectedPhotos(new Set());
setIsSelectionMode(false);
onPhotosDeleted();
} catch (error) {
toast.error('Failed to delete photos');
setDeletingPhotos(new Set());
} finally {
setIsDeleting(false);
}
@@ -162,15 +155,13 @@ export const AdminPhotoGrid: React.FC<AdminPhotoGridProps> = ({
{/* Photo Grid */}
<div className="grid grid-cols-2 md:grid-cols-3 lg:grid-cols-4 xl:grid-cols-5 gap-4">
{photos.map((photo, index) => {
const isDeleting = deletingPhotos.has(photo.id);
return (
<div
key={photo.id}
className={`relative group cursor-pointer rounded-lg overflow-hidden bg-neutral-100 transition-opacity ${
isSelectionMode ? 'ring-2 ring-offset-2 ' + (selectedPhotos.has(photo.id) ? 'ring-primary-500' : 'ring-transparent') : ''
} ${isDeleting ? 'opacity-50' : ''}`}
onClick={() => !isDeleting && (isSelectionMode ? handlePhotoSelect(photo.id) : onPhotoClick(photo, index))}
{photos.map((photo, index) => (
<div
key={photo.id}
className={`relative group cursor-pointer rounded-lg overflow-hidden bg-neutral-100 ${
isSelectionMode ? 'ring-2 ring-offset-2 ' + (selectedPhotos.has(photo.id) ? 'ring-primary-500' : 'ring-transparent') : ''
}`}
onClick={() => isSelectionMode ? handlePhotoSelect(photo.id) : onPhotoClick(photo, index)}
>
{/* Selection Checkbox */}
{isSelectionMode && (
@@ -228,8 +219,8 @@ export const AdminPhotoGrid: React.FC<AdminPhotoGridProps> = ({
</button>
<button
onClick={(e) => handleDeleteSingle(photo, e)}
className="p-1 text-white hover:bg-white/20 rounded disabled:opacity-50"
disabled={isDeleting}
className="p-1 text-white hover:bg-white/20 rounded"
disabled={deletingPhotoId === photo.id}
>
<Trash2 className="w-3 h-3" />
</button>
@@ -247,8 +238,7 @@ export const AdminPhotoGrid: React.FC<AdminPhotoGridProps> = ({
</div>
)}
</div>
);
})}
))}
</div>
{photos.length === 0 && (
@@ -9,8 +9,7 @@ import {
Settings,
X,
Palette,
FileText,
HardDrive
FileText
} from 'lucide-react';
import { useQuery } from '@tanstack/react-query';
import { useTranslation } from 'react-i18next';
@@ -36,7 +35,6 @@ const navigation: NavItem[] = [
{ nameKey: 'navigation.emailSettings', href: '/admin/email', icon: Mail },
{ nameKey: 'navigation.branding', href: '/admin/branding', icon: Palette },
{ nameKey: 'navigation.settings', href: '/admin/settings', icon: Settings },
{ nameKey: 'navigation.backup', href: '/admin/backup', icon: HardDrive },
{ nameKey: 'navigation.cmsPages', href: '/admin/cms', icon: FileText },
];
@@ -1,611 +0,0 @@
import React, { useState, useEffect } from 'react';
import { useTranslation } from 'react-i18next';
import {
Save,
Server,
Cloud,
HardDrive,
Clock,
Calendar,
Shield,
AlertCircle,
Info,
Eye,
EyeOff,
Wifi,
CheckCircle,
XCircle,
Loader2,
FolderOpen,
Database,
Image,
FileArchive
} from 'lucide-react';
import { toast } from 'react-toastify';
import { Button, Card, Input } from '../common';
export const BackupConfiguration = ({ config, onSave, isSaving }) => {
const { t } = useTranslation();
const destinationTypes = [
{
id: 'local',
name: t('backup.configuration.destinationTypes.local.name'),
icon: HardDrive,
description: t('backup.configuration.destinationTypes.local.description'),
fields: ['backup_destination_path']
},
{
id: 'rsync',
name: t('backup.configuration.destinationTypes.rsync.name'),
icon: Server,
description: t('backup.configuration.destinationTypes.rsync.description'),
fields: ['backup_rsync_host', 'backup_rsync_user', 'backup_rsync_path', 'backup_rsync_ssh_key']
},
{
id: 's3',
name: t('backup.configuration.destinationTypes.s3.name'),
icon: Cloud,
description: t('backup.configuration.destinationTypes.s3.description'),
fields: ['backup_s3_endpoint', 'backup_s3_bucket', 'backup_s3_access_key', 'backup_s3_secret_key', 'backup_s3_region']
}
];
const scheduleOptions = [
{ value: 'hourly', label: t('backup.configuration.schedule.options.hourly') },
{ value: 'daily', label: t('backup.configuration.schedule.options.daily') },
{ value: 'weekly', label: t('backup.configuration.schedule.options.weekly') },
{ value: 'custom', label: t('backup.configuration.schedule.options.custom') }
];
const [formData, setFormData] = useState({
backup_enabled: false,
backup_destination_type: 'local',
backup_destination_path: '',
backup_rsync_host: '',
backup_rsync_user: '',
backup_rsync_path: '',
backup_rsync_ssh_key: '',
backup_s3_endpoint: '',
backup_s3_bucket: '',
backup_s3_access_key: '',
backup_s3_secret_key: '',
backup_s3_region: '',
backup_schedule: 'daily',
backup_schedule_cron: '0 3 * * *',
backup_retention_days: 30,
backup_include_database: true,
backup_include_photos: true,
backup_include_archives: true,
backup_include_thumbnails: false,
backup_include_temp: false,
backup_compression: true,
backup_encryption: false,
backup_encryption_passphrase: ''
});
const [showSecrets, setShowSecrets] = useState({
s3_secret_key: false,
ssh_key: false,
encryption_passphrase: false
});
const [testingConnection, setTestingConnection] = useState(false);
useEffect(() => {
if (config) {
setFormData(prev => ({
...prev,
...config
}));
}
}, [config]);
const handleChange = (field, value) => {
setFormData(prev => ({
...prev,
[field]: value
}));
};
const handleSubmit = (e) => {
e.preventDefault();
// Validate required fields
const destinationType = destinationTypes.find(t => t.id === formData.backup_destination_type);
const missingFields = [];
if (formData.backup_enabled && destinationType) {
destinationType.fields.forEach(field => {
if (!formData[field] && !field.includes('optional')) {
missingFields.push(field);
}
});
}
if (missingFields.length > 0) {
toast.error(t('backup.configuration.messages.requiredFields'));
return;
}
onSave(formData);
};
const testConnection = async () => {
setTestingConnection(true);
try {
// TODO: Implement connection test endpoint
await new Promise(resolve => setTimeout(resolve, 2000));
toast.success(t('backup.configuration.messages.connectionSuccess'));
} catch (error) {
toast.error(t('backup.configuration.messages.connectionFailed') + ': ' + error.message);
} finally {
setTestingConnection(false);
}
};
const selectedDestination = destinationTypes.find(t => t.id === formData.backup_destination_type);
return (
<form onSubmit={handleSubmit} className="space-y-6">
{/* Enable/Disable Toggle */}
<Card className="p-6">
<div className="flex items-center justify-between">
<div className="flex-1">
<h3 className="text-lg font-semibold text-gray-900">{t('backup.configuration.enableBackup')}</h3>
<p className="mt-1 text-sm text-gray-600">
{t('backup.configuration.enableBackupHelp')}
</p>
</div>
<label className="relative inline-flex items-center cursor-pointer">
<input
type="checkbox"
checked={formData.backup_enabled}
onChange={(e) => handleChange('backup_enabled', e.target.checked)}
className="sr-only peer"
/>
<div className="w-11 h-6 bg-gray-200 peer-focus:outline-none peer-focus:ring-4 peer-focus:ring-primary-300 rounded-full peer peer-checked:after:translate-x-full peer-checked:after:border-white after:content-[''] after:absolute after:top-[2px] after:left-[2px] after:bg-white after:border-gray-300 after:border after:rounded-full after:h-5 after:w-5 after:transition-all peer-checked:bg-primary"></div>
</label>
</div>
</Card>
{/* Destination Configuration */}
<Card className="p-6">
<h3 className="text-lg font-semibold text-gray-900 mb-4">{t('backup.configuration.destinationType')}</h3>
{/* Destination Type Selection */}
<div className="grid grid-cols-1 md:grid-cols-3 gap-4 mb-6">
{destinationTypes.map((type) => {
const Icon = type.icon;
return (
<button
key={type.id}
type="button"
onClick={() => handleChange('backup_destination_type', type.id)}
className={`p-4 rounded-lg border-2 transition-all ${
formData.backup_destination_type === type.id
? 'border-primary bg-primary-50'
: 'border-gray-200 hover:border-gray-300'
}`}
>
<Icon className={`h-8 w-8 mb-2 mx-auto ${
formData.backup_destination_type === type.id
? 'text-primary'
: 'text-gray-400'
}`} />
<h4 className="font-medium text-gray-900">{type.name}</h4>
<p className="text-xs text-gray-500 mt-1">{type.description}</p>
</button>
);
})}
</div>
{/* Destination-specific fields */}
<div className="space-y-4">
{formData.backup_destination_type === 'local' && (
<>
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
{t('backup.configuration.fields.destinationPath')}
</label>
<Input
type="text"
value={formData.backup_destination_path}
onChange={(e) => handleChange('backup_destination_path', e.target.value)}
placeholder={t('backup.configuration.fields.destinationPathPlaceholder')}
required
/>
<p className="mt-1 text-xs text-gray-500">
{t('backup.configuration.fields.destinationPathHelp')}
</p>
</div>
</>
)}
{formData.backup_destination_type === 'rsync' && (
<>
<div className="grid grid-cols-2 gap-4">
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
{t('backup.configuration.fields.rsyncHost')}
</label>
<Input
type="text"
value={formData.backup_rsync_host}
onChange={(e) => handleChange('backup_rsync_host', e.target.value)}
placeholder={t('backup.configuration.fields.rsyncHostPlaceholder')}
required
/>
</div>
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
{t('backup.configuration.fields.rsyncUser')}
</label>
<Input
type="text"
value={formData.backup_rsync_user}
onChange={(e) => handleChange('backup_rsync_user', e.target.value)}
placeholder={t('backup.configuration.fields.rsyncUserPlaceholder')}
required
/>
</div>
</div>
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
{t('backup.configuration.fields.rsyncPath')}
</label>
<Input
type="text"
value={formData.backup_rsync_path}
onChange={(e) => handleChange('backup_rsync_path', e.target.value)}
placeholder={t('backup.configuration.fields.rsyncPathPlaceholder')}
required
/>
</div>
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
{t('backup.configuration.fields.rsyncSshKey')}
</label>
<div className="relative">
<textarea
value={formData.backup_rsync_ssh_key}
onChange={(e) => handleChange('backup_rsync_ssh_key', e.target.value)}
placeholder={t('backup.configuration.fields.rsyncSshKeyPlaceholder')}
className="w-full px-3 py-2 border border-gray-300 rounded-md focus:outline-none focus:ring-primary focus:border-primary font-mono text-sm"
rows={4}
/>
<button
type="button"
onClick={() => setShowSecrets(prev => ({ ...prev, ssh_key: !prev.ssh_key }))}
className="absolute top-2 right-2 text-gray-400 hover:text-gray-600"
>
{showSecrets.ssh_key ? <EyeOff size={20} /> : <Eye size={20} />}
</button>
</div>
<p className="mt-1 text-xs text-gray-500">
{t('backup.configuration.fields.rsyncSshKeyHelp')}
</p>
</div>
</>
)}
{formData.backup_destination_type === 's3' && (
<>
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
{t('backup.configuration.fields.s3Endpoint')}
</label>
<Input
type="text"
value={formData.backup_s3_endpoint}
onChange={(e) => handleChange('backup_s3_endpoint', e.target.value)}
placeholder="https://s3.amazonaws.com"
required
/>
<p className="mt-1 text-xs text-gray-500">
{t('backup.configuration.fields.s3EndpointHelp')}
</p>
</div>
<div className="grid grid-cols-2 gap-4">
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
{t('backup.configuration.fields.s3Bucket')}
</label>
<Input
type="text"
value={formData.backup_s3_bucket}
onChange={(e) => handleChange('backup_s3_bucket', e.target.value)}
placeholder="my-backup-bucket"
required
/>
</div>
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
{t('backup.configuration.fields.s3Region')}
</label>
<Input
type="text"
value={formData.backup_s3_region}
onChange={(e) => handleChange('backup_s3_region', e.target.value)}
placeholder="us-east-1"
/>
</div>
</div>
<div className="grid grid-cols-2 gap-4">
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
{t('backup.configuration.fields.s3AccessKey')}
</label>
<Input
type="text"
value={formData.backup_s3_access_key}
onChange={(e) => handleChange('backup_s3_access_key', e.target.value)}
placeholder="AKIAIOSFODNN7EXAMPLE"
required
/>
</div>
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
{t('backup.configuration.fields.s3SecretKey')}
</label>
<div className="relative">
<Input
type={showSecrets.s3_secret_key ? 'text' : 'password'}
value={formData.backup_s3_secret_key}
onChange={(e) => handleChange('backup_s3_secret_key', e.target.value)}
placeholder="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
required
/>
<button
type="button"
onClick={() => setShowSecrets(prev => ({ ...prev, s3_secret_key: !prev.s3_secret_key }))}
className="absolute top-1/2 -translate-y-1/2 right-2 text-gray-400 hover:text-gray-600"
>
{showSecrets.s3_secret_key ? <EyeOff size={20} /> : <Eye size={20} />}
</button>
</div>
</div>
</div>
</>
)}
{/* Test Connection Button */}
{formData.backup_destination_type && (
<div className="pt-2">
<Button
type="button"
onClick={testConnection}
disabled={testingConnection}
variant="secondary"
size="sm"
>
{testingConnection ? (
<>
<Loader2 className="mr-2 h-4 w-4 animate-spin" />
{t('backup.configuration.testingConnection')}
</>
) : (
<>
<Wifi className="mr-2 h-4 w-4" />
{t('backup.actions.testConnection')}
</>
)}
</Button>
</div>
)}
</div>
</Card>
{/* Schedule Configuration */}
<Card className="p-6">
<h3 className="text-lg font-semibold text-gray-900 mb-4">{t('backup.configuration.schedule.title')}</h3>
<div className="space-y-4">
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
{t('backup.configuration.schedule.scheduleType')}
</label>
<select
value={formData.backup_schedule}
onChange={(e) => handleChange('backup_schedule', e.target.value)}
className="w-full px-3 py-2 border border-gray-300 rounded-md focus:outline-none focus:ring-primary focus:border-primary"
>
{scheduleOptions.map(option => (
<option key={option.value} value={option.value}>
{option.label}
</option>
))}
</select>
</div>
{formData.backup_schedule === 'custom' && (
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
{t('backup.configuration.schedule.customCron')}
</label>
<Input
type="text"
value={formData.backup_schedule_cron}
onChange={(e) => handleChange('backup_schedule_cron', e.target.value)}
placeholder="0 3 * * *"
/>
<p className="mt-1 text-xs text-gray-500">
{t('backup.configuration.schedule.customCronHelp')}
</p>
</div>
)}
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">
{t('backup.configuration.schedule.retention')}
</label>
<Input
type="number"
value={formData.backup_retention_days}
onChange={(e) => handleChange('backup_retention_days', parseInt(e.target.value))}
min="1"
max="365"
/>
<p className="mt-1 text-xs text-gray-500">
{t('backup.configuration.schedule.retentionHelp')}
</p>
</div>
</div>
</Card>
{/* Backup Content Selection */}
<Card className="p-6">
<h3 className="text-lg font-semibold text-gray-900 mb-4">{t('backup.configuration.whatToBackup.title')}</h3>
<div className="space-y-3">
<label className="flex items-center">
<input
type="checkbox"
checked={formData.backup_include_database}
onChange={(e) => handleChange('backup_include_database', e.target.checked)}
className="h-4 w-4 text-primary focus:ring-primary border-gray-300 rounded"
/>
<div className="ml-3">
<div className="flex items-center space-x-2">
<Database className="h-4 w-4 text-gray-400" />
<span className="text-sm font-medium text-gray-700">{t('backup.configuration.whatToBackup.database')}</span>
</div>
<p className="text-xs text-gray-500">{t('backup.configuration.whatToBackup.databaseHelp')}</p>
</div>
</label>
<label className="flex items-center">
<input
type="checkbox"
checked={formData.backup_include_photos}
onChange={(e) => handleChange('backup_include_photos', e.target.checked)}
className="h-4 w-4 text-primary focus:ring-primary border-gray-300 rounded"
/>
<div className="ml-3">
<div className="flex items-center space-x-2">
<Image className="h-4 w-4 text-gray-400" />
<span className="text-sm font-medium text-gray-700">{t('backup.configuration.whatToBackup.photos')}</span>
</div>
<p className="text-xs text-gray-500">{t('backup.configuration.whatToBackup.photosHelp')}</p>
</div>
</label>
<label className="flex items-center">
<input
type="checkbox"
checked={formData.backup_include_archives}
onChange={(e) => handleChange('backup_include_archives', e.target.checked)}
className="h-4 w-4 text-primary focus:ring-primary border-gray-300 rounded"
/>
<div className="ml-3">
<div className="flex items-center space-x-2">
<FileArchive className="h-4 w-4 text-gray-400" />
<span className="text-sm font-medium text-gray-700">{t('backup.configuration.whatToBackup.archives')}</span>
</div>
<p className="text-xs text-gray-500">{t('backup.configuration.whatToBackup.archivesHelp')}</p>
</div>
</label>
<label className="flex items-center">
<input
type="checkbox"
checked={formData.backup_include_thumbnails}
onChange={(e) => handleChange('backup_include_thumbnails', e.target.checked)}
className="h-4 w-4 text-primary focus:ring-primary border-gray-300 rounded"
/>
<div className="ml-3">
<div className="flex items-center space-x-2">
<Image className="h-4 w-4 text-gray-400" />
<span className="text-sm font-medium text-gray-700">{t('backup.configuration.whatToBackup.thumbnails')}</span>
</div>
<p className="text-xs text-gray-500">{t('backup.configuration.whatToBackup.thumbnailsHelp')}</p>
</div>
</label>
</div>
</Card>
{/* Advanced Options */}
<Card className="p-6">
<h3 className="text-lg font-semibold text-gray-900 mb-4">{t('backup.configuration.advancedOptions.title')}</h3>
<div className="space-y-4">
<label className="flex items-center">
<input
type="checkbox"
checked={formData.backup_compression}
onChange={(e) => handleChange('backup_compression', e.target.checked)}
className="h-4 w-4 text-primary focus:ring-primary border-gray-300 rounded"
/>
<div className="ml-3">
<span className="text-sm font-medium text-gray-700">{t('backup.configuration.advancedOptions.compression')}</span>
<p className="text-xs text-gray-500">{t('backup.configuration.advancedOptions.compressionHelp')}</p>
</div>
</label>
<div>
<label className="flex items-center mb-3">
<input
type="checkbox"
checked={formData.backup_encryption}
onChange={(e) => handleChange('backup_encryption', e.target.checked)}
className="h-4 w-4 text-primary focus:ring-primary border-gray-300 rounded"
/>
<div className="ml-3">
<span className="text-sm font-medium text-gray-700">{t('backup.configuration.advancedOptions.encryption')}</span>
<p className="text-xs text-gray-500">{t('backup.configuration.advancedOptions.encryptionHelp')}</p>
</div>
</label>
{formData.backup_encryption && (
<div className="ml-7">
<label className="block text-sm font-medium text-gray-700 mb-1">
{t('backup.configuration.advancedOptions.encryptionPassphrase')}
</label>
<div className="relative">
<Input
type={showSecrets.encryption_passphrase ? 'text' : 'password'}
value={formData.backup_encryption_passphrase}
onChange={(e) => handleChange('backup_encryption_passphrase', e.target.value)}
placeholder={t('backup.configuration.advancedOptions.encryptionPassphraseHelp')}
required={formData.backup_encryption}
/>
<button
type="button"
onClick={() => setShowSecrets(prev => ({ ...prev, encryption_passphrase: !prev.encryption_passphrase }))}
className="absolute top-1/2 -translate-y-1/2 right-2 text-gray-400 hover:text-gray-600"
>
{showSecrets.encryption_passphrase ? <EyeOff size={20} /> : <Eye size={20} />}
</button>
</div>
<p className="mt-1 text-xs text-red-600">
<AlertCircle className="inline h-3 w-3 mr-1" />
{t('backup.configuration.advancedOptions.encryptionPassphraseHelp')}
</p>
</div>
)}
</div>
</div>
</Card>
{/* Save Button */}
<div className="flex justify-end">
<Button
type="submit"
disabled={isSaving}
>
{isSaving ? (
<>
<Loader2 className="mr-2 h-4 w-4 animate-spin" />
{t('backup.configuration.savingSettings')}
</>
) : (
<>
<Save className="mr-2 h-4 w-4" />
{t('backup.configuration.saveSettings')}
</>
)}
</Button>
</div>
</form>
);
};
@@ -1,327 +0,0 @@
import React from 'react';
import { useTranslation } from 'react-i18next';
import {
HardDrive,
Database,
FileArchive,
Image,
Clock,
CheckCircle,
AlertCircle,
TrendingUp,
Shield,
Server,
Cloud,
Calendar,
Play,
Loader2,
AlertTriangle,
Info
} from 'lucide-react';
import { format, formatDistanceToNow } from 'date-fns';
import { Card, Button } from '../common';
const StatCard = ({ icon: Icon, label, value, color = 'blue', subtext }) => (
<Card className="p-6">
<div className="flex items-center justify-between">
<div className="flex-1">
<p className="text-sm font-medium text-gray-600">{label}</p>
<p className="mt-2 text-3xl font-semibold text-gray-900">{value}</p>
{subtext && (
<p className="mt-1 text-sm text-gray-500">{subtext}</p>
)}
</div>
<div className={`p-3 bg-${color}-100 rounded-lg`}>
<Icon className={`h-6 w-6 text-${color}-600`} />
</div>
</div>
</Card>
);
const formatBytes = (bytes) => {
if (!bytes) return '0 B';
const k = 1024;
const sizes = ['B', 'KB', 'MB', 'GB', 'TB'];
const i = Math.floor(Math.log(bytes) / Math.log(k));
return `${parseFloat((bytes / Math.pow(k, i)).toFixed(2))} ${sizes[i]}`;
};
export const BackupDashboard = ({ status, config, onRunBackup, isBackupRunning }) => {
const { t } = useTranslation();
const lastBackup = status?.lastBackup;
const statistics = lastBackup?.statistics || {};
const isConfigured = config && config.backup_destination_type;
const isEnabled = config?.backup_enabled;
// Calculate backup health score
const getHealthScore = () => {
if (!lastBackup) return { score: 0, status: 'critical', message: t('backup.dashboard.healthMessages.noBackups') };
const hoursSinceBackup = (Date.now() - new Date(lastBackup.created_at)) / (1000 * 60 * 60);
if (lastBackup.status === 'failed') {
return { score: 0, status: 'critical', message: t('backup.dashboard.healthMessages.lastBackupFailed') };
}
if (hoursSinceBackup < 24) {
return { score: 100, status: 'excellent', message: t('backup.dashboard.healthMessages.upToDate') };
} else if (hoursSinceBackup < 48) {
return { score: 75, status: 'good', message: t('backup.dashboard.healthMessages.recent') };
} else if (hoursSinceBackup < 168) { // 1 week
return { score: 50, status: 'warning', message: t('backup.dashboard.healthMessages.gettingOld') };
} else {
return { score: 25, status: 'critical', message: t('backup.dashboard.healthMessages.outdated') };
}
};
const health = getHealthScore();
const healthColors = {
excellent: 'green',
good: 'blue',
warning: 'amber',
critical: 'red'
};
return (
<div className="space-y-6">
{/* Configuration Alert */}
{!isConfigured && (
<div className="bg-amber-50 border border-amber-200 rounded-lg p-4">
<div className="flex">
<AlertTriangle className="h-5 w-5 text-amber-400 mt-0.5" />
<div className="ml-3">
<h3 className="text-sm font-medium text-amber-800">
{t('backup.dashboard.notConfigured.title')}
</h3>
<p className="mt-1 text-sm text-amber-700">
{t('backup.dashboard.notConfigured.message')}
</p>
</div>
</div>
</div>
)}
{/* Health Score Card */}
<Card className="p-6">
<div className="flex items-center justify-between mb-4">
<h3 className="text-lg font-semibold text-gray-900">{t('backup.dashboard.health.title')}</h3>
<span className={`px-3 py-1 rounded-full text-sm font-medium bg-${healthColors[health.status]}-100 text-${healthColors[health.status]}-700`}>
{health.status.charAt(0).toUpperCase() + health.status.slice(1)}
</span>
</div>
<div className="flex items-center space-x-4">
<div className="relative w-24 h-24">
<svg className="w-24 h-24 transform -rotate-90">
<circle
cx="48"
cy="48"
r="36"
stroke="currentColor"
strokeWidth="8"
fill="none"
className="text-gray-200"
/>
<circle
cx="48"
cy="48"
r="36"
stroke="currentColor"
strokeWidth="8"
fill="none"
strokeDasharray={`${(health.score / 100) * 226} 226`}
className={`text-${healthColors[health.status]}-500`}
/>
</svg>
<div className="absolute inset-0 flex items-center justify-center">
<span className="text-2xl font-bold text-gray-900">{health.score}%</span>
</div>
</div>
<div className="flex-1">
<p className="text-gray-700 font-medium">{health.message}</p>
{lastBackup && (
<p className="text-sm text-gray-500 mt-1">
Last successful backup: {formatDistanceToNow(new Date(lastBackup.created_at), { addSuffix: true })}
</p>
)}
<Button
onClick={onRunBackup}
disabled={!isConfigured || !isEnabled || isBackupRunning}
className="mt-3"
size="sm"
>
{isBackupRunning ? (
<>
<Loader2 className="mr-2 h-4 w-4 animate-spin" />
{t('backup.dashboard.actions.running')}
</>
) : (
<>
<Play className="mr-2 h-4 w-4" />
{t('backup.dashboard.actions.runBackupNow')}
</>
)}
</Button>
</div>
</div>
</Card>
{/* Statistics Grid */}
<div className="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-4 gap-4">
<StatCard
icon={FileArchive}
label={t('backup.dashboard.stats.totalBackups')}
value={status?.totalBackups || 0}
color="blue"
subtext={lastBackup ? `${t('backup.dashboard.stats.last')}: ${format(new Date(lastBackup.created_at), 'PP')}` : t('backup.dashboard.stats.noBackupsYet')}
/>
<StatCard
icon={HardDrive}
label={t('backup.dashboard.stats.backupSize')}
value={formatBytes(statistics.total_size || 0)}
color="green"
subtext={`${statistics.files_processed || 0} ${t('backup.dashboard.stats.files')}`}
/>
<StatCard
icon={Clock}
label={t('backup.dashboard.stats.lastDuration')}
value={lastBackup ? `${Math.round(lastBackup.duration_seconds / 60)}m` : 'N/A'}
color="purple"
subtext={lastBackup ? format(new Date(lastBackup.created_at), 'p') : ''}
/>
<StatCard
icon={Shield}
label={t('backup.dashboard.stats.backupStatus')}
value={isEnabled ? t('backup.dashboard.stats.active') : t('backup.dashboard.stats.inactive')}
color={isEnabled ? 'green' : 'gray'}
subtext={config?.backup_destination_type || t('backup.dashboard.notConfigured.title')}
/>
</div>
{/* Recent Activity */}
{status?.recentBackups && status.recentBackups.length > 0 && (
<Card className="p-6">
<h3 className="text-lg font-semibold text-gray-900 mb-4">{t('backup.dashboard.recentActivity.title')}</h3>
<div className="space-y-3">
{status.recentBackups.slice(0, 5).map((backup) => (
<div key={backup.id} className="flex items-center justify-between py-3 border-b border-gray-100 last:border-0">
<div className="flex items-center space-x-3">
{backup.status === 'completed' ? (
<CheckCircle className="h-5 w-5 text-green-500" />
) : backup.status === 'failed' ? (
<AlertCircle className="h-5 w-5 text-red-500" />
) : (
<Loader2 className="h-5 w-5 text-blue-500 animate-spin" />
)}
<div>
<p className="font-medium text-gray-900">
{t('backup.dashboard.backupType', { type: backup.backup_type })}
</p>
<p className="text-sm text-gray-500">
{format(new Date(backup.created_at), 'PPp')}
</p>
</div>
</div>
<div className="text-right">
<p className="text-sm font-medium text-gray-900">
{formatBytes(backup.statistics?.total_size || 0)}
</p>
<p className="text-sm text-gray-500">
{backup.statistics?.files_processed || 0} files
</p>
</div>
</div>
))}
</div>
</Card>
)}
{/* Storage Status */}
<div className="grid grid-cols-1 lg:grid-cols-2 gap-6">
<Card className="p-6">
<h3 className="text-lg font-semibold text-gray-900 mb-4">{t('backup.dashboard.coverage.title')}</h3>
<div className="space-y-4">
<div className="flex items-center justify-between">
<div className="flex items-center space-x-2">
<Database className="h-5 w-5 text-gray-400" />
<span className="text-gray-700">Database</span>
</div>
<span className={`px-2 py-1 rounded text-xs font-medium ${
statistics.database_backed_up ? 'bg-green-100 text-green-700' : 'bg-gray-100 text-gray-700'
}`}>
{statistics.database_backed_up ? t('backup.dashboard.coverage.included') : t('backup.dashboard.coverage.excluded')}
</span>
</div>
<div className="flex items-center justify-between">
<div className="flex items-center space-x-2">
<Image className="h-5 w-5 text-gray-400" />
<span className="text-gray-700">{t('backup.configuration.whatToBackup.photos')}</span>
</div>
<span className="text-sm text-gray-500">
{statistics.photos_backed_up || 0} {t('common.of')} {statistics.total_photos || 0}
</span>
</div>
<div className="flex items-center justify-between">
<div className="flex items-center space-x-2">
<FileArchive className="h-5 w-5 text-gray-400" />
<span className="text-gray-700">{t('backup.configuration.whatToBackup.archives')}</span>
</div>
<span className="text-sm text-gray-500">
{statistics.archives_backed_up || 0} {t('backup.dashboard.stats.files')}
</span>
</div>
</div>
</Card>
<Card className="p-6">
<h3 className="text-lg font-semibold text-gray-900 mb-4">{t('backup.dashboard.storageDestination')}</h3>
<div className="space-y-3">
<div className="flex items-center space-x-3">
{config?.backup_destination_type === 's3' ? (
<Cloud className="h-5 w-5 text-blue-500" />
) : config?.backup_destination_type === 'rsync' ? (
<Server className="h-5 w-5 text-purple-500" />
) : (
<HardDrive className="h-5 w-5 text-gray-500" />
)}
<div>
<p className="font-medium text-gray-900">
{config?.backup_destination_type
? t(`backup.configuration.destinationTypes.${config.backup_destination_type}.name`)
: t('backup.dashboard.notConfigured.title')}
</p>
<p className="text-sm text-gray-500">
{config?.backup_destination_type === 's3' && config?.backup_s3_bucket
? `Bucket: ${config.backup_s3_bucket}`
: config?.backup_destination_type === 'local' && config?.backup_destination_path
? `Path: ${config.backup_destination_path}`
: config?.backup_destination_type === 'rsync' && config?.backup_rsync_host
? `Host: ${config.backup_rsync_host}`
: t('backup.dashboard.noDestinationSet')}
</p>
</div>
</div>
{config?.backup_retention_days && (
<div className="mt-4 p-3 bg-gray-50 rounded-lg">
<div className="flex items-center space-x-2">
<Info className="h-4 w-4 text-gray-400" />
<span className="text-sm text-gray-600">
{t('backup.configuration.schedule.retentionDays')} {config.backup_retention_days} {t('backup.configuration.schedule.retentionHelp').replace('days (older backups will be automatically deleted)', '')}
</span>
</div>
</div>
)}
</div>
</Card>
</div>
</div>
);
};

Some files were not shown because too many files have changed in this diff Show More