Compare commits
113 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 840b8870ec | |||
| ad495a92c4 | |||
| b428543452 | |||
| 6492cb9ec8 | |||
| 0e0a0b91d1 | |||
| f8fb1c3f4b | |||
| b108f6fe1c | |||
| 61299a33c4 | |||
| 96542d7e35 | |||
| ee855a3502 | |||
| 02c407d431 | |||
| 62617f627f | |||
| 1cbeb75094 | |||
| baa08e9ec9 | |||
| 8d85454ef6 | |||
| 596bba2c1b | |||
| 055de06315 | |||
| ccf59d1d4d | |||
| 4e052966d3 | |||
| ba2c021c45 | |||
| 519518ed6c | |||
| 8a0a4436b0 | |||
| 9854ca2f59 | |||
| 0c989ce086 | |||
| 35e360dcf7 | |||
| a209796b16 | |||
| 7c79052681 | |||
| d560453982 | |||
| ecb3263267 | |||
| 4d929a71ce | |||
| bf705674d5 | |||
| fee369a503 | |||
| ad75818566 | |||
| 65f2c8610d | |||
| 517128fd99 | |||
| 55c8384a25 | |||
| 0064122eff | |||
| 0c783c66d0 | |||
| 47e2351dab | |||
| e1aca6b00c | |||
| 2cb6577f26 | |||
| c51d756503 | |||
| c48b9780df | |||
| 618e2695fd | |||
| a289f97a31 | |||
| 7387a5e9f9 | |||
| 4615a5d795 | |||
| f926cd3adf | |||
| 96b05b5e0c | |||
| 99e47785e4 | |||
| 94f10e1645 | |||
| fe4a476e41 | |||
| e9f92e66d0 | |||
| 58f4217756 | |||
| 76a466c077 | |||
| 9dc3777985 | |||
| e006d73831 | |||
| 63e88c8324 | |||
| 105167fb57 | |||
| 22cc40617f | |||
| 7f28917795 | |||
| 77af2a8415 | |||
| 4c42b4c601 | |||
| 856cdc214c | |||
| 15b244292f | |||
| 558a966f85 | |||
| 1238db58c2 | |||
| 0502ed34c9 | |||
| 1f417c7e30 | |||
| a401fbdc54 | |||
| 247e154afe | |||
| cbfd84ddea | |||
| dc1419c051 | |||
| 2624ea6130 | |||
| 08eeac66eb | |||
| 11769219e4 | |||
| 7750170832 | |||
| 833591681a | |||
| b77e60c37c | |||
| 30f6780484 | |||
| aa39e132aa | |||
| f6a79c815e | |||
| 3c6837bd90 | |||
| 1773ed5f95 | |||
| 62dcaf8555 | |||
| 4ed35f1b16 | |||
| 5eff7dd4a6 | |||
| abbcdb1113 | |||
| 17fc40e65d | |||
| a54a2c0fda | |||
| febacb79ad | |||
| c7875102c5 | |||
| 3dc013d7b1 | |||
| b7c8953cb4 | |||
| d6adde4e09 | |||
| 0bf4764a07 | |||
| 08da01f021 | |||
| b4b09c1650 | |||
| b2ae5f18ad | |||
| 4a7a3bba07 | |||
| b3f240b2a5 | |||
| ba0bf11a1d | |||
| d5790ad635 | |||
| e6757bd51b | |||
| 8588133a4e | |||
| 4b18077573 | |||
| c584369d5d | |||
| 95939d57e6 | |||
| be58146dc7 | |||
| 45ce98806d | |||
| 23b7a848ab | |||
| 0fe6d738b2 | |||
| 3f73d44c5a |
@@ -1,286 +0,0 @@
|
||||
# Security Scan Report - Wedding Photo Sharing Application
|
||||
**Date**: July 13, 2025
|
||||
**Scanner**: Claude Security Audit with --security --validate flags
|
||||
**Overall Risk Level**: MEDIUM-HIGH
|
||||
|
||||
## Executive Summary
|
||||
|
||||
The wedding photo sharing application demonstrates strong security fundamentals with comprehensive input validation, proper authentication mechanisms, and good file security practices. However, several critical issues require immediate attention, particularly around hardcoded secrets, token storage, and Content Security Policy configuration.
|
||||
|
||||
### Security Score: 6.5/10
|
||||
|
||||
**Strengths**: Excellent input validation, parameterized queries, file security, rate limiting
|
||||
**Critical Issues**: Hardcoded JWT secrets, localStorage token storage, weak CSP, console logging in production
|
||||
|
||||
---
|
||||
|
||||
## 🔴 CRITICAL FINDINGS (Immediate Action Required)
|
||||
|
||||
### 1. Hardcoded JWT Secret in Development
|
||||
- **Location**: Backend `.env` file
|
||||
- **Risk**: Token forgery, authentication bypass
|
||||
- **Impact**: Complete authentication compromise
|
||||
- **Remediation**:
|
||||
```bash
|
||||
# Generate secure secret
|
||||
openssl rand -base64 32
|
||||
# Never commit to repository
|
||||
echo ".env" >> .gitignore
|
||||
```
|
||||
|
||||
### 2. Gallery Tokens in localStorage
|
||||
- **Location**: Frontend `api.ts` and auth contexts
|
||||
- **Risk**: XSS token theft
|
||||
- **Impact**: Gallery access compromise
|
||||
- **Remediation**: Move to httpOnly cookies:
|
||||
```typescript
|
||||
Cookies.set(`gallery_token_${slug}`, token, {
|
||||
httpOnly: true,
|
||||
secure: true,
|
||||
sameSite: 'strict'
|
||||
});
|
||||
```
|
||||
|
||||
### 3. Weak Content Security Policy
|
||||
- **Location**: Frontend `nginx.conf`
|
||||
- **Risk**: XSS, code injection
|
||||
- **Current**: `unsafe-inline` and `unsafe-eval` allowed
|
||||
- **Remediation**: Implement strict CSP (see detailed recommendations below)
|
||||
|
||||
---
|
||||
|
||||
## 🟠 HIGH SEVERITY FINDINGS
|
||||
|
||||
### 1. Console Logging in Production
|
||||
- **Locations**: 61 instances across frontend
|
||||
- **Risk**: Information disclosure
|
||||
- **Impact**: Leaking sensitive data, debugging info
|
||||
- **Remediation**: Implement environment-aware logging
|
||||
|
||||
### 2. Token Revocation Vulnerability
|
||||
- **Location**: Backend `tokenRevocation.js`
|
||||
- **Risk**: Token manipulation
|
||||
- **Impact**: Bypass revocation checks
|
||||
- **Remediation**: Verify token signature before decoding
|
||||
|
||||
### 3. Source Maps in Production
|
||||
- **Location**: Frontend build configuration
|
||||
- **Risk**: Source code exposure
|
||||
- **Impact**: Reveals application structure
|
||||
- **Remediation**: Disable in production builds
|
||||
|
||||
### 4. Missing Security Headers
|
||||
- **Location**: nginx configuration
|
||||
- **Missing**: HSTS, Permissions-Policy
|
||||
- **Impact**: Various client-side attacks
|
||||
- **Remediation**: Add comprehensive security headers
|
||||
|
||||
---
|
||||
|
||||
## 🟡 MEDIUM SEVERITY FINDINGS
|
||||
|
||||
### 1. Rate Limiting Bypass Potential
|
||||
- **Location**: Backend rate limiter
|
||||
- **Risk**: DoS attacks
|
||||
- **Current**: JWT validation in rate limiter
|
||||
- **Remediation**: Use IP-based limiting only
|
||||
|
||||
### 2. Incomplete SQL Injection Protection
|
||||
- **Location**: Complex dashboard queries
|
||||
- **Risk**: Potential injection in edge cases
|
||||
- **Current**: Mostly parameterized
|
||||
- **Remediation**: Use query builder exclusively
|
||||
|
||||
### 3. Session Management
|
||||
- **Issue**: No gallery token invalidation on password change
|
||||
- **Risk**: Persistent access after compromise
|
||||
- **Remediation**: Implement token revocation
|
||||
|
||||
### 4. Path Traversal in Gallery Slugs
|
||||
- **Location**: Frontend gallery routes
|
||||
- **Risk**: Directory traversal attempts
|
||||
- **Remediation**: Validate and sanitize slugs
|
||||
|
||||
---
|
||||
|
||||
## 🟢 LOW SEVERITY FINDINGS
|
||||
|
||||
### 1. Verbose Error Messages
|
||||
- **Location**: Multiple API endpoints
|
||||
- **Risk**: Information disclosure
|
||||
- **Remediation**: Generic client errors, detailed server logs
|
||||
|
||||
### 2. Weak Gallery Passwords
|
||||
- **Current**: zxcvbn score 2/4 allowed
|
||||
- **Risk**: Brute force attacks
|
||||
- **Remediation**: Increase to score 3/4
|
||||
|
||||
### 3. Missing File Size Validation
|
||||
- **Location**: Frontend upload components
|
||||
- **Risk**: DoS via large uploads
|
||||
- **Remediation**: Add client-side size checks
|
||||
|
||||
---
|
||||
|
||||
## ✅ SECURITY STRENGTHS
|
||||
|
||||
### Authentication & Authorization
|
||||
- JWT with proper expiration (24h/7d)
|
||||
- Token type validation
|
||||
- IP tracking and validation
|
||||
- Password change detection
|
||||
- Token revocation system
|
||||
- Bcrypt with 12 rounds
|
||||
- zxcvbn password strength checking
|
||||
|
||||
### Input Validation & SQL Security
|
||||
- express-validator on all endpoints
|
||||
- Parameterized queries via Knex
|
||||
- SQL injection protection utilities
|
||||
- Path traversal prevention
|
||||
- Comprehensive input sanitization
|
||||
|
||||
### File Security
|
||||
- Magic number verification
|
||||
- MIME type validation
|
||||
- Safe filename generation
|
||||
- Directory traversal protection
|
||||
- File extension whitelist
|
||||
|
||||
### Rate Limiting & DoS Protection
|
||||
- General: 100 req/15min
|
||||
- Auth endpoints: 5 req/15min
|
||||
- Account lockout after failed attempts
|
||||
- Suspicious activity detection
|
||||
|
||||
### Frontend Security
|
||||
- React's built-in XSS protection
|
||||
- DOMPurify for HTML content
|
||||
- No eval() or innerHTML usage
|
||||
- Proper error boundaries
|
||||
- ReCAPTCHA integration
|
||||
|
||||
---
|
||||
|
||||
## 📊 DEPENDENCY ANALYSIS
|
||||
|
||||
### Current Status
|
||||
- **Backend**: 0 vulnerabilities (691 packages)
|
||||
- **Frontend**: 0 vulnerabilities (434 packages)
|
||||
|
||||
### Recommended Updates
|
||||
1. **bcrypt** 5.1.1 → 6.0.0 (performance, compatibility)
|
||||
2. **helmet** 7.2.0 → 8.1.0 (new security features)
|
||||
3. **@tiptap** 2.x → 3.x (security improvements)
|
||||
|
||||
### Supply Chain Assessment
|
||||
- All major dependencies from trusted sources
|
||||
- No typosquatting detected
|
||||
- Regular maintenance observed
|
||||
- MIT/ISC/Apache licenses only
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ REMEDIATION PLAN
|
||||
|
||||
### Phase 1: Critical (Within 24 hours)
|
||||
1. Replace hardcoded JWT secret with secure random value
|
||||
2. Move gallery tokens from localStorage to httpOnly cookies
|
||||
3. Implement strict CSP without unsafe-eval
|
||||
4. Remove or wrap console.log statements
|
||||
|
||||
### Phase 2: High Priority (Within 1 week)
|
||||
1. Disable source maps in production
|
||||
2. Add missing security headers (HSTS, Permissions-Policy)
|
||||
3. Fix token revocation vulnerability
|
||||
4. Update critical dependencies (bcrypt, helmet)
|
||||
|
||||
### Phase 3: Medium Priority (Within 1 month)
|
||||
1. Implement comprehensive logging strategy
|
||||
2. Add gallery slug validation
|
||||
3. Enhance rate limiting logic
|
||||
4. Implement session invalidation on password change
|
||||
|
||||
### Phase 4: Ongoing
|
||||
1. Weekly dependency scanning
|
||||
2. Implement security testing in CI/CD
|
||||
3. Regular penetration testing
|
||||
4. Security awareness training
|
||||
|
||||
---
|
||||
|
||||
## 🔒 RECOMMENDED CSP CONFIGURATION
|
||||
|
||||
```nginx
|
||||
add_header Content-Security-Policy "
|
||||
default-src 'self';
|
||||
script-src 'self' 'nonce-{RANDOM}' https://www.google.com/recaptcha/ https://www.gstatic.com/recaptcha/;
|
||||
style-src 'self' 'unsafe-inline';
|
||||
img-src 'self' data: blob: https:;
|
||||
font-src 'self';
|
||||
connect-src 'self' https://analytics.domain.com;
|
||||
frame-src https://www.google.com/recaptcha/;
|
||||
object-src 'none';
|
||||
base-uri 'self';
|
||||
form-action 'self';
|
||||
frame-ancestors 'none';
|
||||
upgrade-insecure-requests;
|
||||
" always;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚀 SECURITY IMPROVEMENTS ROADMAP
|
||||
|
||||
### Immediate Implementation
|
||||
```bash
|
||||
# 1. Generate secure secrets
|
||||
openssl rand -base64 32 > jwt-secret.txt
|
||||
|
||||
# 2. Update dependencies
|
||||
cd backend && npm install bcrypt@^6.0.0 helmet@^8.1.0
|
||||
cd ../frontend && npm update
|
||||
|
||||
# 3. Add security scanning
|
||||
npm install -D npm-audit-resolver
|
||||
```
|
||||
|
||||
### CI/CD Integration
|
||||
```yaml
|
||||
# Add to CI pipeline
|
||||
- name: Security Scan
|
||||
run: |
|
||||
npm audit --audit-level=moderate
|
||||
npm run test:security
|
||||
```
|
||||
|
||||
### Monitoring & Alerting
|
||||
1. Implement fail2ban for repeated auth failures
|
||||
2. Set up log analysis for suspicious patterns
|
||||
3. Configure alerts for security events
|
||||
4. Regular vulnerability scanning
|
||||
|
||||
---
|
||||
|
||||
## 📋 COMPLIANCE CHECKLIST
|
||||
|
||||
- [ ] OWASP Top 10 addressed
|
||||
- [ ] GDPR compliance (data minimization, right to erasure)
|
||||
- [ ] Security headers implemented
|
||||
- [ ] Dependency scanning automated
|
||||
- [ ] Incident response plan documented
|
||||
- [ ] Security documentation maintained
|
||||
- [ ] Regular security reviews scheduled
|
||||
|
||||
---
|
||||
|
||||
## 🎯 CONCLUSION
|
||||
|
||||
The wedding photo sharing application has a solid security foundation with excellent input validation and authentication mechanisms. However, operational security practices need immediate attention. The critical issues around secret management and token storage must be addressed before production deployment.
|
||||
|
||||
Implementing the recommended fixes will raise the security score from 6.5/10 to approximately 8.5/10, providing a robust and secure platform for wedding photo sharing.
|
||||
|
||||
---
|
||||
|
||||
*Generated by Claude Security Scanner v1.0*
|
||||
*Next scan recommended: After Phase 1 remediation completion*
|
||||
+37
@@ -72,6 +72,43 @@ steps:
|
||||
context: frontend/
|
||||
registry: registry.local.nothaft.cloud
|
||||
|
||||
# -------- NEW: Publish Docker images to GitHub Container Registry --------
|
||||
- name: push-backend-ghcr
|
||||
image: plugins/docker
|
||||
settings:
|
||||
repo: ghcr.io/the-luap/picpeak-backend
|
||||
tags:
|
||||
- ${DRONE_TAG}
|
||||
- latest
|
||||
dockerfile: backend/Dockerfile
|
||||
context: backend/
|
||||
registry: ghcr.io
|
||||
username:
|
||||
from_secret: GITHUB_USERNAME
|
||||
password:
|
||||
from_secret: GITHUB_TOKEN
|
||||
build_args:
|
||||
- VERSION=${DRONE_TAG}
|
||||
|
||||
- name: push-frontend-ghcr
|
||||
image: plugins/docker
|
||||
settings:
|
||||
repo: ghcr.io/the-luap/picpeak-frontend
|
||||
tags:
|
||||
- ${DRONE_TAG}
|
||||
- latest
|
||||
dockerfile: frontend/Dockerfile
|
||||
context: frontend/
|
||||
registry: ghcr.io
|
||||
username:
|
||||
from_secret: GITHUB_USERNAME
|
||||
password:
|
||||
from_secret: GITHUB_TOKEN
|
||||
build_args:
|
||||
- VERSION=${DRONE_TAG}
|
||||
- VITE_API_URL=${VITE_API_URL:-/api}
|
||||
|
||||
|
||||
trigger:
|
||||
event:
|
||||
- tag
|
||||
+42
-26
@@ -1,34 +1,50 @@
|
||||
# Environment Configuration Template
|
||||
# Copy this file to .env and adjust values for your environment
|
||||
# PicPeak Environment Configuration
|
||||
# Copy this file to .env and update with your values
|
||||
|
||||
# Development: Use docker-compose.dev.yml
|
||||
# Production: Use docker-compose.prod.yml with .env.production.example
|
||||
# Environment
|
||||
NODE_ENV=production
|
||||
|
||||
# JWT Secret (CRITICAL for production)
|
||||
# Generate with: openssl rand -base64 32
|
||||
JWT_SECRET=dev-secret-change-in-production
|
||||
# JWT Secret (generate with: openssl rand -base64 64)
|
||||
JWT_SECRET=your_very_long_random_jwt_secret_here
|
||||
|
||||
# Application URLs
|
||||
ADMIN_URL=http://localhost:3005
|
||||
FRONTEND_URL=http://localhost:3005
|
||||
# Database Configuration (PostgreSQL)
|
||||
DATABASE_CLIENT=pg
|
||||
DB_USER=picpeak
|
||||
DB_PASSWORD=your_secure_postgres_password_here
|
||||
DB_NAME=picpeak_prod
|
||||
|
||||
# Database Configuration
|
||||
# SQLite is used for development by default
|
||||
# For production PostgreSQL config, see .env.production.example
|
||||
DATABASE_CLIENT=sqlite3
|
||||
DATABASE_PATH=./data/photo_sharing.db
|
||||
# Redis Configuration
|
||||
REDIS_PASSWORD=your_secure_redis_password_here
|
||||
|
||||
# Admin Account (initial setup)
|
||||
ADMIN_USERNAME=admin
|
||||
ADMIN_EMAIL=admin@yourdomain.com
|
||||
|
||||
# Email Configuration
|
||||
# Development: Uses Mailhog (included in docker-compose.dev.yml)
|
||||
# Production: Configure real SMTP server
|
||||
SMTP_HOST=mailhog
|
||||
SMTP_PORT=1025
|
||||
# For Gmail: use app-specific password
|
||||
# For SendGrid: SMTP_USER=apikey, SMTP_PASS=your-api-key
|
||||
SMTP_HOST=smtp.gmail.com
|
||||
SMTP_PORT=587
|
||||
SMTP_SECURE=false
|
||||
SMTP_USER=
|
||||
SMTP_PASS=
|
||||
EMAIL_FROM=noreply@localhost
|
||||
SMTP_USER=your-email@gmail.com
|
||||
SMTP_PASS=your-app-specific-password
|
||||
EMAIL_FROM=noreply@yourdomain.com
|
||||
|
||||
# Optional: Umami Analytics
|
||||
UMAMI_URL=
|
||||
UMAMI_WEBSITE_ID=
|
||||
UMAMI_HASH_SALT=
|
||||
# Application URLs
|
||||
FRONTEND_URL=https://yourdomain.com
|
||||
ADMIN_URL=https://yourdomain.com:3001
|
||||
VITE_API_URL=https://yourdomain.com:3001/api
|
||||
|
||||
# Port Configuration (optional)
|
||||
# BACKEND_PORT=3001
|
||||
# FRONTEND_PORT=3000
|
||||
# DB_PORT=5432
|
||||
# REDIS_PORT=6379
|
||||
|
||||
# Timezone
|
||||
TZ=UTC
|
||||
|
||||
# Analytics (Optional - Umami)
|
||||
VITE_UMAMI_URL=
|
||||
VITE_UMAMI_WEBSITE_ID=
|
||||
VITE_UMAMI_SHARE_URL=
|
||||
@@ -1,44 +0,0 @@
|
||||
# PicPeak Production Configuration
|
||||
# Copy this file to .env and update with your values
|
||||
|
||||
# Required: Security
|
||||
JWT_SECRET=CHANGE_THIS_TO_RANDOM_32_CHAR_STRING
|
||||
|
||||
# Required: URLs (update with your domain)
|
||||
FRONTEND_URL=https://your-domain.com
|
||||
BACKEND_URL=https://your-domain.com
|
||||
ADMIN_URL=https://your-domain.com
|
||||
|
||||
# Required: Email Settings
|
||||
SMTP_HOST=smtp.gmail.com
|
||||
SMTP_PORT=587
|
||||
SMTP_USER=your-email@gmail.com
|
||||
SMTP_PASS=your-app-password
|
||||
SMTP_FROM=your-email@gmail.com
|
||||
|
||||
# Required: Initial Admin Account
|
||||
ADMIN_EMAIL=admin@your-domain.com
|
||||
ADMIN_PASSWORD=change-this-password
|
||||
|
||||
# Database (PostgreSQL recommended for production)
|
||||
DATABASE_CLIENT=pg
|
||||
DB_HOST=postgres
|
||||
DB_PORT=5432
|
||||
DB_NAME=picpeak
|
||||
DB_USER=picpeak
|
||||
DB_PASSWORD=secure-database-password
|
||||
|
||||
# Optional: Customization
|
||||
SITE_NAME=PicPeak
|
||||
DEFAULT_EXPIRATION_DAYS=30
|
||||
SESSION_TIMEOUT_MINUTES=60
|
||||
|
||||
# Optional: Analytics (Umami)
|
||||
VITE_UMAMI_URL=
|
||||
VITE_UMAMI_WEBSITE_ID=
|
||||
|
||||
# Advanced: Performance Tuning
|
||||
NODE_ENV=production
|
||||
BCRYPT_ROUNDS=12
|
||||
RATE_LIMIT_WINDOW_MS=900000
|
||||
RATE_LIMIT_MAX_REQUESTS=100
|
||||
@@ -1,12 +0,0 @@
|
||||
# Files to exclude from GitHub mirror
|
||||
.env* export-ignore
|
||||
docker-compose.prod.yml export-ignore
|
||||
.claudedocs/ export-ignore
|
||||
backend/data/ export-ignore
|
||||
backend/storage/ export-ignore
|
||||
backend/.env* export-ignore
|
||||
frontend/.env* export-ignore
|
||||
secrets/ export-ignore
|
||||
*.key export-ignore
|
||||
*.pem export-ignore
|
||||
.gitea/ export-ignore
|
||||
@@ -13,55 +13,43 @@ jobs:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v3
|
||||
with:
|
||||
fetch-depth: 0 # Full history needed for mirroring
|
||||
fetch-depth: 0 # Full history for proper mirroring
|
||||
|
||||
- name: Setup Git
|
||||
run: |
|
||||
git config --global user.name "the-luap"
|
||||
git config --global user.email "paul-nothaft@hotmail.de"
|
||||
|
||||
- name: Debug - Show current branch and status
|
||||
- name: Remove sensitive files and directories
|
||||
run: |
|
||||
echo "Current branch:"
|
||||
git branch -a
|
||||
echo "Git status:"
|
||||
git status
|
||||
echo "Remote info:"
|
||||
git remote -v
|
||||
|
||||
- name: Create filtered branch
|
||||
run: |
|
||||
# Clean up any existing github-mirror branch
|
||||
git branch -D github-mirror || true
|
||||
echo "Current files before cleanup:"
|
||||
ls -la | head -10 || true
|
||||
echo "..."
|
||||
|
||||
# Create a new branch for GitHub
|
||||
git checkout --orphan github-mirror
|
||||
|
||||
# Remove sensitive files/directories
|
||||
# Example: Remove .env files, private configs, etc.
|
||||
git rm -r --cached .env* || true
|
||||
git rm -r --cached backend/.env* || true
|
||||
git rm -r --cached frontend/.env* || true
|
||||
git rm -r --cached docker-compose.prod.yml || true
|
||||
git rm -r --cached .claudedocs/ || true
|
||||
git rm -r --cached backend/data/ || true
|
||||
git rm -r --cached backend/storage/ || true
|
||||
git rm -r --cached .gitea/ || true
|
||||
git rm -r --cached scripts/install-gitea-runner.sh || true
|
||||
git rm -r --cached .drone* || true
|
||||
git rm -r --cached .github-mirror-exclude || true
|
||||
git rm -r --cached .gitattributes-github || true
|
||||
git rm -r --cached photo-sharing-prd.md || true
|
||||
git rm -r --cached CLAUDE.md || true
|
||||
git rm -r --cached PRODUCTION_DEPLOYMENT_GUIDE.md || true
|
||||
git rm -r --cached logs/ || true
|
||||
git rm -r --cached frontend/.claudedocs/ || true
|
||||
git rm -r --cached test-maintenance.sh || true
|
||||
git rm -r --cached storage/ || true
|
||||
# Remove sensitive files/directories if they exist
|
||||
echo "Removing sensitive files..."
|
||||
rm -rf .gitea/ || true
|
||||
rm -rf scripts/ || true
|
||||
rm -rf .drone* || true
|
||||
rm -rf photo-sharing-prd.md || true
|
||||
rm -rf CLAUDE.md || true
|
||||
rm -rf storage/ || true
|
||||
|
||||
|
||||
# Commit the changes
|
||||
git commit -m "Remove sensitive files for GitHub mirror" || true
|
||||
|
||||
|
||||
echo "Sensitive files removal completed"
|
||||
|
||||
# Add and commit the cleanup if there are changes
|
||||
git add -A
|
||||
if ! git diff --cached --quiet; then
|
||||
git commit -m "chore: remove sensitive files for GitHub mirror"
|
||||
echo "✅ Committed cleanup of sensitive files"
|
||||
else
|
||||
echo "✅ No sensitive files to remove"
|
||||
fi
|
||||
|
||||
echo "Final file structure (top level):"
|
||||
ls -la | head -10 || true
|
||||
|
||||
- name: Check GitHub token
|
||||
env:
|
||||
@@ -88,12 +76,13 @@ jobs:
|
||||
echo "GitHub remote added:"
|
||||
git remote -v
|
||||
|
||||
# Force push the filtered branch to GitHub main
|
||||
# Push to GitHub main branch
|
||||
echo "Pushing to GitHub..."
|
||||
git push github github-mirror:main --force
|
||||
echo "Push completed successfully!"
|
||||
git push github main --force
|
||||
echo "✅ Push to GitHub completed!"
|
||||
|
||||
- name: Workflow completed
|
||||
run: |
|
||||
echo "✅ Mirror to GitHub workflow completed successfully!"
|
||||
echo "Check https://github.com/the-luap/picpeak to verify the mirror."
|
||||
echo "📊 Repository mirrored to: https://github.com/the-luap/picpeak"
|
||||
echo "🔒 Sensitive files have been removed from the mirror"
|
||||
@@ -14,6 +14,7 @@ jobs:
|
||||
outputs:
|
||||
new_version: ${{ steps.version.outputs.new_version }}
|
||||
version_changed: ${{ steps.version.outputs.version_changed }}
|
||||
component_changed: ${{ steps.version.outputs.component_changed }}
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
with:
|
||||
@@ -30,15 +31,104 @@ jobs:
|
||||
git config --global user.name 'Gitea Actions Bot'
|
||||
git config --global user.email 'actions@gitea.local'
|
||||
|
||||
- name: Bump version
|
||||
- name: Detect changes and bump version
|
||||
id: version
|
||||
run: |
|
||||
# Get current version from backend package.json
|
||||
CURRENT_VERSION=$(node -p "require('./backend/package.json').version")
|
||||
echo "Current version: $CURRENT_VERSION"
|
||||
set -e # Exit on error
|
||||
|
||||
# Split version into parts
|
||||
IFS='.' read -r -a version_parts <<< "$CURRENT_VERSION"
|
||||
echo "=== Debug Info ==="
|
||||
echo "GitHub event before: ${{ github.event.before }}"
|
||||
echo "GitHub SHA: ${{ github.sha }}"
|
||||
echo "Current directory: $(pwd)"
|
||||
echo "Git log (last 5): $(git log --oneline -5)"
|
||||
|
||||
# Get the commit range for changed files
|
||||
if [ "${{ github.event.before }}" != "0000000000000000000000000000000000000000" ] && [ "${{ github.event.before }}" != "" ]; then
|
||||
COMMIT_RANGE="${{ github.event.before }}..${{ github.sha }}"
|
||||
echo "Using commit range: $COMMIT_RANGE"
|
||||
CHANGED_FILES=$(git diff --name-only $COMMIT_RANGE || echo "")
|
||||
else
|
||||
# First commit or no previous commit, check against HEAD~1 if it exists
|
||||
if git rev-parse HEAD~1 >/dev/null 2>&1; then
|
||||
COMMIT_RANGE="HEAD~1..HEAD"
|
||||
echo "Using commit range: $COMMIT_RANGE"
|
||||
CHANGED_FILES=$(git diff --name-only $COMMIT_RANGE || echo "")
|
||||
else
|
||||
echo "First commit detected, checking all files"
|
||||
CHANGED_FILES=$(git ls-files)
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "Changed files:"
|
||||
echo "$CHANGED_FILES"
|
||||
|
||||
# Check what changed (using echo to pipe to grep to avoid grep exit codes)
|
||||
BACKEND_CHANGED=$(echo "$CHANGED_FILES" | grep -c '^backend/' || echo "0")
|
||||
FRONTEND_CHANGED=$(echo "$CHANGED_FILES" | grep -c '^frontend/' || echo "0")
|
||||
ROOT_CHANGED=$(echo "$CHANGED_FILES" | grep -c -E '^(package\.json|docker-compose|Dockerfile|scripts/)' || echo "0")
|
||||
|
||||
echo "Backend files changed: $BACKEND_CHANGED"
|
||||
echo "Frontend files changed: $FRONTEND_CHANGED"
|
||||
echo "Root files changed: $ROOT_CHANGED"
|
||||
|
||||
# Get current versions
|
||||
BACKEND_VERSION=$(node -p "require('./backend/package.json').version" 2>/dev/null || echo "1.0.0")
|
||||
FRONTEND_VERSION=$(node -p "require('./frontend/package.json').version" 2>/dev/null || echo "1.0.0")
|
||||
|
||||
echo "Current backend version: $BACKEND_VERSION"
|
||||
echo "Current frontend version: $FRONTEND_VERSION"
|
||||
|
||||
# Determine what to update based on changes
|
||||
BACKEND_UPDATE=false
|
||||
FRONTEND_UPDATE=false
|
||||
COMPONENT_CHANGED="none"
|
||||
|
||||
if [ "$ROOT_CHANGED" -gt 0 ]; then
|
||||
# Root changes affect both components
|
||||
BACKEND_UPDATE=true
|
||||
FRONTEND_UPDATE=true
|
||||
COMPONENT_CHANGED="both"
|
||||
SOURCE_VERSION=$BACKEND_VERSION
|
||||
echo "Root changes detected - updating both components"
|
||||
elif [ "$BACKEND_CHANGED" -gt 0 ] && [ "$FRONTEND_CHANGED" -gt 0 ]; then
|
||||
# Both components changed
|
||||
BACKEND_UPDATE=true
|
||||
FRONTEND_UPDATE=true
|
||||
COMPONENT_CHANGED="both"
|
||||
# Use the higher version as source
|
||||
if [ "$(printf '%s\n' "$BACKEND_VERSION" "$FRONTEND_VERSION" | sort -V | tail -n1)" = "$BACKEND_VERSION" ]; then
|
||||
SOURCE_VERSION=$BACKEND_VERSION
|
||||
else
|
||||
SOURCE_VERSION=$FRONTEND_VERSION
|
||||
fi
|
||||
echo "Both backend and frontend changed - updating both"
|
||||
elif [ "$BACKEND_CHANGED" -gt 0 ]; then
|
||||
# Only backend changed
|
||||
BACKEND_UPDATE=true
|
||||
COMPONENT_CHANGED="backend"
|
||||
SOURCE_VERSION=$BACKEND_VERSION
|
||||
echo "Only backend changed - updating backend"
|
||||
elif [ "$FRONTEND_CHANGED" -gt 0 ]; then
|
||||
# Only frontend changed
|
||||
FRONTEND_UPDATE=true
|
||||
COMPONENT_CHANGED="frontend"
|
||||
SOURCE_VERSION=$FRONTEND_VERSION
|
||||
echo "Only frontend changed - updating frontend"
|
||||
else
|
||||
echo "No relevant changes detected"
|
||||
echo "version_changed=false" >> $GITHUB_OUTPUT
|
||||
echo "component_changed=none" >> $GITHUB_OUTPUT
|
||||
echo "new_version=" >> $GITHUB_OUTPUT
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "Component changed: $COMPONENT_CHANGED"
|
||||
echo "Source version: $SOURCE_VERSION"
|
||||
echo "Backend update: $BACKEND_UPDATE"
|
||||
echo "Frontend update: $FRONTEND_UPDATE"
|
||||
|
||||
# Calculate new version
|
||||
IFS='.' read -r -a version_parts <<< "$SOURCE_VERSION"
|
||||
MAJOR="${version_parts[0]}"
|
||||
MINOR="${version_parts[1]}"
|
||||
PATCH="${version_parts[2]}"
|
||||
@@ -49,14 +139,23 @@ jobs:
|
||||
|
||||
echo "New version: $NEW_VERSION"
|
||||
echo "new_version=$NEW_VERSION" >> $GITHUB_OUTPUT
|
||||
echo "component_changed=$COMPONENT_CHANGED" >> $GITHUB_OUTPUT
|
||||
|
||||
# Update version in package.json files
|
||||
cd backend && npm version $NEW_VERSION --no-git-tag-version
|
||||
cd ../frontend && npm version $NEW_VERSION --no-git-tag-version
|
||||
cd ..
|
||||
# Update versions in package.json files
|
||||
if [ "$BACKEND_UPDATE" = true ]; then
|
||||
echo "Updating backend version to $NEW_VERSION"
|
||||
cd backend && npm version $NEW_VERSION --no-git-tag-version
|
||||
cd ..
|
||||
fi
|
||||
|
||||
# Check if there are changes
|
||||
if [[ -n $(git status -s) ]]; then
|
||||
if [ "$FRONTEND_UPDATE" = true ]; then
|
||||
echo "Updating frontend version to $NEW_VERSION"
|
||||
cd frontend && npm version $NEW_VERSION --no-git-tag-version
|
||||
cd ..
|
||||
fi
|
||||
|
||||
# Check if there are changes to commit
|
||||
if [[ -n $(git status --porcelain) ]]; then
|
||||
echo "version_changed=true" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "version_changed=false" >> $GITHUB_OUTPUT
|
||||
@@ -65,15 +164,94 @@ jobs:
|
||||
- name: Commit version bump
|
||||
if: steps.version.outputs.version_changed == 'true'
|
||||
run: |
|
||||
git add backend/package.json backend/package-lock.json
|
||||
git add frontend/package.json frontend/package-lock.json
|
||||
git commit -m "chore: bump version to ${{ steps.version.outputs.new_version }}"
|
||||
git push
|
||||
set -e # Exit on any error
|
||||
|
||||
# First, ensure we have the latest changes
|
||||
echo "Fetching latest changes..."
|
||||
git fetch origin main
|
||||
|
||||
# Check if we're behind and need to update
|
||||
LOCAL=$(git rev-parse HEAD)
|
||||
REMOTE=$(git rev-parse origin/main)
|
||||
|
||||
if [ "$LOCAL" != "$REMOTE" ]; then
|
||||
echo "Local is behind remote, pulling changes..."
|
||||
git pull origin main --no-rebase
|
||||
fi
|
||||
|
||||
COMPONENT="${{ steps.version.outputs.component_changed }}"
|
||||
|
||||
if [ "$COMPONENT" = "both" ]; then
|
||||
git add backend/package.json backend/package-lock.json frontend/package.json frontend/package-lock.json
|
||||
git commit -m "chore: bump version to ${{ steps.version.outputs.new_version }} (backend + frontend)"
|
||||
elif [ "$COMPONENT" = "backend" ]; then
|
||||
git add backend/package.json backend/package-lock.json
|
||||
git commit -m "chore: bump backend version to ${{ steps.version.outputs.new_version }}"
|
||||
elif [ "$COMPONENT" = "frontend" ]; then
|
||||
git add frontend/package.json frontend/package-lock.json
|
||||
git commit -m "chore: bump frontend version to ${{ steps.version.outputs.new_version }}"
|
||||
fi
|
||||
|
||||
# Pull latest changes before pushing to avoid conflicts
|
||||
echo "Pulling latest changes from origin/main..."
|
||||
if ! git pull --rebase origin main; then
|
||||
echo "Rebase failed, attempting to resolve..."
|
||||
# If rebase fails, abort and try a regular merge
|
||||
git rebase --abort || true
|
||||
git pull origin main --no-rebase
|
||||
fi
|
||||
|
||||
# Push the changes with retry logic
|
||||
echo "Pushing version bump..."
|
||||
PUSH_SUCCESS=false
|
||||
|
||||
for i in 1 2 3; do
|
||||
echo "Push attempt $i of 3..."
|
||||
|
||||
# Try to push
|
||||
if git push origin main 2>&1; then
|
||||
echo "Successfully pushed version bump on attempt $i"
|
||||
PUSH_SUCCESS=true
|
||||
break
|
||||
else
|
||||
echo "Push failed on attempt $i"
|
||||
|
||||
if [ $i -lt 3 ]; then
|
||||
echo "Waiting 5 seconds before retry..."
|
||||
sleep 5
|
||||
|
||||
echo "Pulling latest changes..."
|
||||
git fetch origin main
|
||||
|
||||
# Try rebase first, fall back to merge
|
||||
if ! git rebase origin/main; then
|
||||
echo "Rebase failed, trying merge..."
|
||||
git rebase --abort 2>/dev/null || true
|
||||
git pull origin main --no-rebase
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
if [ "$PUSH_SUCCESS" = "false" ]; then
|
||||
echo "ERROR: Failed to push after 3 attempts"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Create Git tag
|
||||
if: steps.version.outputs.version_changed == 'true'
|
||||
run: |
|
||||
git tag -a "v${{ steps.version.outputs.new_version }}" -m "Release v${{ steps.version.outputs.new_version }}"
|
||||
COMPONENT="${{ steps.version.outputs.component_changed }}"
|
||||
|
||||
if [ "$COMPONENT" = "both" ]; then
|
||||
TAG_MESSAGE="Release v${{ steps.version.outputs.new_version }} (backend + frontend)"
|
||||
elif [ "$COMPONENT" = "backend" ]; then
|
||||
TAG_MESSAGE="Release v${{ steps.version.outputs.new_version }} (backend)"
|
||||
elif [ "$COMPONENT" = "frontend" ]; then
|
||||
TAG_MESSAGE="Release v${{ steps.version.outputs.new_version }} (frontend)"
|
||||
fi
|
||||
|
||||
git tag -a "v${{ steps.version.outputs.new_version }}" -m "$TAG_MESSAGE"
|
||||
git push origin "v${{ steps.version.outputs.new_version }}"
|
||||
|
||||
trigger-drone:
|
||||
@@ -84,5 +262,6 @@ jobs:
|
||||
- name: Trigger Drone Build
|
||||
run: |
|
||||
echo "Version bumped to ${{ needs.version-bump.outputs.new_version }}"
|
||||
echo "Component(s) changed: ${{ needs.version-bump.outputs.component_changed }}"
|
||||
echo "Drone will automatically trigger on the new tag"
|
||||
# Drone CI will automatically trigger on the tag push event
|
||||
@@ -1,24 +0,0 @@
|
||||
# Exclude patterns for GitHub mirror
|
||||
.env
|
||||
.env.*
|
||||
.env*
|
||||
docker-compose.prod.yml
|
||||
docker-compose.traefik.yml
|
||||
.claudedocs/
|
||||
backend/data/
|
||||
backend/storage/
|
||||
backend/.env*
|
||||
frontend/.env*
|
||||
secrets/
|
||||
*.key
|
||||
*.pem
|
||||
.gitea/
|
||||
node_modules/
|
||||
dist/
|
||||
build/
|
||||
*.log
|
||||
.DS_Store
|
||||
deploy/
|
||||
certbot/
|
||||
nginx/
|
||||
photo-sharing-prd.md
|
||||
+18
@@ -11,6 +11,9 @@ yarn-error.log*
|
||||
.env.test.local
|
||||
.env.production.local
|
||||
|
||||
# Docker override file
|
||||
docker-compose.override.yml
|
||||
|
||||
# Security - Never commit credentials
|
||||
ADMIN_CREDENTIALS.txt
|
||||
ADMIN_PASSWORD_RESET.txt
|
||||
@@ -48,9 +51,24 @@ coverage/
|
||||
*.tmp
|
||||
*.temp
|
||||
|
||||
# Backup and test directories
|
||||
backups/
|
||||
test-archiver/
|
||||
|
||||
# Keep directory structure
|
||||
!storage/events/active/.gitkeep
|
||||
!storage/events/archived/.gitkeep
|
||||
!storage/thumbnails/.gitkeep
|
||||
!data/.gitkeep
|
||||
!logs/.gitkeep
|
||||
|
||||
# development files
|
||||
backend/.swarm/
|
||||
.claudedocs/
|
||||
backend/data/
|
||||
backend/docs/
|
||||
backend/logs/
|
||||
logs/
|
||||
storage/
|
||||
data/
|
||||
certbot/
|
||||
|
||||
@@ -33,11 +33,20 @@ npm test -- path/to/test.test.js
|
||||
npm test -- --testNamePattern="test name"
|
||||
```
|
||||
|
||||
### Production
|
||||
```bash
|
||||
docker-compose -f docker-compose.prod.yml up -d # Production deployment
|
||||
pm2 start ecosystem.config.js # Alternative: PM2 deployment
|
||||
```
|
||||
### Production Deployment
|
||||
See [DEPLOYMENT_GUIDE.md](./DEPLOYMENT_GUIDE.md) for comprehensive deployment instructions including:
|
||||
- Docker Compose deployment
|
||||
- PM2 deployment
|
||||
- Manual installation
|
||||
- Non-nginx deployment options
|
||||
- SSL/HTTPS setup
|
||||
- Troubleshooting guide
|
||||
|
||||
**⚠️ CRITICAL PRODUCTION NOTICE:**
|
||||
- Production runs on a SEPARATE SERVER - never assume local changes affect production
|
||||
- ALWAYS request production server details before any troubleshooting
|
||||
- NO trial-and-error approaches in production - data loss is unacceptable
|
||||
- Every change must be thoroughly analyzed and tested locally first
|
||||
|
||||
## Key Product Requirements (from PRD)
|
||||
|
||||
@@ -111,6 +120,7 @@ Background services run as separate processes:
|
||||
- **archiveService**: Creates ZIP archives of expired events
|
||||
- **expirationChecker**: Cron job for expiration warnings
|
||||
- **fileWatcher**: Monitors for new photo uploads
|
||||
- **backupService**: Scheduled backups with checksum-based change detection
|
||||
|
||||
### API Structure
|
||||
- `/api/admin/*` - Admin panel endpoints (requires adminAuth)
|
||||
@@ -127,6 +137,41 @@ Background services run as separate processes:
|
||||
5. **Frontend Status**: Only skeleton exists - requires full implementation based on PRD
|
||||
6. **Umami Analytics**: Track password entries, downloads, views, expiration warnings
|
||||
|
||||
## Troubleshooting Guidelines
|
||||
|
||||
### Before ANY Production Troubleshooting:
|
||||
1. **ALWAYS request specific details**:
|
||||
- Production server URL/IP
|
||||
- Current error messages/logs
|
||||
- Recent changes or deployments
|
||||
- Affected users/galleries
|
||||
- Time of issue occurrence
|
||||
|
||||
2. **Thorough Analysis Required**:
|
||||
- Use detailed thinking/analysis for EVERY troubleshooting task
|
||||
- Review all related code before suggesting changes
|
||||
- Consider all potential side effects
|
||||
- Never make assumptions about production environment
|
||||
|
||||
3. **Safe Troubleshooting Steps**:
|
||||
- First, reproduce issue in local/dev environment
|
||||
- Analyze logs without modifying production
|
||||
- Create detailed action plan before any changes
|
||||
- Always have rollback strategy ready
|
||||
- Document every step taken
|
||||
|
||||
### Common Issues & Safe Approaches:
|
||||
- **Email not sending**: Check email_queue table, SMTP settings, service status
|
||||
- **Photos not loading**: Verify file permissions, storage paths, nginx config
|
||||
- **Gallery access issues**: Check JWT tokens, expiration dates, access_logs
|
||||
- **Performance problems**: Analyze with monitoring tools first, never experiment
|
||||
|
||||
### Data Safety Rules:
|
||||
- NEVER delete or modify production data without explicit backup confirmation
|
||||
- ALWAYS verify backups exist before any data operations
|
||||
- NO direct database modifications without transaction safety
|
||||
- Log all actions for audit trail
|
||||
|
||||
## Environment Variables
|
||||
|
||||
### Backend (.env)
|
||||
@@ -253,9 +298,71 @@ const { theme, setTheme, setThemeByName } = useTheme();
|
||||
--border-radius: 0.5rem;
|
||||
```
|
||||
|
||||
## Backup Service
|
||||
|
||||
### Overview
|
||||
The backup service provides automated, scheduled backups of all photo data with checksum-based change detection to minimize transfer overhead.
|
||||
|
||||
### Features
|
||||
- **Multiple Destinations**: Local directory, remote server (rsync), S3-compatible storage
|
||||
- **Change Detection**: SHA256 checksums track file changes, only modified files are backed up
|
||||
- **Scheduled Execution**: Configurable cron-based scheduling (default: 2 AM daily)
|
||||
- **Email Notifications**: Alerts on backup failure, optional success notifications
|
||||
- **Retention Management**: Automatic cleanup of old backup runs based on retention policy
|
||||
- **Progress Tracking**: Database storage of backup history, file states, and statistics
|
||||
|
||||
### Configuration
|
||||
Backup settings are stored in `app_settings` table with `backup_` prefix:
|
||||
- `backup_enabled`: Enable/disable the service
|
||||
- `backup_schedule`: Cron expression (e.g., '0 2 * * *')
|
||||
- `backup_destination_type`: 'local', 'rsync', or 's3'
|
||||
- `backup_retention_days`: How long to keep backup history
|
||||
- `backup_include_archived`: Whether to backup archived events
|
||||
- `backup_exclude_patterns`: File patterns to exclude
|
||||
|
||||
### API Endpoints
|
||||
- `GET /api/admin/backup/config` - Get current configuration
|
||||
- `PUT /api/admin/backup/config` - Update configuration
|
||||
- `GET /api/admin/backup/status` - Get backup status and history
|
||||
- `POST /api/admin/backup/run` - Trigger manual backup
|
||||
- `POST /api/admin/backup/test-connection` - Test destination connectivity
|
||||
|
||||
### Testing
|
||||
Run backup service test: `npm run test-backup`
|
||||
|
||||
### Database Tables
|
||||
- `backup_runs`: Tracks each backup execution with statistics
|
||||
- `backup_file_states`: Stores file checksums for change detection
|
||||
|
||||
## Success Metrics (from PRD)
|
||||
- Time to generate gallery: <2 minutes
|
||||
- Guest satisfaction: >90%
|
||||
- System uptime: 99.9%
|
||||
- Email delivery rate: >98%
|
||||
- Successful archiving: 100%
|
||||
- Successful archiving: 100%
|
||||
|
||||
## Documentation & Development Practices
|
||||
|
||||
### Documentation Guidelines:
|
||||
- **NEVER create new documentation files for simple tasks**
|
||||
- **ALWAYS update existing documentation (like this CLAUDE.md)**
|
||||
- Only create new .md files when explicitly requested
|
||||
- Avoid creating temporary scripts for one-off tasks
|
||||
|
||||
### Development Best Practices:
|
||||
- Test all changes thoroughly in local environment first
|
||||
- Use version control for all changes
|
||||
- Keep commits atomic and well-described
|
||||
- Review impact on all integrated services
|
||||
- Consider backward compatibility
|
||||
- Update tests when changing functionality
|
||||
|
||||
### Production Deployment Checklist:
|
||||
- [ ] All tests passing locally
|
||||
- [ ] Linting and type checks pass
|
||||
- [ ] Database migrations tested with rollback plan
|
||||
- [ ] Environment variables documented
|
||||
- [ ] Backup strategy confirmed
|
||||
- [ ] Monitoring alerts configured
|
||||
- [ ] Rollback procedure documented
|
||||
- [ ] Stakeholders notified of maintenance window
|
||||
+1
-1
@@ -20,7 +20,7 @@ We are committed to providing a welcoming and inspiring community for all photog
|
||||
|
||||
## Enforcement
|
||||
|
||||
Instances of unacceptable behavior may be reported to the project team at conduct@example.com. All complaints will be reviewed and investigated promptly and fairly.
|
||||
Instances of unacceptable behavior may be reported by [opening an issue](https://github.com/the-luap/picpeak/issues/new?labels=conduct) on GitHub. All complaints will be reviewed and investigated promptly and fairly.
|
||||
|
||||
## Attribution
|
||||
|
||||
|
||||
+3
-3
@@ -153,8 +153,8 @@ picpeak/
|
||||
|
||||
## 📮 Contact
|
||||
|
||||
- Create an issue for bugs or features
|
||||
- Join discussions for questions
|
||||
- Email: picpeak@example.com for security issues
|
||||
- Create an [issue](https://github.com/the-luap/picpeak/issues) for bugs or features
|
||||
- Join [discussions](https://github.com/the-luap/picpeak/discussions) for questions
|
||||
- Security issues: Open a [security issue](https://github.com/the-luap/picpeak/issues/new?labels=security) on GitHub
|
||||
|
||||
Thank you for contributing! 🎉
|
||||
-220
@@ -1,220 +0,0 @@
|
||||
# 🚀 PicPeak Deployment Guide
|
||||
|
||||
This guide will help you deploy PicPeak in production. The entire process takes about 10-15 minutes.
|
||||
|
||||
## 📋 Prerequisites
|
||||
|
||||
- A server with Docker and Docker Compose installed
|
||||
- A domain name (for SSL certificates)
|
||||
- SMTP credentials for sending emails
|
||||
- Basic command line knowledge
|
||||
|
||||
## 🏃 Quick Deploy (Recommended)
|
||||
|
||||
### 1. Clone and Configure
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone https://github.com/the-luap/picpeak.git
|
||||
cd picpeak
|
||||
|
||||
# Copy environment template
|
||||
cp .env.production.example .env
|
||||
|
||||
# Generate a secure JWT secret
|
||||
echo "JWT_SECRET=$(openssl rand -base64 32)" >> .env
|
||||
|
||||
# Edit configuration
|
||||
nano .env
|
||||
```
|
||||
|
||||
### 2. Required Environment Variables
|
||||
|
||||
Edit your `.env` file with these essential settings:
|
||||
|
||||
```env
|
||||
# Application URLs
|
||||
FRONTEND_URL=https://your-domain.com
|
||||
BACKEND_URL=https://your-domain.com
|
||||
|
||||
# Email Configuration (Required for notifications)
|
||||
SMTP_HOST=smtp.gmail.com
|
||||
SMTP_PORT=587
|
||||
SMTP_USER=your-email@gmail.com
|
||||
SMTP_PASS=your-app-password
|
||||
SMTP_FROM=your-email@gmail.com
|
||||
|
||||
# Admin Configuration
|
||||
ADMIN_EMAIL=admin@your-domain.com
|
||||
ADMIN_PASSWORD=your-secure-password
|
||||
|
||||
# Database (PostgreSQL for production)
|
||||
DATABASE_CLIENT=pg
|
||||
DB_HOST=postgres
|
||||
DB_NAME=picpeak
|
||||
DB_USER=picpeak
|
||||
DB_PASSWORD=secure-db-password
|
||||
```
|
||||
|
||||
### 3. Deploy with Docker Compose
|
||||
|
||||
```bash
|
||||
# Start all services
|
||||
docker-compose -f docker-compose.prod.yml up -d
|
||||
|
||||
# Check logs
|
||||
docker-compose logs -f
|
||||
|
||||
# Access your site at https://your-domain.com
|
||||
```
|
||||
|
||||
## 🔧 Configuration Options
|
||||
|
||||
### Storage Settings
|
||||
|
||||
```env
|
||||
# Storage paths (default: ./storage)
|
||||
STORAGE_PATH=./storage
|
||||
ARCHIVE_PATH=./storage/archives
|
||||
|
||||
# Gallery expiration (days)
|
||||
DEFAULT_EXPIRATION_DAYS=30
|
||||
WARNING_DAYS_BEFORE_EXPIRY=7
|
||||
```
|
||||
|
||||
### Security Settings
|
||||
|
||||
```env
|
||||
# Session timeout (minutes)
|
||||
SESSION_TIMEOUT=60
|
||||
|
||||
# Rate limiting
|
||||
RATE_LIMIT_WINDOW_MS=900000 # 15 minutes
|
||||
RATE_LIMIT_MAX_REQUESTS=100
|
||||
```
|
||||
|
||||
### Analytics (Optional)
|
||||
|
||||
```env
|
||||
# Umami Analytics
|
||||
VITE_UMAMI_URL=https://analytics.your-domain.com
|
||||
VITE_UMAMI_WEBSITE_ID=your-website-id
|
||||
```
|
||||
|
||||
## 🔒 SSL/TLS Setup
|
||||
|
||||
The production Docker Compose includes automatic SSL via Let's Encrypt:
|
||||
|
||||
1. **Ensure your domain points to your server**
|
||||
2. **Update nginx configuration**:
|
||||
```bash
|
||||
nano nginx/nginx.conf
|
||||
# Replace your-domain.com with your actual domain
|
||||
```
|
||||
3. **Start services** - Certbot will automatically obtain certificates
|
||||
|
||||
## 📁 Directory Structure
|
||||
|
||||
After deployment, your directory structure will be:
|
||||
|
||||
```
|
||||
picpeak/
|
||||
├── backend/ # API server
|
||||
├── frontend/ # React app
|
||||
├── storage/ # Photo storage
|
||||
│ ├── events/ # Active galleries
|
||||
│ │ ├── active/ # Current photos
|
||||
│ │ └── archived/ # Expired galleries
|
||||
│ ├── thumbnails/ # Generated thumbnails
|
||||
│ └── uploads/ # User uploads
|
||||
├── data/ # Database files
|
||||
└── logs/ # Application logs
|
||||
```
|
||||
|
||||
## 🔄 Maintenance
|
||||
|
||||
### Backup
|
||||
|
||||
```bash
|
||||
# Backup database and photos
|
||||
./scripts/backup.sh
|
||||
|
||||
# Backups are stored in ./backups/
|
||||
```
|
||||
|
||||
### Update
|
||||
|
||||
```bash
|
||||
# Pull latest changes
|
||||
git pull
|
||||
|
||||
# Rebuild and restart
|
||||
docker-compose -f docker-compose.prod.yml up -d --build
|
||||
```
|
||||
|
||||
### Logs
|
||||
|
||||
```bash
|
||||
# View all logs
|
||||
docker-compose logs
|
||||
|
||||
# View specific service
|
||||
docker-compose logs backend
|
||||
docker-compose logs frontend
|
||||
```
|
||||
|
||||
## 🚨 Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**Photos not appearing:**
|
||||
- Check storage permissions: `chmod -R 755 storage/`
|
||||
- Verify file watcher is running: `docker-compose logs backend | grep watcher`
|
||||
|
||||
**Email not sending:**
|
||||
- Test SMTP settings: Admin Panel → Settings → Email → Send Test
|
||||
- Check email queue: Admin Panel → System → Email Queue
|
||||
|
||||
**Can't access admin panel:**
|
||||
- Default login: Use email/password from `.env`
|
||||
- Reset password: `docker exec picpeak-backend npm run reset-admin`
|
||||
|
||||
### Health Check
|
||||
|
||||
```bash
|
||||
# Check service status
|
||||
docker-compose ps
|
||||
|
||||
# Test backend API
|
||||
curl https://your-domain.com/api/health
|
||||
|
||||
# Check disk space
|
||||
df -h storage/
|
||||
```
|
||||
|
||||
## 🐳 Alternative Deployment Methods
|
||||
|
||||
### Using Docker Swarm
|
||||
|
||||
For high availability deployments, see [Docker Swarm Setup](deploy/README.md).
|
||||
|
||||
### Manual Installation
|
||||
|
||||
If you prefer not to use Docker:
|
||||
|
||||
1. Install Node.js 18+
|
||||
2. Install PostgreSQL
|
||||
3. Clone repository
|
||||
4. Install dependencies: `npm install` in both `/backend` and `/frontend`
|
||||
5. Build frontend: `cd frontend && npm run build`
|
||||
6. Start services with PM2
|
||||
|
||||
## 📞 Support
|
||||
|
||||
- 📘 [Documentation](https://github.com/the-luap/picpeak)
|
||||
- 🐛 [Report Issues](https://github.com/the-luap/picpeak/issues)
|
||||
- 💬 [Discussions](https://github.com/the-luap/picpeak/discussions)
|
||||
|
||||
---
|
||||
|
||||
**Need help?** Open an issue on GitHub and we'll assist you!
|
||||
@@ -0,0 +1,461 @@
|
||||
# 🚀 PicPeak Deployment Guide
|
||||
|
||||
This guide covers deploying PicPeak using Docker Compose with direct port exposure. For internet-facing deployments, you'll need to add a reverse proxy (nginx, Traefik, Caddy, etc.) for SSL/HTTPS.
|
||||
|
||||
## 📋 Table of Contents
|
||||
|
||||
- [Prerequisites](#prerequisites)
|
||||
- [Quick Start](#quick-start)
|
||||
- [Configuration](#configuration)
|
||||
- [Deployment](#deployment)
|
||||
- [Reverse Proxy Setup](#reverse-proxy-setup)
|
||||
- [Maintenance](#maintenance)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Docker and Docker Compose installed
|
||||
- Domain name (for production)
|
||||
- SMTP server credentials for emails
|
||||
- At least 2GB RAM and 20GB storage
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
1. **Clone the repository**
|
||||
```bash
|
||||
git clone https://github.com/the-luap/picpeak.git
|
||||
cd picpeak
|
||||
```
|
||||
|
||||
2. **Set up environment**
|
||||
```bash
|
||||
cp .env.example .env
|
||||
nano .env # Edit with your values
|
||||
```
|
||||
|
||||
3. **Create required directories**
|
||||
```bash
|
||||
mkdir -p events/active events/archived data logs backup storage
|
||||
chmod -R 755 events data logs backup storage
|
||||
```
|
||||
|
||||
4. **Deploy**
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
5. **Check logs**
|
||||
```bash
|
||||
docker compose logs -f
|
||||
```
|
||||
|
||||
## 🔧 Configuration
|
||||
|
||||
### Essential Environment Variables
|
||||
|
||||
Generate secure values:
|
||||
```bash
|
||||
# JWT Secret
|
||||
openssl rand -base64 64
|
||||
|
||||
# Database Password
|
||||
openssl rand -base64 32
|
||||
|
||||
# Redis Password
|
||||
openssl rand -base64 32
|
||||
```
|
||||
|
||||
Update `.env` with:
|
||||
- `JWT_SECRET` - Authentication secret
|
||||
- `DB_PASSWORD` - PostgreSQL password
|
||||
- `REDIS_PASSWORD` - Redis password
|
||||
- `SMTP_*` - Email configuration
|
||||
- `FRONTEND_URL` - Your domain URL
|
||||
- `ADMIN_URL` - Backend admin URL
|
||||
- `VITE_API_URL` - API URL for frontend
|
||||
|
||||
### Email Configuration Examples
|
||||
|
||||
#### Gmail
|
||||
```env
|
||||
SMTP_HOST=smtp.gmail.com
|
||||
SMTP_PORT=587
|
||||
SMTP_SECURE=false
|
||||
SMTP_USER=your-email@gmail.com
|
||||
SMTP_PASS=your-app-specific-password
|
||||
```
|
||||
|
||||
#### SendGrid
|
||||
```env
|
||||
SMTP_HOST=smtp.sendgrid.net
|
||||
SMTP_PORT=587
|
||||
SMTP_SECURE=false
|
||||
SMTP_USER=apikey
|
||||
SMTP_PASS=your-sendgrid-api-key
|
||||
```
|
||||
|
||||
## 📦 Deployment
|
||||
|
||||
### Build and Start Services
|
||||
|
||||
```bash
|
||||
# Build images
|
||||
docker compose build
|
||||
|
||||
# Start all services
|
||||
docker compose up -d
|
||||
|
||||
# View running containers
|
||||
docker compose ps
|
||||
```
|
||||
|
||||
### Access Points
|
||||
|
||||
By default, services are exposed on:
|
||||
- Frontend: http://localhost:3000
|
||||
- Backend/API: http://localhost:3001
|
||||
- PostgreSQL: localhost:5432 (if needed)
|
||||
- Redis: localhost:6379 (if needed)
|
||||
|
||||
### Initial Admin Setup
|
||||
|
||||
When deploying for the first time, an admin account is automatically created with a secure random password. You need to retrieve this password to access the admin panel.
|
||||
|
||||
#### Finding the Admin Password
|
||||
|
||||
**Option 1: Check the backend logs** (recommended)
|
||||
```bash
|
||||
# View the initial setup logs
|
||||
docker compose logs backend | grep -A 10 "Admin user created"
|
||||
```
|
||||
|
||||
You should see output like:
|
||||
```
|
||||
========================================
|
||||
✅ Admin user created successfully!
|
||||
========================================
|
||||
Email: admin@example.com
|
||||
Password: BraveTiger6231!
|
||||
|
||||
⚠️ IMPORTANT:
|
||||
1. Save these credentials securely
|
||||
2. Please change the password after first login
|
||||
========================================
|
||||
```
|
||||
|
||||
**Note**: You login with the **email address**, not a username!
|
||||
|
||||
**Option 2: Check the saved credentials file**
|
||||
```bash
|
||||
# The password is saved in the backend container
|
||||
docker exec picpeak-backend cat data/ADMIN_CREDENTIALS.txt
|
||||
```
|
||||
|
||||
**Option 3: Use the helper script**
|
||||
```bash
|
||||
# Show current admin username and email (password is hidden)
|
||||
docker exec picpeak-backend node scripts/show-admin-credentials.js
|
||||
|
||||
# Reset the admin password to a new random password
|
||||
docker exec picpeak-backend node scripts/show-admin-credentials.js --reset
|
||||
```
|
||||
|
||||
#### Important Notes
|
||||
|
||||
- **Login requires the email address**, not username
|
||||
- The admin password is only shown once during initial setup
|
||||
- If you lose the password, use the `--reset` option to generate a new one
|
||||
- You must change the password on first login (enforced by the system)
|
||||
- Password requirements: minimum 12 characters, mixed case, numbers, and special characters
|
||||
|
||||
#### Configuring Admin Email
|
||||
|
||||
By default, the admin email is `admin@example.com`. To use a different email address, set it in your `.env` file before first deployment:
|
||||
|
||||
```env
|
||||
# .env
|
||||
ADMIN_EMAIL=your-email@yourdomain.com
|
||||
```
|
||||
|
||||
**Note**: This only works on first deployment. To change the admin email after deployment, you'll need to update it in the database or create a new admin user through the admin panel.
|
||||
|
||||
## 🔒 Reverse Proxy Setup
|
||||
|
||||
For production deployments, you should use a reverse proxy for SSL/HTTPS. The application exposes ports directly, allowing you to use any reverse proxy solution.
|
||||
|
||||
### Option 1: Nginx
|
||||
|
||||
Install nginx and create `/etc/nginx/sites-available/picpeak`:
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name your-domain.com;
|
||||
return 301 https://$server_name$request_uri;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 443 ssl http2;
|
||||
server_name your-domain.com;
|
||||
|
||||
ssl_certificate /etc/letsencrypt/live/your-domain.com/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/your-domain.com/privkey.pem;
|
||||
|
||||
# Frontend
|
||||
location / {
|
||||
proxy_pass http://localhost:3000;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
|
||||
# Backend API
|
||||
location /api {
|
||||
proxy_pass http://localhost:3001;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
|
||||
# Protected photos and uploads
|
||||
location ~ ^/(photos|thumbnails|uploads) {
|
||||
proxy_pass http://localhost:3001;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
}
|
||||
|
||||
# Admin routes
|
||||
location /admin {
|
||||
proxy_pass http://localhost:3001;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Enable the site:
|
||||
```bash
|
||||
sudo ln -s /etc/nginx/sites-available/picpeak /etc/nginx/sites-enabled/
|
||||
sudo nginx -t
|
||||
sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
### Option 2: Traefik
|
||||
|
||||
Add labels to `docker-compose.override.yml`:
|
||||
|
||||
```yaml
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
frontend:
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.picpeak.rule=Host(`your-domain.com`)"
|
||||
- "traefik.http.routers.picpeak.entrypoints=websecure"
|
||||
- "traefik.http.routers.picpeak.tls.certresolver=letsencrypt"
|
||||
- "traefik.http.services.picpeak.loadbalancer.server.port=80"
|
||||
|
||||
backend:
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.picpeak-api.rule=Host(`your-domain.com`) && PathPrefix(`/api`)"
|
||||
- "traefik.http.routers.picpeak-api.entrypoints=websecure"
|
||||
- "traefik.http.routers.picpeak-api.tls.certresolver=letsencrypt"
|
||||
- "traefik.http.services.picpeak-api.loadbalancer.server.port=3001"
|
||||
```
|
||||
|
||||
### Option 3: Caddy
|
||||
|
||||
Create a `Caddyfile`:
|
||||
|
||||
```caddyfile
|
||||
your-domain.com {
|
||||
# Frontend
|
||||
handle /* {
|
||||
reverse_proxy localhost:3000
|
||||
}
|
||||
|
||||
# Backend API and admin
|
||||
handle /api/* {
|
||||
reverse_proxy localhost:3001
|
||||
}
|
||||
|
||||
handle /admin/* {
|
||||
reverse_proxy localhost:3001
|
||||
}
|
||||
|
||||
# Protected resources
|
||||
handle /photos/* {
|
||||
reverse_proxy localhost:3001
|
||||
}
|
||||
|
||||
handle /thumbnails/* {
|
||||
reverse_proxy localhost:3001
|
||||
}
|
||||
|
||||
handle /uploads/* {
|
||||
reverse_proxy localhost:3001
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### SSL Certificates
|
||||
|
||||
For any reverse proxy, you can use Let's Encrypt:
|
||||
|
||||
```bash
|
||||
# With Certbot
|
||||
sudo certbot certonly --webroot -w /var/www/certbot -d your-domain.com
|
||||
|
||||
# Or use your reverse proxy's built-in ACME support
|
||||
```
|
||||
|
||||
## 🔧 Maintenance
|
||||
|
||||
### Viewing Logs
|
||||
|
||||
```bash
|
||||
# All services
|
||||
docker compose logs -f
|
||||
|
||||
# Specific service
|
||||
docker compose logs -f backend
|
||||
docker compose logs -f frontend
|
||||
```
|
||||
|
||||
### Backup
|
||||
|
||||
#### Manual Backup
|
||||
```bash
|
||||
# Database backup
|
||||
docker exec picpeak-postgres pg_dump -U picpeak picpeak_prod > backup/db_$(date +%Y%m%d_%H%M%S).sql
|
||||
|
||||
# Files backup
|
||||
tar -czf backup/photos_$(date +%Y%m%d_%H%M%S).tar.gz events/
|
||||
```
|
||||
|
||||
#### Automated Backup
|
||||
The application includes a built-in backup service. Configure it in the admin panel:
|
||||
1. Login to admin panel
|
||||
2. Go to Settings → Backup
|
||||
3. Configure destination and schedule
|
||||
4. Enable backup service
|
||||
|
||||
### Updates
|
||||
|
||||
```bash
|
||||
# Pull latest changes
|
||||
git pull
|
||||
|
||||
# Rebuild and restart
|
||||
docker compose down
|
||||
docker compose build
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
### Database Migrations
|
||||
|
||||
Migrations run automatically on startup, but you can run them manually:
|
||||
|
||||
```bash
|
||||
docker exec picpeak-backend npm run migrate
|
||||
```
|
||||
|
||||
## 🚨 Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
#### Port Already in Use
|
||||
```bash
|
||||
# Check what's using the port
|
||||
sudo lsof -i :3000
|
||||
sudo lsof -i :3001
|
||||
|
||||
# Change ports in .env
|
||||
FRONTEND_PORT=3002
|
||||
BACKEND_PORT=3003
|
||||
```
|
||||
|
||||
#### Permission Errors
|
||||
```bash
|
||||
# Fix ownership
|
||||
sudo chown -R 1000:1000 events data logs backup storage
|
||||
chmod -R 755 events data logs backup storage
|
||||
```
|
||||
|
||||
#### Database Connection Issues
|
||||
```bash
|
||||
# Check if database is running
|
||||
docker compose ps
|
||||
docker compose logs postgres
|
||||
|
||||
# Test connection
|
||||
docker exec picpeak-postgres pg_isready
|
||||
```
|
||||
|
||||
#### Email Not Sending
|
||||
- Verify SMTP settings in .env
|
||||
- Check email queue: `docker exec picpeak-backend psql -U picpeak -d picpeak_prod -c "SELECT * FROM email_queue ORDER BY created_at DESC LIMIT 10;"`
|
||||
- For Gmail, use app-specific password
|
||||
- Check logs: `docker compose logs backend | grep email`
|
||||
|
||||
### Health Checks
|
||||
|
||||
```bash
|
||||
# Backend health
|
||||
curl http://localhost:3001/api/health
|
||||
|
||||
# Frontend health
|
||||
curl http://localhost:3000
|
||||
|
||||
# Database health
|
||||
docker exec picpeak-postgres pg_isready
|
||||
```
|
||||
|
||||
### Useful Commands
|
||||
|
||||
```bash
|
||||
# Enter backend container
|
||||
docker exec -it picpeak-backend sh
|
||||
|
||||
# Enter database
|
||||
docker exec -it picpeak-postgres psql -U picpeak picpeak_prod
|
||||
|
||||
# Reset admin password
|
||||
docker exec picpeak-backend node scripts/show-admin-credentials.js --reset
|
||||
|
||||
# Check disk usage
|
||||
df -h
|
||||
du -sh events/ storage/ backup/
|
||||
|
||||
# View running processes
|
||||
docker compose top
|
||||
```
|
||||
|
||||
## Security Recommendations
|
||||
|
||||
1. **Use HTTPS**: Always use a reverse proxy with SSL in production
|
||||
2. **Firewall**: Only expose necessary ports (80, 443)
|
||||
3. **Secure passwords**: Use strong, unique passwords for all services
|
||||
4. **Regular updates**: Keep Docker images and system packages updated
|
||||
5. **Backup strategy**: Set up automated backups and test restoration
|
||||
6. **Monitor logs**: Regularly check logs for suspicious activity
|
||||
7. **Rate limiting**: The app includes built-in rate limiting, configure as needed
|
||||
|
||||
## Support
|
||||
|
||||
For issues and questions:
|
||||
- Check logs first: `docker compose logs`
|
||||
- Review documentation in the repository
|
||||
- Check existing issues on GitHub
|
||||
- Create a new issue with:
|
||||
- Error messages
|
||||
- Log output
|
||||
- Environment details (without secrets)
|
||||
- Steps to reproduce
|
||||
@@ -1,98 +0,0 @@
|
||||
# Production Deployment Guide
|
||||
|
||||
This guide explains how to deploy PicPeak in production behind a reverse proxy like Traefik.
|
||||
|
||||
## Environment Configuration
|
||||
|
||||
### Frontend Configuration
|
||||
|
||||
For production deployment behind a reverse proxy (Traefik, Nginx, etc.), the frontend should use relative URLs to automatically inherit the protocol (HTTPS) and domain.
|
||||
|
||||
1. Copy the production environment template:
|
||||
```bash
|
||||
cp frontend/.env.production.example frontend/.env.production
|
||||
```
|
||||
|
||||
2. Set the API URL to use relative path:
|
||||
```env
|
||||
# frontend/.env.production
|
||||
VITE_API_URL=/api
|
||||
```
|
||||
|
||||
This ensures all API calls will use the same domain and protocol as the frontend.
|
||||
|
||||
### Backend Configuration
|
||||
|
||||
Ensure your backend `.env` file has the correct URLs:
|
||||
```env
|
||||
# backend/.env
|
||||
FRONTEND_URL=https://yourdomain.com
|
||||
ADMIN_URL=https://yourdomain.com
|
||||
```
|
||||
|
||||
## Docker Compose Production
|
||||
|
||||
When using Docker Compose in production:
|
||||
|
||||
1. Build with production environment:
|
||||
```bash
|
||||
docker-compose -f docker-compose.prod.yml build --build-arg NODE_ENV=production
|
||||
```
|
||||
|
||||
2. The frontend nginx configuration already includes proper proxy settings for:
|
||||
- `/api` → Backend API
|
||||
- `/photos` → Protected photo access
|
||||
- `/thumbnails` → Thumbnail images
|
||||
- `/uploads` → Public uploads (logos, favicons)
|
||||
|
||||
## Traefik Configuration
|
||||
|
||||
Example Traefik labels for docker-compose:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
frontend:
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.picpeak.rule=Host(`yourdomain.com`)"
|
||||
- "traefik.http.routers.picpeak.entrypoints=websecure"
|
||||
- "traefik.http.routers.picpeak.tls.certresolver=letsencrypt"
|
||||
- "traefik.http.services.picpeak.loadbalancer.server.port=80"
|
||||
```
|
||||
|
||||
## Important Notes
|
||||
|
||||
1. **No Hardcoded URLs**: The application uses environment variables with relative URL fallbacks, making it production-ready.
|
||||
|
||||
2. **HTTPS Only**: When `VITE_API_URL=/api`, all requests will use the same protocol as the page (HTTPS in production).
|
||||
|
||||
3. **CORS Configuration**: The backend CORS is configured to accept requests from the URLs specified in `FRONTEND_URL` and `ADMIN_URL`.
|
||||
|
||||
4. **Static Assets**: All static assets (photos, thumbnails, uploads) are served through the nginx proxy, inheriting authentication headers.
|
||||
|
||||
## Verification
|
||||
|
||||
After deployment, verify:
|
||||
|
||||
1. Check browser console for any localhost URLs (there should be none)
|
||||
2. Verify all API calls use HTTPS
|
||||
3. Check that images load correctly with authentication
|
||||
4. Test favicon and logo display
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
If you see console errors about localhost:
|
||||
|
||||
1. Ensure `VITE_API_URL=/api` in frontend environment
|
||||
2. Clear browser cache
|
||||
3. Rebuild frontend with production environment:
|
||||
```bash
|
||||
cd frontend
|
||||
npm run build
|
||||
```
|
||||
|
||||
If images don't load:
|
||||
|
||||
1. Check that nginx proxy locations are configured
|
||||
2. Verify authentication tokens are being sent
|
||||
3. Check backend logs for authentication errors
|
||||
@@ -1,312 +0,0 @@
|
||||
# Production Deployment Guide
|
||||
|
||||
This guide addresses all known production deployment issues and provides solutions.
|
||||
|
||||
## Pre-Deployment Checklist
|
||||
|
||||
### 1. Environment Variables
|
||||
Create a `.env` file with ALL required variables:
|
||||
|
||||
```bash
|
||||
# Required
|
||||
JWT_SECRET=<generate-with-openssl-rand-base64-32>
|
||||
DB_PASSWORD=<strong-password>
|
||||
ADMIN_URL=https://yourdomain.com
|
||||
FRONTEND_URL=https://yourdomain.com
|
||||
|
||||
# Database
|
||||
DB_USER=picpeak
|
||||
DB_NAME=picpeak
|
||||
|
||||
# Email (Optional but recommended)
|
||||
SMTP_HOST=smtp.gmail.com
|
||||
SMTP_PORT=587
|
||||
SMTP_SECURE=false
|
||||
SMTP_USER=your-email@gmail.com
|
||||
SMTP_PASS=your-app-password
|
||||
EMAIL_FROM=noreply@yourdomain.com
|
||||
|
||||
# Umami Analytics (Optional)
|
||||
UMAMI_URL=https://analytics.yourdomain.com
|
||||
UMAMI_WEBSITE_ID=your-website-id
|
||||
UMAMI_HASH_SALT=<generate-random-string>
|
||||
```
|
||||
|
||||
### 2. Generate Secrets
|
||||
|
||||
```bash
|
||||
# Generate JWT Secret
|
||||
openssl rand -base64 32
|
||||
|
||||
# Generate Database Password
|
||||
openssl rand -base64 24
|
||||
|
||||
# Generate Umami Hash Salt
|
||||
openssl rand -hex 32
|
||||
```
|
||||
|
||||
## Deployment Steps
|
||||
|
||||
### 1. Initial Setup
|
||||
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone https://github.com/the-luap/wedding-photo-sharing.git
|
||||
cd wedding-photo-sharing
|
||||
|
||||
# Create required directories
|
||||
mkdir -p storage/events/active storage/events/archived storage/thumbnails storage/uploads
|
||||
mkdir -p data logs
|
||||
mkdir -p certbot/conf certbot/www
|
||||
|
||||
# Set permissions (important!)
|
||||
chmod -R 755 storage data logs
|
||||
```
|
||||
|
||||
### 2. Fix Docker Volume Permissions
|
||||
|
||||
Create `docker-compose.override.yml` for local volume configuration:
|
||||
|
||||
```yaml
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
backend:
|
||||
volumes:
|
||||
- ./storage:/app/storage:delegated
|
||||
- ./data:/app/data:delegated
|
||||
- ./logs:/app/logs:delegated
|
||||
user: "1001:1001" # nodejs user
|
||||
|
||||
db:
|
||||
volumes:
|
||||
- ./postgres-data:/var/lib/postgresql/data
|
||||
```
|
||||
|
||||
### 3. Build and Deploy
|
||||
|
||||
```bash
|
||||
# Build images
|
||||
docker-compose -f docker-compose.prod.yml build
|
||||
|
||||
# Start services
|
||||
docker-compose -f docker-compose.prod.yml up -d
|
||||
|
||||
# Check logs
|
||||
docker-compose -f docker-compose.prod.yml logs -f backend
|
||||
```
|
||||
|
||||
### 4. Create Admin User
|
||||
|
||||
After deployment, create the first admin user:
|
||||
|
||||
```bash
|
||||
# Enter backend container
|
||||
docker-compose -f docker-compose.prod.yml exec backend sh
|
||||
|
||||
# Create admin
|
||||
node scripts/create-admin.js \
|
||||
--username admin \
|
||||
--email admin@yourdomain.com \
|
||||
--password <your-secure-password>
|
||||
|
||||
# Exit container
|
||||
exit
|
||||
```
|
||||
|
||||
### 5. Configure Email (if using database config)
|
||||
|
||||
1. Login to admin panel: https://yourdomain.com/admin
|
||||
2. Go to Settings > Email Configuration
|
||||
3. Enter SMTP details
|
||||
4. Test email sending
|
||||
|
||||
## Common Issues and Solutions
|
||||
|
||||
### Issue 1: Migration Failures
|
||||
|
||||
**Error**: "relation already exists"
|
||||
|
||||
**Solution**: The safe migration runner handles this automatically. If issues persist:
|
||||
|
||||
```bash
|
||||
# Reset migrations tracking
|
||||
docker-compose -f docker-compose.prod.yml exec db psql -U picpeak -d picpeak
|
||||
|
||||
# In PostgreSQL:
|
||||
DROP TABLE IF EXISTS migrations;
|
||||
\q
|
||||
|
||||
# Re-run migrations
|
||||
docker-compose -f docker-compose.prod.yml exec backend npm run migrate:safe
|
||||
```
|
||||
|
||||
### Issue 2: Permission Denied Errors
|
||||
|
||||
**Error**: "EACCES: permission denied"
|
||||
|
||||
**Solution**: Fix container permissions:
|
||||
|
||||
```bash
|
||||
# Stop containers
|
||||
docker-compose -f docker-compose.prod.yml down
|
||||
|
||||
# Fix permissions on host
|
||||
sudo chown -R 1001:1001 storage data logs
|
||||
|
||||
# Restart
|
||||
docker-compose -f docker-compose.prod.yml up -d
|
||||
```
|
||||
|
||||
### Issue 3: Database Connection Failed
|
||||
|
||||
**Error**: "no pg_hba.conf entry"
|
||||
|
||||
**Solution**: Already fixed in docker-compose.prod.yml with:
|
||||
- SSL disabled for internal Docker network
|
||||
- Proper authentication method (scram-sha-256)
|
||||
|
||||
### Issue 4: Frontend Can't Connect to Backend
|
||||
|
||||
**Error**: CORS errors or connection refused
|
||||
|
||||
**Solution**: Ensure environment variables match:
|
||||
- Backend: `FRONTEND_URL` must match your frontend URL
|
||||
- Frontend: `VITE_API_URL` must be set during build
|
||||
|
||||
### Issue 5: Email Not Sending
|
||||
|
||||
**Solution**: Check email configuration:
|
||||
|
||||
```bash
|
||||
# Check backend logs
|
||||
docker-compose -f docker-compose.prod.yml logs backend | grep email
|
||||
|
||||
# Verify SMTP settings
|
||||
# Gmail users: Use app password, not regular password
|
||||
# Enable "Less secure app access" or use OAuth2
|
||||
```
|
||||
|
||||
## SSL/HTTPS Setup
|
||||
|
||||
1. Update `nginx/sites-enabled/default` with your domain
|
||||
2. Run certbot:
|
||||
|
||||
```bash
|
||||
# Initial certificate
|
||||
docker-compose -f docker-compose.prod.yml run --rm certbot certonly \
|
||||
--webroot --webroot-path=/var/www/certbot \
|
||||
-d yourdomain.com -d www.yourdomain.com
|
||||
|
||||
# Auto-renewal is handled by the certbot container
|
||||
```
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Health Checks
|
||||
|
||||
```bash
|
||||
# Backend health
|
||||
curl http://localhost/api/health
|
||||
|
||||
# Database connection
|
||||
docker-compose -f docker-compose.prod.yml exec backend \
|
||||
psql -U picpeak -d picpeak -c "SELECT 1"
|
||||
```
|
||||
|
||||
### Logs
|
||||
|
||||
```bash
|
||||
# All services
|
||||
docker-compose -f docker-compose.prod.yml logs -f
|
||||
|
||||
# Specific service
|
||||
docker-compose -f docker-compose.prod.yml logs -f backend
|
||||
```
|
||||
|
||||
## Backup and Restore
|
||||
|
||||
### Backup
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# backup.sh
|
||||
DATE=$(date +%Y%m%d_%H%M%S)
|
||||
BACKUP_DIR="./backups/$DATE"
|
||||
|
||||
mkdir -p $BACKUP_DIR
|
||||
|
||||
# Database
|
||||
docker-compose -f docker-compose.prod.yml exec -T db \
|
||||
pg_dump -U picpeak picpeak > $BACKUP_DIR/database.sql
|
||||
|
||||
# Files
|
||||
tar -czf $BACKUP_DIR/storage.tar.gz storage/
|
||||
|
||||
echo "Backup completed: $BACKUP_DIR"
|
||||
```
|
||||
|
||||
### Restore
|
||||
|
||||
```bash
|
||||
# Database
|
||||
docker-compose -f docker-compose.prod.yml exec -T db \
|
||||
psql -U picpeak picpeak < ./backups/20240713_120000/database.sql
|
||||
|
||||
# Files
|
||||
tar -xzf ./backups/20240713_120000/storage.tar.gz
|
||||
```
|
||||
|
||||
## Production Best Practices
|
||||
|
||||
1. **Always use named volumes** in production for better data persistence
|
||||
2. **Set up monitoring** with Prometheus/Grafana
|
||||
3. **Enable backups** with automated scripts
|
||||
4. **Use a reverse proxy** (Nginx) for SSL termination
|
||||
5. **Implement rate limiting** at the Nginx level
|
||||
6. **Regular updates** - Keep Docker images updated
|
||||
7. **Log rotation** - Configure log rotation for application logs
|
||||
|
||||
## Troubleshooting Commands
|
||||
|
||||
```bash
|
||||
# Check running containers
|
||||
docker-compose -f docker-compose.prod.yml ps
|
||||
|
||||
# Restart a service
|
||||
docker-compose -f docker-compose.prod.yml restart backend
|
||||
|
||||
# View real-time logs
|
||||
docker-compose -f docker-compose.prod.yml logs -f --tail=100
|
||||
|
||||
# Execute commands in container
|
||||
docker-compose -f docker-compose.prod.yml exec backend sh
|
||||
|
||||
# Database shell
|
||||
docker-compose -f docker-compose.prod.yml exec db psql -U picpeak
|
||||
|
||||
# Clean restart
|
||||
docker-compose -f docker-compose.prod.yml down
|
||||
docker-compose -f docker-compose.prod.yml up -d
|
||||
```
|
||||
|
||||
## Security Checklist
|
||||
|
||||
- [ ] Strong JWT_SECRET (min 32 chars)
|
||||
- [ ] Strong database password
|
||||
- [ ] SSL/HTTPS enabled
|
||||
- [ ] Firewall configured (only 80/443 open)
|
||||
- [ ] Regular security updates
|
||||
- [ ] Backup encryption
|
||||
- [ ] Access logs monitored
|
||||
- [ ] Rate limiting enabled
|
||||
- [ ] File upload restrictions configured
|
||||
|
||||
## Support
|
||||
|
||||
For issues not covered here:
|
||||
1. Check application logs
|
||||
2. Review error messages carefully
|
||||
3. Ensure all environment variables are set
|
||||
4. Verify file permissions
|
||||
5. Check Docker daemon logs
|
||||
@@ -94,6 +94,20 @@ Perfect for:
|
||||
- **Email**: SMTP with customizable templates
|
||||
- **Analytics**: Privacy-focused with Umami integration
|
||||
|
||||
## 💻 System Requirements
|
||||
|
||||
### Minimum Requirements
|
||||
- **CPU**: 2 CPU cores
|
||||
- **RAM**: 2GB minimum
|
||||
- **Storage**: 20GB minimum (plus photo storage needs)
|
||||
- **OS**: Linux (Ubuntu 20.04+), macOS, or Windows with WSL2
|
||||
- **Node.js**: v18.0.0 or higher
|
||||
- **Database**: SQLite (included) or PostgreSQL 12+
|
||||
|
||||
### Docker Requirements (Recommended)
|
||||
- **Docker**: v20.10.0+
|
||||
- **Docker Compose**: v2.0.0+
|
||||
|
||||
## 🤝 Contributing
|
||||
|
||||
We love contributions! PicPeak is built by photographers, for photographers. Whether you're fixing bugs, adding features, or improving documentation, your help is welcome.
|
||||
@@ -124,7 +138,7 @@ PicPeak takes security seriously:
|
||||
- 📝 Activity logging
|
||||
- 🔒 Secure file access
|
||||
|
||||
Found a security issue? Please email security@example.com
|
||||
Found a security issue? Please open a [security issue](https://github.com/the-luap/picpeak/issues/new?labels=security) on GitHub
|
||||
|
||||
## 📸 Screenshots
|
||||
|
||||
@@ -159,10 +173,35 @@ Organize and manage your photo galleries with intuitive event management tools.
|
||||
|
||||
</details>
|
||||
|
||||
## 🗺️ Roadmap
|
||||
|
||||
We're constantly improving PicPeak and welcome contributions from our community! If you have ideas for new features or want to help implement existing ones, please open an issue or submit a pull request. Your contributions help make PicPeak better for everyone.
|
||||
|
||||
| Feature | Description | Priority | Status |
|
||||
|---------|-------------|----------|---------|
|
||||
| **Backup & Restore** | Comprehensive backup system with S3/MinIO support, automated scheduling, and safe restore functionality | High | ✅ Implemented |
|
||||
| **Gallery Templates** | Additional gallery layouts and themes (masonry, slideshow, story-style) for different event types | Medium | 🔄 Open |
|
||||
| **Face Recognition** | AI-powered face detection to help guests find their photos and create automatic person-based albums | Low | 🔄 Open |
|
||||
| **Gallery Feedback** | Allow guests to like, rate, and comment on photos with admin notifications and moderation | Medium | ✅ Implemented (not tested) |
|
||||
| **Video Support** | Upload and display videos alongside photos in galleries with streaming support | Low | 🔄 Open |
|
||||
| **Multiple Administrators** | Support for multiple admin accounts with role-based permissions and activity tracking | Low | 📋 Planned |
|
||||
|
||||
**Status Legend:** ✅ Implemented | 🚧 In Progress | 🔄 Open | 📋 Planned
|
||||
|
||||
## 🙏 Acknowledgments
|
||||
|
||||
PicPeak is inspired by the best features of commercial platforms while remaining completely open source. Special thanks to all contributors who make this project possible.
|
||||
|
||||
### 🤖 AI-Assisted Development
|
||||
|
||||
This project was generated with the assistance of AI technology, but has been:
|
||||
- ✅ **Fully tested end-to-end** by human developers
|
||||
- 🔒 **Security audited** with comprehensive security checks
|
||||
- 👨💻 **Human-reviewed** for code quality and best practices
|
||||
- 🧪 **Production-tested** in real-world scenarios
|
||||
|
||||
We believe in transparent development practices and the responsible use of AI as a tool to accelerate development while maintaining high standards of quality and security.
|
||||
|
||||
## 📄 License
|
||||
|
||||
PicPeak is released under the [MIT License](LICENSE). Use it freely for personal or commercial projects.
|
||||
|
||||
+10
-7
@@ -15,11 +15,14 @@ We take the security of PicPeak seriously. If you have discovered a security vul
|
||||
|
||||
### 1. **Do NOT create a public GitHub issue**
|
||||
|
||||
### 2. Email us at security@example.com with:
|
||||
- Description of the vulnerability
|
||||
- Steps to reproduce
|
||||
- Potential impact
|
||||
- Suggested fix (if any)
|
||||
### 2. Report the vulnerability by:
|
||||
- Opening a [security issue](https://github.com/the-luap/picpeak/issues/new?labels=security) on GitHub
|
||||
- Mark it clearly as "SECURITY" in the title
|
||||
- Include:
|
||||
- Description of the vulnerability
|
||||
- Steps to reproduce
|
||||
- Potential impact
|
||||
- Suggested fix (if any)
|
||||
|
||||
### 3. You can expect:
|
||||
- Acknowledgment within 48 hours
|
||||
@@ -79,7 +82,7 @@ We believe in responsible disclosure. Once a vulnerability is fixed:
|
||||
|
||||
## Contact
|
||||
|
||||
- Security issues: security@example.com
|
||||
- General support: https://github.com/the-luap/picpeak/issues
|
||||
- Security issues: [Create a security issue](https://github.com/the-luap/picpeak/issues/new?labels=security) on GitHub
|
||||
- General support: [GitHub Issues](https://github.com/the-luap/picpeak/issues)
|
||||
|
||||
Thank you for helping keep PicPeak and its users safe!
|
||||
+34
-16
@@ -3,39 +3,57 @@
|
||||
|
||||
# Application
|
||||
NODE_ENV=production
|
||||
PORT=3000
|
||||
PORT=3001
|
||||
|
||||
# Security
|
||||
JWT_SECRET=your-very-secure-jwt-secret-at-least-32-characters-long
|
||||
# Generate with: openssl rand -base64 32
|
||||
JWT_SECRET=your-very-secure-jwt-secret-at-least-32-characters-long-example123456
|
||||
|
||||
# URLs
|
||||
ADMIN_URL=https://yourdomain.com
|
||||
FRONTEND_URL=https://yourdomain.com
|
||||
# URLs (adjust for your domain)
|
||||
ADMIN_URL=https://photos.example.com
|
||||
FRONTEND_URL=https://photos.example.com
|
||||
BACKEND_URL=https://photos.example.com # Or https://api.photos.example.com if separate
|
||||
|
||||
# Database Configuration
|
||||
DATABASE_CLIENT=pg
|
||||
DB_HOST=db
|
||||
DB_HOST=localhost
|
||||
DB_PORT=5432
|
||||
DB_USER=picpeak
|
||||
DB_PASSWORD=your-secure-database-password
|
||||
DB_PASSWORD=your-secure-database-password-change-this
|
||||
DB_NAME=picpeak
|
||||
|
||||
# Email Configuration
|
||||
SMTP_HOST=smtp.example.com
|
||||
# Email Configuration (Examples for common providers)
|
||||
# Gmail example:
|
||||
# SMTP_HOST=smtp.gmail.com
|
||||
# SMTP_PORT=587
|
||||
# SMTP_SECURE=false
|
||||
# SMTP_USER=your-email@gmail.com
|
||||
# SMTP_PASS=your-app-specific-password
|
||||
|
||||
# SendGrid example:
|
||||
SMTP_HOST=smtp.sendgrid.net
|
||||
SMTP_PORT=587
|
||||
SMTP_SECURE=false
|
||||
SMTP_USER=your-smtp-username
|
||||
SMTP_PASS=your-smtp-password
|
||||
EMAIL_FROM=noreply@yourdomain.com
|
||||
SMTP_USER=apikey
|
||||
SMTP_PASS=your-sendgrid-api-key
|
||||
EMAIL_FROM=noreply@example.com
|
||||
|
||||
# Storage Paths (Docker)
|
||||
# Storage Paths
|
||||
# Docker deployment:
|
||||
STORAGE_PATH=/app/storage
|
||||
EVENTS_PATH=/app/storage/events
|
||||
ARCHIVE_PATH=/app/storage/events/archived
|
||||
|
||||
# Analytics (Optional)
|
||||
UMAMI_URL=https://analytics.yourdomain.com
|
||||
UMAMI_WEBSITE_ID=your-website-id
|
||||
# Local development:
|
||||
# STORAGE_PATH=./storage
|
||||
# EVENTS_PATH=./storage/events
|
||||
# ARCHIVE_PATH=./storage/events/archived
|
||||
|
||||
# Analytics Backend Configuration (OPTIONAL)
|
||||
# Used for server-side tracking only
|
||||
# Primary configuration should be done through Admin UI > Settings > Analytics
|
||||
# UMAMI_URL=https://analytics.example.com
|
||||
# UMAMI_WEBSITE_ID=b4d3c2a1-5678-90ab-cdef-1234567890ab
|
||||
|
||||
# Logging
|
||||
LOG_LEVEL=info
|
||||
@@ -1,5 +1,8 @@
|
||||
FROM node:18-alpine AS builder
|
||||
|
||||
# Add build argument for cache busting
|
||||
ARG CACHEBUST=1
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Copy package files
|
||||
|
||||
@@ -0,0 +1,229 @@
|
||||
# Enhanced Backup System Test Suite
|
||||
|
||||
This directory contains comprehensive tests for the enhanced backup system with S3 support.
|
||||
|
||||
## Test Structure
|
||||
|
||||
### Unit Tests
|
||||
- `services/backupService.enhanced.test.js` - Unit tests for the enhanced backup service
|
||||
- Configuration management
|
||||
- S3 backup functionality
|
||||
- Manifest generation
|
||||
- Error handling and recovery
|
||||
- Backward compatibility (local and rsync)
|
||||
- Service lifecycle management
|
||||
|
||||
### Integration Tests
|
||||
- `integration/backup-s3.test.js` - Integration tests for S3 backups
|
||||
- Real S3/MinIO connection tests
|
||||
- Full backup process with actual files
|
||||
- Incremental backup verification
|
||||
- Manifest storage and retrieval
|
||||
- Error recovery scenarios
|
||||
|
||||
### Manual Integration Test Script
|
||||
- `../scripts/test-backup-integration.js` - Comprehensive manual testing script
|
||||
- Can test against MinIO, AWS S3, or any S3-compatible service
|
||||
- Tests all backup types (S3, local, rsync)
|
||||
- Performance testing with large files
|
||||
- Detailed progress reporting
|
||||
|
||||
## Running Tests
|
||||
|
||||
### Prerequisites
|
||||
|
||||
1. **For Unit Tests**: No special setup required, all dependencies are mocked.
|
||||
|
||||
2. **For Integration Tests**: Requires a running S3-compatible service (MinIO recommended)
|
||||
```bash
|
||||
# Start MinIO using Docker
|
||||
docker run -d \
|
||||
-p 9000:9000 \
|
||||
-p 9001:9001 \
|
||||
--name minio-test \
|
||||
-e MINIO_ROOT_USER=minioadmin \
|
||||
-e MINIO_ROOT_PASSWORD=minioadmin \
|
||||
minio/minio server /data --console-address ":9001"
|
||||
```
|
||||
|
||||
3. **Environment Variables** (for integration tests):
|
||||
```bash
|
||||
# Optional - defaults work with local MinIO
|
||||
export TEST_S3_ENDPOINT=http://localhost:9000
|
||||
export TEST_S3_ACCESS_KEY=minioadmin
|
||||
export TEST_S3_SECRET_KEY=minioadmin
|
||||
|
||||
# Skip S3 tests if no S3 service available
|
||||
export SKIP_S3_TESTS=true
|
||||
```
|
||||
|
||||
### Running Unit Tests
|
||||
|
||||
```bash
|
||||
# Run all backup service tests
|
||||
npm test -- __tests__/services/backupService.enhanced.test.js
|
||||
|
||||
# Run specific test suite
|
||||
npm test -- __tests__/services/backupService.enhanced.test.js -t "S3 Backup Functionality"
|
||||
|
||||
# Run with coverage
|
||||
npm test -- --coverage __tests__/services/backupService.enhanced.test.js
|
||||
```
|
||||
|
||||
### Running Integration Tests
|
||||
|
||||
```bash
|
||||
# Ensure MinIO is running first!
|
||||
|
||||
# Run S3 integration tests
|
||||
npm test -- __tests__/integration/backup-s3.test.js
|
||||
|
||||
# Run with verbose output
|
||||
npm test -- __tests__/integration/backup-s3.test.js --verbose
|
||||
|
||||
# Skip S3 tests if needed
|
||||
SKIP_S3_TESTS=true npm test -- __tests__/integration/backup-s3.test.js
|
||||
```
|
||||
|
||||
### Running Manual Integration Tests
|
||||
|
||||
```bash
|
||||
# Test with local MinIO (default)
|
||||
node scripts/test-backup-integration.js
|
||||
|
||||
# Test with AWS S3
|
||||
node scripts/test-backup-integration.js \
|
||||
--endpoint https://s3.amazonaws.com \
|
||||
--access-key YOUR_ACCESS_KEY \
|
||||
--secret-key YOUR_SECRET_KEY \
|
||||
--bucket your-test-bucket
|
||||
|
||||
# Test local backup
|
||||
node scripts/test-backup-integration.js --type local
|
||||
|
||||
# Test with cleanup after completion
|
||||
node scripts/test-backup-integration.js --cleanup
|
||||
|
||||
# Verbose output
|
||||
node scripts/test-backup-integration.js --verbose
|
||||
```
|
||||
|
||||
## Test Coverage
|
||||
|
||||
The test suite covers:
|
||||
|
||||
### Configuration
|
||||
- ✅ Database configuration retrieval
|
||||
- ✅ JSON parsing and error handling
|
||||
- ✅ Configuration validation
|
||||
- ✅ Required field validation
|
||||
|
||||
### S3 Functionality
|
||||
- ✅ S3 client initialization
|
||||
- ✅ Connection testing
|
||||
- ✅ File upload with progress tracking
|
||||
- ✅ Large file handling (multipart upload)
|
||||
- ✅ Metadata and custom headers
|
||||
- ✅ Error handling and retries
|
||||
|
||||
### Backup Process
|
||||
- ✅ Full backup execution
|
||||
- ✅ Incremental backup (changed files only)
|
||||
- ✅ File checksum calculation and comparison
|
||||
- ✅ Database backup inclusion
|
||||
- ✅ Archive inclusion toggle
|
||||
- ✅ File size limits
|
||||
|
||||
### Manifest Generation
|
||||
- ✅ Full manifest generation
|
||||
- ✅ Incremental manifest with parent reference
|
||||
- ✅ JSON and YAML format support
|
||||
- ✅ Manifest validation
|
||||
- ✅ S3 manifest storage and retrieval
|
||||
- ✅ Checksum verification
|
||||
|
||||
### Error Handling
|
||||
- ✅ S3 connection failures
|
||||
- ✅ File read errors
|
||||
- ✅ Individual file failure recovery
|
||||
- ✅ Retry logic with exponential backoff
|
||||
- ✅ Email notifications on failure
|
||||
- ✅ Concurrent backup prevention
|
||||
|
||||
### Backward Compatibility
|
||||
- ✅ Local directory backup
|
||||
- ✅ Rsync backup
|
||||
- ✅ Existing manifest format support
|
||||
|
||||
### Service Management
|
||||
- ✅ Cron job scheduling
|
||||
- ✅ Service start/stop
|
||||
- ✅ Manual backup triggering
|
||||
- ✅ Backup history and status
|
||||
|
||||
## Mock Setup
|
||||
|
||||
The unit tests use comprehensive mocking:
|
||||
|
||||
```javascript
|
||||
// Database mocking
|
||||
jest.mock('../../src/database/db');
|
||||
|
||||
// S3 client mocking
|
||||
jest.mock('../../src/services/storage/s3Storage');
|
||||
|
||||
// File system mocking
|
||||
const mockFs = require('mock-fs');
|
||||
|
||||
// Cron job mocking
|
||||
jest.mock('node-cron');
|
||||
```
|
||||
|
||||
## CI/CD Integration
|
||||
|
||||
To run tests in CI/CD pipeline:
|
||||
|
||||
```yaml
|
||||
# Example GitHub Actions
|
||||
- name: Run Unit Tests
|
||||
run: npm test -- __tests__/services/backupService.enhanced.test.js
|
||||
|
||||
- name: Start MinIO
|
||||
run: |
|
||||
docker run -d \
|
||||
-p 9000:9000 \
|
||||
--name minio-test \
|
||||
-e MINIO_ROOT_USER=minioadmin \
|
||||
-e MINIO_ROOT_PASSWORD=minioadmin \
|
||||
minio/minio server /data
|
||||
|
||||
- name: Run Integration Tests
|
||||
run: npm test -- __tests__/integration/backup-s3.test.js
|
||||
```
|
||||
|
||||
## Debugging Tests
|
||||
|
||||
```bash
|
||||
# Run tests in debug mode
|
||||
node --inspect-brk ./node_modules/.bin/jest __tests__/services/backupService.enhanced.test.js
|
||||
|
||||
# Run single test with console output
|
||||
npm test -- __tests__/services/backupService.enhanced.test.js -t "should perform S3 backup" --verbose
|
||||
```
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
- Integration tests create real files and S3 objects
|
||||
- Each test run creates a unique S3 bucket to avoid conflicts
|
||||
- Cleanup is automatic but can be disabled for debugging
|
||||
- Large file tests (10MB+) are included but can be slow
|
||||
|
||||
## Adding New Tests
|
||||
|
||||
When adding new backup features:
|
||||
|
||||
1. Add unit tests to `backupService.enhanced.test.js`
|
||||
2. Add integration tests to `backup-s3.test.js` if S3-specific
|
||||
3. Update manual test script for comprehensive testing
|
||||
4. Ensure mocks are properly configured
|
||||
5. Document any new environment requirements
|
||||
@@ -0,0 +1,506 @@
|
||||
const { describe, it, expect, jest, beforeAll, afterAll, beforeEach, afterEach } = require('@jest/globals');
|
||||
const { S3Client, CreateBucketCommand, DeleteBucketCommand, ListObjectsV2Command, DeleteObjectsCommand } = require('@aws-sdk/client-s3');
|
||||
const path = require('path');
|
||||
const fs = require('fs').promises;
|
||||
const crypto = require('crypto');
|
||||
|
||||
// Load services
|
||||
const backupService = require('../../src/services/backupService');
|
||||
const S3StorageAdapter = require('../../src/services/storage/s3Storage');
|
||||
const { db, initialize: initDb } = require('../../src/database/db');
|
||||
const logger = require('../../src/utils/logger');
|
||||
|
||||
// Test configuration
|
||||
const TEST_CONFIG = {
|
||||
endpoint: process.env.TEST_S3_ENDPOINT || 'http://localhost:9000',
|
||||
accessKeyId: process.env.TEST_S3_ACCESS_KEY || 'minioadmin',
|
||||
secretAccessKey: process.env.TEST_S3_SECRET_KEY || 'minioadmin',
|
||||
bucket: 'test-backup-bucket-' + Date.now(),
|
||||
region: 'us-east-1'
|
||||
};
|
||||
|
||||
describe('S3 Backup Integration Tests', () => {
|
||||
let s3Client;
|
||||
let testStoragePath;
|
||||
let originalEnv;
|
||||
|
||||
beforeAll(async () => {
|
||||
// Skip if no S3 endpoint configured
|
||||
if (process.env.SKIP_S3_TESTS === 'true') {
|
||||
console.log('Skipping S3 integration tests (SKIP_S3_TESTS=true)');
|
||||
return;
|
||||
}
|
||||
|
||||
// Save original environment
|
||||
originalEnv = { ...process.env };
|
||||
|
||||
// Initialize S3 client for test setup
|
||||
s3Client = new S3Client({
|
||||
endpoint: TEST_CONFIG.endpoint,
|
||||
region: TEST_CONFIG.region,
|
||||
credentials: {
|
||||
accessKeyId: TEST_CONFIG.accessKeyId,
|
||||
secretAccessKey: TEST_CONFIG.secretAccessKey
|
||||
},
|
||||
forcePathStyle: true
|
||||
});
|
||||
|
||||
// Create test bucket
|
||||
try {
|
||||
await s3Client.send(new CreateBucketCommand({ Bucket: TEST_CONFIG.bucket }));
|
||||
console.log(`Created test bucket: ${TEST_CONFIG.bucket}`);
|
||||
} catch (error) {
|
||||
if (error.name !== 'BucketAlreadyOwnedByYou') {
|
||||
console.error('Failed to create test bucket:', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
// Initialize database
|
||||
await initDb();
|
||||
await db.migrate.latest();
|
||||
|
||||
// Create test storage directory
|
||||
testStoragePath = path.join(__dirname, '../fixtures/test-storage');
|
||||
await fs.mkdir(testStoragePath, { recursive: true });
|
||||
process.env.STORAGE_PATH = testStoragePath;
|
||||
|
||||
// Set up test data
|
||||
await setupTestData();
|
||||
|
||||
// Mock logger to reduce noise
|
||||
logger.info = jest.fn();
|
||||
logger.debug = jest.fn();
|
||||
logger.warn = jest.fn();
|
||||
logger.error = jest.fn();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
if (process.env.SKIP_S3_TESTS === 'true') return;
|
||||
|
||||
try {
|
||||
// Clean up S3 bucket
|
||||
await cleanupS3Bucket();
|
||||
await s3Client.send(new DeleteBucketCommand({ Bucket: TEST_CONFIG.bucket }));
|
||||
console.log(`Deleted test bucket: ${TEST_CONFIG.bucket}`);
|
||||
} catch (error) {
|
||||
console.error('Failed to cleanup S3 bucket:', error);
|
||||
}
|
||||
|
||||
// Clean up test storage
|
||||
await fs.rm(testStoragePath, { recursive: true, force: true });
|
||||
|
||||
// Restore environment
|
||||
process.env = originalEnv;
|
||||
|
||||
// Close database
|
||||
await db.destroy();
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
if (process.env.SKIP_S3_TESTS === 'true') {
|
||||
return;
|
||||
}
|
||||
|
||||
// Clean backup tables
|
||||
await db('backup_runs').del();
|
||||
await db('backup_file_states').del();
|
||||
await db('database_backup_runs').del();
|
||||
|
||||
// Configure S3 backup settings
|
||||
await configureS3Backup();
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
if (process.env.SKIP_S3_TESTS === 'true') return;
|
||||
|
||||
// Clean up S3 objects created during test
|
||||
await cleanupS3Bucket();
|
||||
});
|
||||
|
||||
describe('S3 Connection and Configuration', () => {
|
||||
it('should successfully connect to S3-compatible storage', async () => {
|
||||
if (process.env.SKIP_S3_TESTS === 'true') return;
|
||||
|
||||
const s3Adapter = new S3StorageAdapter({
|
||||
...TEST_CONFIG,
|
||||
bucket: TEST_CONFIG.bucket,
|
||||
forcePathStyle: true,
|
||||
sslEnabled: false
|
||||
});
|
||||
|
||||
const connected = await s3Adapter.testConnection();
|
||||
expect(connected).toBe(true);
|
||||
});
|
||||
|
||||
it('should validate S3 configuration before backup', async () => {
|
||||
if (process.env.SKIP_S3_TESTS === 'true') return;
|
||||
|
||||
// Remove required configuration
|
||||
await db('app_settings')
|
||||
.where('setting_key', 'backup_s3_secret_key')
|
||||
.del();
|
||||
|
||||
await backupService.runBackup();
|
||||
|
||||
const lastRun = await db('backup_runs')
|
||||
.orderBy('started_at', 'desc')
|
||||
.first();
|
||||
|
||||
expect(lastRun.status).toBe('failed');
|
||||
expect(lastRun.error_message).toContain('S3 backup configuration incomplete');
|
||||
});
|
||||
});
|
||||
|
||||
describe('Full S3 Backup Process', () => {
|
||||
it('should perform complete S3 backup with all file types', async () => {
|
||||
if (process.env.SKIP_S3_TESTS === 'true') return;
|
||||
|
||||
// Run backup
|
||||
await backupService.runBackup();
|
||||
|
||||
// Verify backup run completed
|
||||
const backupRun = await db('backup_runs')
|
||||
.orderBy('started_at', 'desc')
|
||||
.first();
|
||||
|
||||
expect(backupRun.status).toBe('completed');
|
||||
expect(backupRun.files_backed_up).toBeGreaterThan(0);
|
||||
expect(backupRun.total_size_bytes).toBeGreaterThan(0);
|
||||
|
||||
// Verify files in S3
|
||||
const s3Objects = await listS3Objects();
|
||||
expect(s3Objects.length).toBeGreaterThan(0);
|
||||
|
||||
// Check for expected file types
|
||||
const hasPhotos = s3Objects.some(obj => obj.Key.includes('events/active'));
|
||||
const hasThumbnails = s3Objects.some(obj => obj.Key.includes('thumbnails'));
|
||||
const hasManifest = s3Objects.some(obj => obj.Key.includes('backup-manifest'));
|
||||
const hasSummary = s3Objects.some(obj => obj.Key.includes('backup-summary.json'));
|
||||
|
||||
expect(hasPhotos).toBe(true);
|
||||
expect(hasThumbnails).toBe(true);
|
||||
expect(hasManifest).toBe(true);
|
||||
expect(hasSummary).toBe(true);
|
||||
});
|
||||
|
||||
it('should handle large file uploads with multipart', async () => {
|
||||
if (process.env.SKIP_S3_TESTS === 'true') return;
|
||||
|
||||
// Create a large test file (15MB)
|
||||
const largeFilePath = path.join(testStoragePath, 'events/active/large-photo.jpg');
|
||||
const largeFileSize = 15 * 1024 * 1024; // 15MB
|
||||
const largeFileContent = Buffer.alloc(largeFileSize, 'x');
|
||||
await fs.writeFile(largeFilePath, largeFileContent);
|
||||
|
||||
// Run backup
|
||||
await backupService.runBackup();
|
||||
|
||||
// Verify large file was uploaded
|
||||
const s3Objects = await listS3Objects();
|
||||
const largeFileUploaded = s3Objects.some(obj =>
|
||||
obj.Key.includes('large-photo.jpg') && obj.Size === largeFileSize
|
||||
);
|
||||
|
||||
expect(largeFileUploaded).toBe(true);
|
||||
});
|
||||
|
||||
it('should include database backup when available', async () => {
|
||||
if (process.env.SKIP_S3_TESTS === 'true') return;
|
||||
|
||||
// Create a mock database backup
|
||||
const dbBackupPath = path.join(testStoragePath, 'backups/db-backup.sql');
|
||||
await fs.mkdir(path.dirname(dbBackupPath), { recursive: true });
|
||||
await fs.writeFile(dbBackupPath, 'CREATE TABLE test (id INT);');
|
||||
|
||||
// Record database backup
|
||||
await db('database_backup_runs').insert({
|
||||
started_at: new Date(),
|
||||
completed_at: new Date(),
|
||||
status: 'completed',
|
||||
backup_type: 'sqlite',
|
||||
file_path: dbBackupPath,
|
||||
file_size_bytes: 100,
|
||||
checksum: 'test123',
|
||||
statistics: JSON.stringify({ tables: {} }),
|
||||
table_checksums: JSON.stringify({})
|
||||
});
|
||||
|
||||
// Configure to include database
|
||||
await db('app_settings')
|
||||
.where('setting_key', 'backup_include_database')
|
||||
.update({ setting_value: 'true' });
|
||||
|
||||
// Run backup
|
||||
await backupService.runBackup();
|
||||
|
||||
// Verify database backup in S3
|
||||
const s3Objects = await listS3Objects();
|
||||
const hasDbBackup = s3Objects.some(obj => obj.Key.includes('database/db-backup.sql'));
|
||||
expect(hasDbBackup).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Incremental Backup', () => {
|
||||
it('should only upload changed files in incremental backup', async () => {
|
||||
if (process.env.SKIP_S3_TESTS === 'true') return;
|
||||
|
||||
// First backup - full
|
||||
await backupService.runBackup();
|
||||
|
||||
const firstRun = await db('backup_runs')
|
||||
.orderBy('started_at', 'desc')
|
||||
.first();
|
||||
|
||||
const firstObjectCount = (await listS3Objects()).length;
|
||||
|
||||
// Wait a moment to ensure different timestamps
|
||||
await new Promise(resolve => setTimeout(resolve, 100));
|
||||
|
||||
// Modify one file
|
||||
const modifiedFile = path.join(testStoragePath, 'events/active/event1/photo1.jpg');
|
||||
await fs.writeFile(modifiedFile, 'modified content');
|
||||
|
||||
// Second backup - incremental
|
||||
await backupService.runBackup();
|
||||
|
||||
const secondRun = await db('backup_runs')
|
||||
.orderBy('started_at', 'desc')
|
||||
.first();
|
||||
|
||||
expect(secondRun.id).not.toBe(firstRun.id);
|
||||
expect(secondRun.files_backed_up).toBe(1); // Only modified file
|
||||
|
||||
// Check manifest indicates incremental
|
||||
if (secondRun.manifest_path) {
|
||||
const manifest = await backupService.getBackupManifest(secondRun.id);
|
||||
expect(manifest.manifest.incremental).toBeDefined();
|
||||
expect(manifest.manifest.incremental.modified_files_count).toBe(1);
|
||||
}
|
||||
});
|
||||
|
||||
it('should track file states across backups', async () => {
|
||||
if (process.env.SKIP_S3_TESTS === 'true') return;
|
||||
|
||||
await backupService.runBackup();
|
||||
|
||||
// Check file states are recorded
|
||||
const fileStates = await db('backup_file_states').select('*');
|
||||
expect(fileStates.length).toBeGreaterThan(0);
|
||||
|
||||
// Verify checksums are stored
|
||||
const hasChecksums = fileStates.every(state => state.checksum !== null);
|
||||
expect(hasChecksums).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('S3 Manifest Storage', () => {
|
||||
it('should upload manifest to S3 and retrieve it', async () => {
|
||||
if (process.env.SKIP_S3_TESTS === 'true') return;
|
||||
|
||||
// Configure YAML manifest format
|
||||
await db('app_settings')
|
||||
.where('setting_key', 'backup_manifest_format')
|
||||
.update({ setting_value: '"yaml"' });
|
||||
|
||||
await backupService.runBackup();
|
||||
|
||||
const backupRun = await db('backup_runs')
|
||||
.orderBy('started_at', 'desc')
|
||||
.first();
|
||||
|
||||
expect(backupRun.manifest_path).toMatch(/^s3:\/\//);
|
||||
|
||||
// Retrieve manifest
|
||||
const { manifest, summary } = await backupService.getBackupManifest(backupRun.id);
|
||||
|
||||
expect(manifest).toBeDefined();
|
||||
expect(manifest.backup.id).toBeDefined();
|
||||
expect(summary).toContain('BACKUP MANIFEST SUMMARY');
|
||||
});
|
||||
|
||||
it('should validate manifest integrity', async () => {
|
||||
if (process.env.SKIP_S3_TESTS === 'true') return;
|
||||
|
||||
await backupService.runBackup();
|
||||
|
||||
const backupRun = await db('backup_runs')
|
||||
.orderBy('started_at', 'desc')
|
||||
.first();
|
||||
|
||||
const validationResult = await backupService.validateBackupManifest(backupRun.manifest_path);
|
||||
|
||||
expect(validationResult.valid).toBe(true);
|
||||
expect(validationResult.manifest).toBeDefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('Error Recovery', () => {
|
||||
it('should handle S3 connection failures gracefully', async () => {
|
||||
if (process.env.SKIP_S3_TESTS === 'true') return;
|
||||
|
||||
// Configure with invalid endpoint
|
||||
await db('app_settings')
|
||||
.where('setting_key', 'backup_s3_endpoint')
|
||||
.update({ setting_value: '"http://invalid-endpoint:9999"' });
|
||||
|
||||
await backupService.runBackup();
|
||||
|
||||
const backupRun = await db('backup_runs')
|
||||
.orderBy('started_at', 'desc')
|
||||
.first();
|
||||
|
||||
expect(backupRun.status).toBe('failed');
|
||||
expect(backupRun.error_message).toBeDefined();
|
||||
});
|
||||
|
||||
it('should continue backup despite individual file failures', async () => {
|
||||
if (process.env.SKIP_S3_TESTS === 'true') return;
|
||||
|
||||
// Create a file that will be deleted during backup
|
||||
const tempFile = path.join(testStoragePath, 'events/active/temp.jpg');
|
||||
await fs.writeFile(tempFile, 'temporary');
|
||||
|
||||
// Mock file deletion during backup
|
||||
const originalUpload = S3StorageAdapter.prototype.upload;
|
||||
let callCount = 0;
|
||||
S3StorageAdapter.prototype.upload = jest.fn(async function(localPath, s3Key, options) {
|
||||
callCount++;
|
||||
if (callCount === 2) {
|
||||
// Delete the temp file to cause an error
|
||||
await fs.unlink(tempFile).catch(() => {});
|
||||
}
|
||||
return originalUpload.call(this, localPath, s3Key, options);
|
||||
});
|
||||
|
||||
await backupService.runBackup();
|
||||
|
||||
const backupRun = await db('backup_runs')
|
||||
.orderBy('started_at', 'desc')
|
||||
.first();
|
||||
|
||||
// Should complete despite one file error
|
||||
expect(backupRun.status).toBe('completed');
|
||||
expect(backupRun.files_backed_up).toBeGreaterThan(0);
|
||||
|
||||
// Restore original method
|
||||
S3StorageAdapter.prototype.upload = originalUpload;
|
||||
});
|
||||
|
||||
it('should retry failed uploads with exponential backoff', async () => {
|
||||
if (process.env.SKIP_S3_TESTS === 'true') return;
|
||||
|
||||
// Mock S3 upload to fail twice then succeed
|
||||
const originalUpload = S3StorageAdapter.prototype.upload;
|
||||
let attemptCount = 0;
|
||||
S3StorageAdapter.prototype.upload = jest.fn(async function(localPath, s3Key, options) {
|
||||
attemptCount++;
|
||||
if (attemptCount <= 2) {
|
||||
const error = new Error('Network timeout');
|
||||
error.code = 'ETIMEDOUT';
|
||||
throw error;
|
||||
}
|
||||
return originalUpload.call(this, localPath, s3Key, options);
|
||||
});
|
||||
|
||||
await backupService.runBackup();
|
||||
|
||||
const backupRun = await db('backup_runs')
|
||||
.orderBy('started_at', 'desc')
|
||||
.first();
|
||||
|
||||
// Should succeed after retries
|
||||
expect(backupRun.status).toBe('completed');
|
||||
expect(attemptCount).toBeGreaterThan(2);
|
||||
|
||||
// Restore original method
|
||||
S3StorageAdapter.prototype.upload = originalUpload;
|
||||
});
|
||||
});
|
||||
|
||||
// Helper functions
|
||||
|
||||
async function setupTestData() {
|
||||
// Create test directory structure
|
||||
const dirs = [
|
||||
'events/active/event1',
|
||||
'events/active/event2',
|
||||
'events/archived',
|
||||
'thumbnails',
|
||||
'uploads'
|
||||
];
|
||||
|
||||
for (const dir of dirs) {
|
||||
await fs.mkdir(path.join(testStoragePath, dir), { recursive: true });
|
||||
}
|
||||
|
||||
// Create test files
|
||||
const files = [
|
||||
{ path: 'events/active/event1/photo1.jpg', content: 'photo1 content' },
|
||||
{ path: 'events/active/event1/photo2.jpg', content: 'photo2 content' },
|
||||
{ path: 'events/active/event2/photo3.jpg', content: 'photo3 content' },
|
||||
{ path: 'events/archived/old-event.zip', content: 'archived content' },
|
||||
{ path: 'thumbnails/thumb1.jpg', content: 'thumbnail content' },
|
||||
{ path: 'uploads/logo.png', content: 'logo content' }
|
||||
];
|
||||
|
||||
for (const file of files) {
|
||||
await fs.writeFile(
|
||||
path.join(testStoragePath, file.path),
|
||||
file.content
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
async function configureS3Backup() {
|
||||
const settings = [
|
||||
{ setting_key: 'backup_enabled', setting_value: 'true' },
|
||||
{ setting_key: 'backup_destination_type', setting_value: '"s3"' },
|
||||
{ setting_key: 'backup_s3_bucket', setting_value: `"${TEST_CONFIG.bucket}"` },
|
||||
{ setting_key: 'backup_s3_region', setting_value: `"${TEST_CONFIG.region}"` },
|
||||
{ setting_key: 'backup_s3_endpoint', setting_value: `"${TEST_CONFIG.endpoint}"` },
|
||||
{ setting_key: 'backup_s3_access_key', setting_value: `"${TEST_CONFIG.accessKeyId}"` },
|
||||
{ setting_key: 'backup_s3_secret_key', setting_value: `"${TEST_CONFIG.secretAccessKey}"` },
|
||||
{ setting_key: 'backup_s3_force_path_style', setting_value: 'true' },
|
||||
{ setting_key: 'backup_s3_ssl_enabled', setting_value: 'false' },
|
||||
{ setting_key: 'backup_include_archived', setting_value: 'true' },
|
||||
{ setting_key: 'backup_incremental', setting_value: 'true' },
|
||||
{ setting_key: 'backup_max_file_size_mb', setting_value: '100' }
|
||||
];
|
||||
|
||||
for (const setting of settings) {
|
||||
await db('app_settings')
|
||||
.insert({
|
||||
setting_type: 'backup',
|
||||
...setting,
|
||||
created_at: new Date(),
|
||||
updated_at: new Date()
|
||||
})
|
||||
.onConflict(['setting_type', 'setting_key'])
|
||||
.merge();
|
||||
}
|
||||
}
|
||||
|
||||
async function listS3Objects() {
|
||||
const response = await s3Client.send(new ListObjectsV2Command({
|
||||
Bucket: TEST_CONFIG.bucket
|
||||
}));
|
||||
return response.Contents || [];
|
||||
}
|
||||
|
||||
async function cleanupS3Bucket() {
|
||||
try {
|
||||
const objects = await listS3Objects();
|
||||
if (objects.length > 0) {
|
||||
await s3Client.send(new DeleteObjectsCommand({
|
||||
Bucket: TEST_CONFIG.bucket,
|
||||
Delete: {
|
||||
Objects: objects.map(obj => ({ Key: obj.Key }))
|
||||
}
|
||||
}));
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Failed to cleanup S3 objects:', error);
|
||||
}
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,751 @@
|
||||
const { describe, it, expect, jest, beforeEach, afterEach } = require('@jest/globals');
|
||||
const mockFs = require('mock-fs');
|
||||
const path = require('path');
|
||||
const crypto = require('crypto');
|
||||
const { EventEmitter } = require('events');
|
||||
|
||||
// Mock dependencies before requiring the module
|
||||
jest.mock('../../src/database/db');
|
||||
jest.mock('../../src/utils/logger');
|
||||
jest.mock('../../src/services/emailProcessor');
|
||||
jest.mock('node-cron');
|
||||
jest.mock('../../src/services/backupManifest');
|
||||
jest.mock('../../src/services/storage/s3Storage');
|
||||
|
||||
const backupService = require('../../src/services/backupService');
|
||||
const { db } = require('../../src/database/db');
|
||||
const logger = require('../../src/utils/logger');
|
||||
const { queueEmail } = require('../../src/services/emailProcessor');
|
||||
const cron = require('node-cron');
|
||||
const backupManifest = require('../../src/services/backupManifest');
|
||||
const S3StorageAdapter = require('../../src/services/storage/s3Storage');
|
||||
|
||||
describe('Enhanced Backup Service Tests', () => {
|
||||
let mockDb;
|
||||
let mockS3Client;
|
||||
let mockCronJob;
|
||||
|
||||
beforeEach(() => {
|
||||
// Reset all mocks
|
||||
jest.clearAllMocks();
|
||||
|
||||
// Mock database
|
||||
mockDb = {
|
||||
select: jest.fn().mockReturnThis(),
|
||||
where: jest.fn().mockReturnThis(),
|
||||
orderBy: jest.fn().mockReturnThis(),
|
||||
limit: jest.fn().mockReturnThis(),
|
||||
first: jest.fn(),
|
||||
insert: jest.fn(),
|
||||
update: jest.fn(),
|
||||
delete: jest.fn()
|
||||
};
|
||||
db.mockReturnValue(mockDb);
|
||||
|
||||
// Mock cron job
|
||||
mockCronJob = {
|
||||
stop: jest.fn()
|
||||
};
|
||||
cron.schedule.mockReturnValue(mockCronJob);
|
||||
|
||||
// Mock S3 client
|
||||
mockS3Client = {
|
||||
testConnection: jest.fn().mockResolvedValue(true),
|
||||
upload: jest.fn().mockResolvedValue({ Location: 's3://bucket/key' }),
|
||||
uploadStream: jest.fn().mockResolvedValue({ Location: 's3://bucket/key' }),
|
||||
download: jest.fn().mockResolvedValue(),
|
||||
exists: jest.fn().mockResolvedValue(false),
|
||||
delete: jest.fn().mockResolvedValue(),
|
||||
list: jest.fn().mockResolvedValue({ Contents: [] })
|
||||
};
|
||||
S3StorageAdapter.mockImplementation(() => mockS3Client);
|
||||
|
||||
// Mock backup manifest
|
||||
backupManifest.generateManifest = jest.fn().mockResolvedValue({
|
||||
backup: { id: 'test-backup-123' },
|
||||
version: '2.0'
|
||||
});
|
||||
backupManifest.saveManifest = jest.fn().mockResolvedValue('/path/to/manifest.json');
|
||||
backupManifest.loadManifest = jest.fn().mockResolvedValue({});
|
||||
backupManifest.validateManifest = jest.fn();
|
||||
backupManifest.generateSummaryReport = jest.fn().mockReturnValue('Summary report');
|
||||
|
||||
// Mock logger
|
||||
logger.info = jest.fn();
|
||||
logger.error = jest.fn();
|
||||
logger.warn = jest.fn();
|
||||
logger.debug = jest.fn();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
mockFs.restore();
|
||||
});
|
||||
|
||||
describe('getBackupConfig', () => {
|
||||
it('should retrieve and parse backup configuration from database', async () => {
|
||||
const mockSettings = [
|
||||
{ setting_key: 'backup_enabled', setting_value: 'true' },
|
||||
{ setting_key: 'backup_destination_type', setting_value: '"s3"' },
|
||||
{ setting_key: 'backup_s3_bucket', setting_value: '"test-bucket"' },
|
||||
{ setting_key: 'backup_retention_days', setting_value: '30' }
|
||||
];
|
||||
|
||||
mockDb.select.mockResolvedValue(mockSettings);
|
||||
|
||||
const config = await backupService.getBackupConfig();
|
||||
|
||||
expect(config).toEqual({
|
||||
backup_enabled: true,
|
||||
backup_destination_type: 's3',
|
||||
backup_s3_bucket: 'test-bucket',
|
||||
backup_retention_days: 30
|
||||
});
|
||||
|
||||
expect(db).toHaveBeenCalledWith('app_settings');
|
||||
expect(mockDb.where).toHaveBeenCalledWith('setting_type', 'backup');
|
||||
});
|
||||
|
||||
it('should handle JSON parse errors gracefully', async () => {
|
||||
const mockSettings = [
|
||||
{ setting_key: 'backup_enabled', setting_value: 'invalid-json' }
|
||||
];
|
||||
|
||||
mockDb.select.mockResolvedValue(mockSettings);
|
||||
|
||||
const config = await backupService.getBackupConfig();
|
||||
|
||||
expect(config).toEqual({
|
||||
backup_enabled: 'invalid-json'
|
||||
});
|
||||
});
|
||||
|
||||
it('should return null on database error', async () => {
|
||||
mockDb.select.mockRejectedValue(new Error('Database error'));
|
||||
|
||||
const config = await backupService.getBackupConfig();
|
||||
|
||||
expect(config).toBeNull();
|
||||
expect(logger.error).toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe('S3 Backup Functionality', () => {
|
||||
beforeEach(() => {
|
||||
// Mock file system
|
||||
mockFs({
|
||||
'/storage/events/active/event1': {
|
||||
'photo1.jpg': Buffer.from('photo1 content'),
|
||||
'photo2.jpg': Buffer.from('photo2 content')
|
||||
},
|
||||
'/storage/events/archived/event2.zip': Buffer.from('archived content'),
|
||||
'/storage/thumbnails': {
|
||||
'thumb1.jpg': Buffer.from('thumb1 content')
|
||||
},
|
||||
'/storage/uploads': {
|
||||
'logo.png': Buffer.from('logo content')
|
||||
}
|
||||
});
|
||||
|
||||
process.env.STORAGE_PATH = '/storage';
|
||||
});
|
||||
|
||||
it('should perform S3 backup with correct configuration', async () => {
|
||||
const config = {
|
||||
backup_enabled: true,
|
||||
backup_destination_type: 's3',
|
||||
backup_s3_bucket: 'test-bucket',
|
||||
backup_s3_region: 'us-east-1',
|
||||
backup_s3_endpoint: 'https://s3.amazonaws.com',
|
||||
backup_s3_access_key: 'test-key',
|
||||
backup_s3_secret_key: 'test-secret',
|
||||
backup_include_archived: true,
|
||||
backup_max_file_size_mb: 100
|
||||
};
|
||||
|
||||
mockDb.select.mockResolvedValue([]);
|
||||
mockDb.where.mockReturnThis();
|
||||
mockDb.first.mockResolvedValue(null);
|
||||
mockDb.insert.mockResolvedValue([1]);
|
||||
|
||||
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
|
||||
jest.spyOn(backupService, 'getDatabaseBackupInfo').mockResolvedValue({
|
||||
type: 'sqlite',
|
||||
backupFile: null,
|
||||
hasChanged: true
|
||||
});
|
||||
|
||||
await backupService.runBackup();
|
||||
|
||||
expect(S3StorageAdapter).toHaveBeenCalledWith({
|
||||
bucket: 'test-bucket',
|
||||
region: 'us-east-1',
|
||||
endpoint: 'https://s3.amazonaws.com',
|
||||
accessKeyId: 'test-key',
|
||||
secretAccessKey: 'test-secret',
|
||||
forcePathStyle: false,
|
||||
sslEnabled: true,
|
||||
maxRetries: 3,
|
||||
retryDelay: 1000
|
||||
});
|
||||
|
||||
expect(mockS3Client.testConnection).toHaveBeenCalled();
|
||||
expect(mockS3Client.upload).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('should handle S3 upload failures gracefully', async () => {
|
||||
const config = {
|
||||
backup_enabled: true,
|
||||
backup_destination_type: 's3',
|
||||
backup_s3_bucket: 'test-bucket',
|
||||
backup_s3_access_key: 'test-key',
|
||||
backup_s3_secret_key: 'test-secret'
|
||||
};
|
||||
|
||||
mockDb.select.mockResolvedValue([]);
|
||||
mockDb.insert.mockResolvedValue([1]);
|
||||
mockDb.first.mockResolvedValue(null);
|
||||
|
||||
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
|
||||
|
||||
mockS3Client.testConnection.mockRejectedValue(new Error('Connection failed'));
|
||||
|
||||
await backupService.runBackup();
|
||||
|
||||
expect(logger.error).toHaveBeenCalledWith('S3 backup failed:', expect.any(Error));
|
||||
expect(mockDb.update).toHaveBeenCalledWith(expect.objectContaining({
|
||||
status: 'failed',
|
||||
error_message: expect.stringContaining('Connection failed')
|
||||
}));
|
||||
});
|
||||
|
||||
it('should skip unchanged files in incremental backup', async () => {
|
||||
const config = {
|
||||
backup_enabled: true,
|
||||
backup_destination_type: 's3',
|
||||
backup_s3_bucket: 'test-bucket',
|
||||
backup_s3_access_key: 'test-key',
|
||||
backup_s3_secret_key: 'test-secret',
|
||||
backup_incremental: true
|
||||
};
|
||||
|
||||
// Mock existing file state
|
||||
mockDb.first.mockImplementation((query) => {
|
||||
if (query === undefined) {
|
||||
return Promise.resolve({
|
||||
file_path: 'events/active/event1/photo1.jpg',
|
||||
checksum: crypto.createHash('sha256').update('photo1 content').digest('hex')
|
||||
});
|
||||
}
|
||||
return Promise.resolve(null);
|
||||
});
|
||||
|
||||
mockDb.select.mockResolvedValue([]);
|
||||
mockDb.insert.mockResolvedValue([1]);
|
||||
|
||||
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
|
||||
|
||||
await backupService.runBackup();
|
||||
|
||||
// Should skip unchanged file
|
||||
const uploadCalls = mockS3Client.upload.mock.calls;
|
||||
const photo1Uploaded = uploadCalls.some(call =>
|
||||
call[1].includes('photo1.jpg')
|
||||
);
|
||||
expect(photo1Uploaded).toBe(false);
|
||||
});
|
||||
|
||||
it('should include database backup when configured', async () => {
|
||||
const config = {
|
||||
backup_enabled: true,
|
||||
backup_destination_type: 's3',
|
||||
backup_s3_bucket: 'test-bucket',
|
||||
backup_s3_access_key: 'test-key',
|
||||
backup_s3_secret_key: 'test-secret',
|
||||
backup_include_database: true
|
||||
};
|
||||
|
||||
mockDb.select.mockResolvedValue([]);
|
||||
mockDb.insert.mockResolvedValue([1]);
|
||||
|
||||
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
|
||||
jest.spyOn(backupService, 'getDatabaseBackupInfo').mockResolvedValue({
|
||||
type: 'sqlite',
|
||||
backupFile: '/backup/db-backup.sql',
|
||||
size: 1024000,
|
||||
checksum: 'abc123',
|
||||
hasChanged: false
|
||||
});
|
||||
|
||||
// Mock database backup file
|
||||
mockFs({
|
||||
'/storage/events/active': {},
|
||||
'/backup/db-backup.sql': Buffer.from('database backup content')
|
||||
});
|
||||
|
||||
await backupService.runBackup();
|
||||
|
||||
// Verify database backup was uploaded
|
||||
const uploadCalls = mockS3Client.upload.mock.calls;
|
||||
const dbBackupUploaded = uploadCalls.some(call =>
|
||||
call[1].includes('database/db-backup.sql')
|
||||
);
|
||||
expect(dbBackupUploaded).toBe(true);
|
||||
});
|
||||
|
||||
it('should validate required S3 configuration', async () => {
|
||||
const config = {
|
||||
backup_enabled: true,
|
||||
backup_destination_type: 's3',
|
||||
backup_s3_bucket: 'test-bucket'
|
||||
// Missing access key and secret key
|
||||
};
|
||||
|
||||
mockDb.select.mockResolvedValue([]);
|
||||
mockDb.insert.mockResolvedValue([1]);
|
||||
|
||||
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
|
||||
|
||||
await backupService.runBackup();
|
||||
|
||||
expect(logger.error).toHaveBeenCalledWith(
|
||||
'S3 backup failed:',
|
||||
expect.objectContaining({
|
||||
message: expect.stringContaining('S3 backup configuration incomplete')
|
||||
})
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Manifest Generation', () => {
|
||||
it('should generate and save manifest for successful backup', async () => {
|
||||
const config = {
|
||||
backup_enabled: true,
|
||||
backup_destination_type: 'local',
|
||||
backup_destination_path: '/backup',
|
||||
backup_manifest_format: 'json'
|
||||
};
|
||||
|
||||
mockDb.select.mockResolvedValue([]);
|
||||
mockDb.insert.mockResolvedValue([1]);
|
||||
mockDb.first.mockResolvedValue(null);
|
||||
|
||||
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
|
||||
|
||||
mockFs({
|
||||
'/storage/events/active/event1': {
|
||||
'photo1.jpg': Buffer.from('photo1 content')
|
||||
},
|
||||
'/backup': {}
|
||||
});
|
||||
|
||||
await backupService.runBackup();
|
||||
|
||||
expect(backupManifest.generateManifest).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
backupType: 'full',
|
||||
backupPath: '/backup',
|
||||
format: 'json'
|
||||
})
|
||||
);
|
||||
|
||||
expect(backupManifest.saveManifest).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('should generate incremental manifest when parent exists', async () => {
|
||||
const config = {
|
||||
backup_enabled: true,
|
||||
backup_destination_type: 'local',
|
||||
backup_destination_path: '/backup'
|
||||
};
|
||||
|
||||
const lastBackup = {
|
||||
id: 1,
|
||||
manifest_path: '/backup/manifests/previous.json',
|
||||
manifest_id: 'previous-backup-123'
|
||||
};
|
||||
|
||||
mockDb.select.mockResolvedValue([]);
|
||||
mockDb.insert.mockResolvedValue([2]);
|
||||
mockDb.first.mockImplementation(() => Promise.resolve(lastBackup));
|
||||
mockDb.orderBy.mockReturnThis();
|
||||
mockDb.where.mockReturnThis();
|
||||
mockDb.whereNot = jest.fn().mockReturnThis();
|
||||
|
||||
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
|
||||
|
||||
mockFs({
|
||||
'/storage/events/active': {},
|
||||
'/backup': {}
|
||||
});
|
||||
|
||||
await backupService.runBackup();
|
||||
|
||||
expect(backupManifest.loadManifest).toHaveBeenCalledWith('/backup/manifests/previous.json');
|
||||
expect(backupManifest.generateIncrementalManifest).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('should upload manifest to S3 for S3 backups', async () => {
|
||||
const config = {
|
||||
backup_enabled: true,
|
||||
backup_destination_type: 's3',
|
||||
backup_s3_bucket: 'test-bucket',
|
||||
backup_s3_access_key: 'test-key',
|
||||
backup_s3_secret_key: 'test-secret',
|
||||
backup_manifest_format: 'yaml'
|
||||
};
|
||||
|
||||
mockDb.select.mockResolvedValue([]);
|
||||
mockDb.insert.mockResolvedValue([1]);
|
||||
mockDb.first.mockResolvedValue(null);
|
||||
|
||||
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
|
||||
|
||||
const manifest = {
|
||||
backup: { id: 'backup-123' },
|
||||
version: '2.0'
|
||||
};
|
||||
backupManifest.generateManifest.mockResolvedValue(manifest);
|
||||
|
||||
mockFs({
|
||||
'/storage/events/active': {},
|
||||
'/storage/temp': {}
|
||||
});
|
||||
|
||||
await backupService.runBackup();
|
||||
|
||||
// Verify manifest was uploaded to S3
|
||||
const uploadCalls = mockS3Client.upload.mock.calls;
|
||||
const manifestUploaded = uploadCalls.some(call =>
|
||||
call[1].includes('manifests/backup-manifest-backup-123.yaml')
|
||||
);
|
||||
expect(manifestUploaded).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Backward Compatibility', () => {
|
||||
it('should support local backup destination', async () => {
|
||||
const config = {
|
||||
backup_enabled: true,
|
||||
backup_destination_type: 'local',
|
||||
backup_destination_path: '/backup/local'
|
||||
};
|
||||
|
||||
mockDb.select.mockResolvedValue([]);
|
||||
mockDb.insert.mockResolvedValue([1]);
|
||||
mockDb.first.mockResolvedValue(null);
|
||||
|
||||
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
|
||||
|
||||
mockFs({
|
||||
'/storage/events/active/event1': {
|
||||
'photo1.jpg': Buffer.from('photo1 content')
|
||||
},
|
||||
'/backup/local': {}
|
||||
});
|
||||
|
||||
await backupService.runBackup();
|
||||
|
||||
// Verify files were copied to local destination
|
||||
const fs = require('fs');
|
||||
const destPath = '/backup/local/events/active/event1/photo1.jpg';
|
||||
expect(fs.existsSync(destPath)).toBe(true);
|
||||
});
|
||||
|
||||
it('should support rsync backup destination', async () => {
|
||||
const config = {
|
||||
backup_enabled: true,
|
||||
backup_destination_type: 'rsync',
|
||||
backup_rsync_host: 'backup.example.com',
|
||||
backup_rsync_user: 'backup',
|
||||
backup_rsync_path: '/remote/backup'
|
||||
};
|
||||
|
||||
mockDb.select.mockResolvedValue([]);
|
||||
mockDb.insert.mockResolvedValue([1]);
|
||||
mockDb.first.mockResolvedValue(null);
|
||||
|
||||
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
|
||||
|
||||
// Mock exec for rsync
|
||||
const { exec } = require('child_process');
|
||||
const mockExec = jest.fn((cmd, callback) => {
|
||||
callback(null, { stdout: 'Number of files transferred: 1\nTotal file size: 1024 bytes' });
|
||||
});
|
||||
exec.mockImplementation(mockExec);
|
||||
|
||||
mockFs({
|
||||
'/storage/events/active': {}
|
||||
});
|
||||
|
||||
await backupService.runBackup();
|
||||
|
||||
expect(mockExec).toHaveBeenCalledWith(
|
||||
expect.stringContaining('rsync'),
|
||||
expect.any(Function)
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Error Handling and Recovery', () => {
|
||||
it('should handle file read errors gracefully', async () => {
|
||||
const config = {
|
||||
backup_enabled: true,
|
||||
backup_destination_type: 's3',
|
||||
backup_s3_bucket: 'test-bucket',
|
||||
backup_s3_access_key: 'test-key',
|
||||
backup_s3_secret_key: 'test-secret'
|
||||
};
|
||||
|
||||
mockDb.select.mockResolvedValue([]);
|
||||
mockDb.insert.mockResolvedValue([1]);
|
||||
mockDb.first.mockResolvedValue(null);
|
||||
|
||||
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
|
||||
|
||||
// Mock file that throws error on read
|
||||
const fs = require('fs');
|
||||
const originalCreateReadStream = fs.createReadStream;
|
||||
fs.createReadStream = jest.fn((path) => {
|
||||
if (path.includes('error.jpg')) {
|
||||
const stream = new EventEmitter();
|
||||
process.nextTick(() => stream.emit('error', new Error('File read error')));
|
||||
return stream;
|
||||
}
|
||||
return originalCreateReadStream(path);
|
||||
});
|
||||
|
||||
mockFs({
|
||||
'/storage/events/active': {
|
||||
'error.jpg': Buffer.from('content'),
|
||||
'good.jpg': Buffer.from('content')
|
||||
}
|
||||
});
|
||||
|
||||
await backupService.runBackup();
|
||||
|
||||
// Should continue with other files despite error
|
||||
expect(logger.error).toHaveBeenCalledWith(
|
||||
expect.stringContaining('Failed to backup file'),
|
||||
expect.any(Error)
|
||||
);
|
||||
|
||||
fs.createReadStream = originalCreateReadStream;
|
||||
});
|
||||
|
||||
it('should send failure email on backup error', async () => {
|
||||
const config = {
|
||||
backup_enabled: true,
|
||||
backup_destination_type: 's3',
|
||||
backup_s3_bucket: 'test-bucket',
|
||||
backup_email_on_failure: true
|
||||
};
|
||||
|
||||
const admins = [
|
||||
{ email: 'admin1@example.com', is_active: true },
|
||||
{ email: 'admin2@example.com', is_active: true }
|
||||
];
|
||||
|
||||
mockDb.select.mockResolvedValue([]);
|
||||
mockDb.insert.mockResolvedValue([1]);
|
||||
mockDb.where.mockReturnThis();
|
||||
|
||||
jest.spyOn(backupService, 'getBackupConfig')
|
||||
.mockResolvedValueOnce(config)
|
||||
.mockResolvedValueOnce(config);
|
||||
|
||||
// Force an error
|
||||
jest.spyOn(backupService, 'getFilesToBackup').mockRejectedValue(new Error('Storage error'));
|
||||
|
||||
// Mock admin users query
|
||||
db.mockImplementation((table) => {
|
||||
if (table === 'admin_users') {
|
||||
return {
|
||||
where: jest.fn().mockResolvedValue(admins)
|
||||
};
|
||||
}
|
||||
return mockDb;
|
||||
});
|
||||
|
||||
await backupService.runBackup();
|
||||
|
||||
expect(queueEmail).toHaveBeenCalledTimes(2);
|
||||
expect(queueEmail).toHaveBeenCalledWith(
|
||||
null,
|
||||
'admin1@example.com',
|
||||
'backup_failed',
|
||||
expect.objectContaining({
|
||||
error_message: 'Storage error'
|
||||
})
|
||||
);
|
||||
});
|
||||
|
||||
it('should handle concurrent backup attempts', async () => {
|
||||
const config = {
|
||||
backup_enabled: true,
|
||||
backup_destination_type: 'local',
|
||||
backup_destination_path: '/backup'
|
||||
};
|
||||
|
||||
jest.spyOn(backupService, 'getBackupConfig').mockResolvedValue(config);
|
||||
|
||||
mockFs({
|
||||
'/storage/events/active': {},
|
||||
'/backup': {}
|
||||
});
|
||||
|
||||
// Start two backups concurrently
|
||||
const backup1 = backupService.runBackup();
|
||||
const backup2 = backupService.runBackup();
|
||||
|
||||
await Promise.all([backup1, backup2]);
|
||||
|
||||
// Second backup should be skipped
|
||||
expect(logger.warn).toHaveBeenCalledWith('Backup already running, skipping');
|
||||
});
|
||||
});
|
||||
|
||||
describe('Service Lifecycle', () => {
|
||||
it('should start backup service with cron schedule', async () => {
|
||||
const config = {
|
||||
backup_enabled: true,
|
||||
backup_schedule: '0 3 * * *' // 3 AM daily
|
||||
};
|
||||
|
||||
mockDb.select.mockResolvedValue(
|
||||
Object.entries(config).map(([key, value]) => ({
|
||||
setting_key: key,
|
||||
setting_value: value.toString()
|
||||
}))
|
||||
);
|
||||
|
||||
await backupService.startBackupService();
|
||||
|
||||
expect(cron.schedule).toHaveBeenCalledWith('0 3 * * *', expect.any(Function));
|
||||
expect(logger.info).toHaveBeenCalledWith('Backup service started with schedule: 0 3 * * *');
|
||||
});
|
||||
|
||||
it('should stop existing job when restarting service', async () => {
|
||||
const config = {
|
||||
backup_enabled: true,
|
||||
backup_schedule: '0 2 * * *'
|
||||
};
|
||||
|
||||
mockDb.select.mockResolvedValue(
|
||||
Object.entries(config).map(([key, value]) => ({
|
||||
setting_key: key,
|
||||
setting_value: value.toString()
|
||||
}))
|
||||
);
|
||||
|
||||
// Start service twice
|
||||
await backupService.startBackupService();
|
||||
await backupService.startBackupService();
|
||||
|
||||
expect(mockCronJob.stop).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('should not start service when backup is disabled', async () => {
|
||||
const config = {
|
||||
backup_enabled: false
|
||||
};
|
||||
|
||||
mockDb.select.mockResolvedValue([
|
||||
{ setting_key: 'backup_enabled', setting_value: 'false' }
|
||||
]);
|
||||
|
||||
await backupService.startBackupService();
|
||||
|
||||
expect(cron.schedule).not.toHaveBeenCalled();
|
||||
expect(logger.info).toHaveBeenCalledWith('Backup service is disabled');
|
||||
});
|
||||
});
|
||||
|
||||
describe('Backup Status and History', () => {
|
||||
it('should return backup status with recent runs', async () => {
|
||||
const recentRuns = [
|
||||
{
|
||||
id: 1,
|
||||
started_at: new Date(),
|
||||
completed_at: new Date(),
|
||||
status: 'completed',
|
||||
files_backed_up: 100,
|
||||
total_size_bytes: 1024000,
|
||||
manifest_path: '/backup/manifest.json'
|
||||
}
|
||||
];
|
||||
|
||||
mockDb.limit.mockResolvedValue(recentRuns);
|
||||
|
||||
backupManifest.validateManifest.mockImplementation(() => true);
|
||||
|
||||
const status = await backupService.getBackupStatus();
|
||||
|
||||
expect(status).toEqual({
|
||||
isRunning: false,
|
||||
isHealthy: true,
|
||||
lastRun: expect.objectContaining({
|
||||
...recentRuns[0],
|
||||
manifestValid: true
|
||||
}),
|
||||
recentRuns: recentRuns,
|
||||
nextScheduledRun: expect.any(String)
|
||||
});
|
||||
});
|
||||
|
||||
it('should clean up old backup runs', async () => {
|
||||
mockDb.delete.mockResolvedValue(5);
|
||||
|
||||
await backupService.cleanupOldBackupRuns(30);
|
||||
|
||||
expect(mockDb.where).toHaveBeenCalledWith('started_at', '<', expect.any(Date));
|
||||
expect(mockDb.delete).toHaveBeenCalled();
|
||||
expect(logger.info).toHaveBeenCalledWith('Cleaned up 5 old backup runs');
|
||||
});
|
||||
});
|
||||
|
||||
describe('getBackupManifest', () => {
|
||||
it('should retrieve manifest from local filesystem', async () => {
|
||||
const backupRun = {
|
||||
id: 1,
|
||||
manifest_path: '/backup/manifests/backup-123.json'
|
||||
};
|
||||
|
||||
mockDb.first.mockResolvedValue(backupRun);
|
||||
|
||||
const manifest = { backup: { id: 'backup-123' } };
|
||||
backupManifest.loadManifest.mockResolvedValue(manifest);
|
||||
backupManifest.generateSummaryReport.mockReturnValue('Summary');
|
||||
|
||||
const result = await backupService.getBackupManifest(1);
|
||||
|
||||
expect(result).toEqual({
|
||||
manifest: manifest,
|
||||
summary: 'Summary'
|
||||
});
|
||||
});
|
||||
|
||||
it('should retrieve manifest from S3', async () => {
|
||||
const backupRun = {
|
||||
id: 1,
|
||||
manifest_path: 's3://test-bucket/backups/manifests/backup-123.json'
|
||||
};
|
||||
|
||||
mockDb.first.mockResolvedValue(backupRun);
|
||||
mockDb.select.mockResolvedValue([
|
||||
{ setting_key: 'backup_s3_access_key', setting_value: '"test-key"' },
|
||||
{ setting_key: 'backup_s3_secret_key', setting_value: '"test-secret"' }
|
||||
]);
|
||||
|
||||
const manifest = { backup: { id: 'backup-123' } };
|
||||
backupManifest.loadManifest.mockResolvedValue(manifest);
|
||||
|
||||
await backupService.getBackupManifest(1);
|
||||
|
||||
expect(S3StorageAdapter).toHaveBeenCalled();
|
||||
expect(mockS3Client.download).toHaveBeenCalledWith(
|
||||
'backups/manifests/backup-123.json',
|
||||
expect.any(String)
|
||||
);
|
||||
});
|
||||
});
|
||||
});
|
||||
Binary file not shown.
@@ -0,0 +1,161 @@
|
||||
# Security Logging Documentation
|
||||
|
||||
## Overview
|
||||
This document describes the comprehensive security logging implemented in the PicPeak application to track authentication failures, rate limiting, and suspicious activities.
|
||||
|
||||
## Log Files
|
||||
|
||||
### 1. **security.log**
|
||||
- Location: `logs/security.log`
|
||||
- Contains: All security-related events (authentication, rate limiting, suspicious activity)
|
||||
- Max Size: 20MB with rotation (keeps 10 files)
|
||||
- Format: JSON with timestamp
|
||||
|
||||
### 2. **error.log**
|
||||
- Location: `logs/error.log`
|
||||
- Contains: All error-level logs including auth failures
|
||||
- Max Size: 10MB with rotation (keeps 5 files)
|
||||
|
||||
### 3. **combined.log**
|
||||
- Location: `logs/combined.log`
|
||||
- Contains: All logs (info, warn, error)
|
||||
- Max Size: 50MB with rotation (keeps 10 files)
|
||||
|
||||
## Security Events Logged
|
||||
|
||||
### Rate Limiting
|
||||
When rate limits are exceeded, the following is logged:
|
||||
```json
|
||||
{
|
||||
"timestamp": "2024-01-18 14:23:45.123",
|
||||
"level": "warn",
|
||||
"message": "Rate limit exceeded",
|
||||
"security": true,
|
||||
"ip": "192.168.1.1",
|
||||
"path": "/api/admin/login",
|
||||
"method": "POST",
|
||||
"authenticated": false,
|
||||
"userAgent": "Mozilla/5.0...",
|
||||
"referer": "https://app.example.com",
|
||||
"origin": "https://app.example.com",
|
||||
"headers": {
|
||||
"x-forwarded-for": "192.168.1.1",
|
||||
"x-real-ip": "192.168.1.1"
|
||||
},
|
||||
"requestUrl": "/api/admin/login",
|
||||
"rateLimitInfo": {
|
||||
"limit": 5,
|
||||
"current": 6,
|
||||
"remaining": 0,
|
||||
"resetTime": "2024-01-18T14:38:45.123Z"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Authentication Failures
|
||||
|
||||
#### Admin Login Failures
|
||||
- Tracked in `login_attempts` table
|
||||
- Logged with: IP address, username, user agent, timestamp
|
||||
- Account lockout after 5 failures in 15 minutes
|
||||
|
||||
#### Gallery Password Failures
|
||||
- Tracked in `access_logs` table with action='login_fail'
|
||||
- Logged with: event_id, IP address, user agent
|
||||
- Gallery lockout after 5 failures in 15 minutes
|
||||
|
||||
### JWT Validation Failures
|
||||
```json
|
||||
{
|
||||
"timestamp": "2024-01-18 14:23:45.123",
|
||||
"level": "warn",
|
||||
"message": "JWT validation failed",
|
||||
"ip": "192.168.1.1",
|
||||
"path": "/api/admin/events",
|
||||
"method": "GET",
|
||||
"userAgent": "Mozilla/5.0...",
|
||||
"error": "TokenExpiredError",
|
||||
"message": "jwt expired"
|
||||
}
|
||||
```
|
||||
|
||||
### Suspicious Activity
|
||||
- Multiple IPs attempting login for same account
|
||||
- Token usage from different IP than issued
|
||||
- Token usage after password change
|
||||
- Revoked token usage attempts
|
||||
|
||||
## Configuration Settings
|
||||
|
||||
All rate limiting settings are configurable via the admin panel:
|
||||
|
||||
| Setting | Default | Range | Description |
|
||||
|---------|---------|-------|-------------|
|
||||
| rate_limit_enabled | true | - | Enable/disable rate limiting |
|
||||
| rate_limit_window_minutes | 15 | 1-60 | Time window for rate limit |
|
||||
| rate_limit_max_requests | 1000 | 10-10000 | Max requests for general endpoints |
|
||||
| rate_limit_auth_max_requests | 5 | 1-100 | Max requests for auth endpoints |
|
||||
| rate_limit_skip_authenticated | true | - | Skip rate limit for authenticated requests |
|
||||
| rate_limit_public_endpoints_only | false | - | Only rate limit public endpoints |
|
||||
|
||||
## Database Tables
|
||||
|
||||
### login_attempts
|
||||
```sql
|
||||
- id
|
||||
- username
|
||||
- ip_address
|
||||
- user_agent
|
||||
- success (boolean)
|
||||
- created_at
|
||||
```
|
||||
|
||||
### access_logs
|
||||
```sql
|
||||
- id
|
||||
- event_id
|
||||
- ip_address
|
||||
- user_agent
|
||||
- action ('view', 'download', 'login_success', 'login_fail')
|
||||
- photo_id (nullable)
|
||||
- created_at
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
- `LOG_LEVEL`: Set logging level (default: 'info')
|
||||
- `LOG_TO_CONSOLE`: Enable console logging in production (default: false)
|
||||
|
||||
## Monitoring Recommendations
|
||||
|
||||
1. **Set up alerts for:**
|
||||
- Rate limit exceeded events (possible DDoS)
|
||||
- Multiple failed login attempts from same IP
|
||||
- Account lockout events
|
||||
- JWT validation failures spike
|
||||
|
||||
2. **Regular review:**
|
||||
- Check security.log for patterns
|
||||
- Review login_attempts table for brute force attempts
|
||||
- Monitor access_logs for suspicious gallery access patterns
|
||||
|
||||
3. **Log analysis tools:**
|
||||
- Use log aggregation tools (ELK stack, Splunk)
|
||||
- Set up dashboards for security metrics
|
||||
- Configure alerts for threshold breaches
|
||||
|
||||
## Production Deployment Notes
|
||||
|
||||
1. Ensure logs directory has proper permissions
|
||||
2. Set up log rotation outside of application if needed
|
||||
3. Consider shipping logs to centralized logging service
|
||||
4. Monitor disk space for log files
|
||||
5. Set `LOG_TO_CONSOLE=true` for container deployments
|
||||
|
||||
## Security Best Practices
|
||||
|
||||
1. Never log sensitive data (passwords, tokens)
|
||||
2. Use generic error messages to prevent user enumeration
|
||||
3. Clean up old login attempts regularly (7 days retention)
|
||||
4. Monitor for unusual patterns in real-time
|
||||
5. Keep rate limit settings appropriate for your usage
|
||||
+4
-4
@@ -40,10 +40,10 @@ const config = {
|
||||
keepAliveInitialDelayMillis: 0
|
||||
},
|
||||
pool: {
|
||||
min: 2,
|
||||
max: 10,
|
||||
acquireTimeoutMillis: 30000,
|
||||
createTimeoutMillis: 30000,
|
||||
min: 5,
|
||||
max: 25,
|
||||
acquireTimeoutMillis: 60000,
|
||||
createTimeoutMillis: 60000,
|
||||
idleTimeoutMillis: 30000,
|
||||
reapIntervalMillis: 1000,
|
||||
createRetryIntervalMillis: 200,
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
# Database Migrations
|
||||
|
||||
This directory contains database migrations for the Wedding Photo Sharing platform.
|
||||
|
||||
## Directory Structure
|
||||
|
||||
### `/core`
|
||||
Essential migrations that are always run for new deployments. These include:
|
||||
- `init.js` - Initial database schema creation
|
||||
- Backup service tables (029-035)
|
||||
- Gallery feedback tables (033)
|
||||
|
||||
### `/legacy`
|
||||
Migrations needed only when upgrading from older versions. New deployments can skip these as the core schema already includes all necessary tables and columns.
|
||||
|
||||
## For New Deployments
|
||||
|
||||
If you're deploying this application for the first time:
|
||||
1. The `initializeDatabase()` function in `src/database/db.js` will create all necessary tables
|
||||
2. Only migrations in the `/core` directory will be run
|
||||
3. This ensures a clean, optimized database schema
|
||||
|
||||
## For Existing Deployments
|
||||
|
||||
If you're upgrading from an older version:
|
||||
1. All migrations (both core and legacy) will be run in sequence
|
||||
2. The migration system tracks which migrations have been applied
|
||||
3. Only new migrations will be executed
|
||||
|
||||
## Running Migrations
|
||||
|
||||
```bash
|
||||
# Development
|
||||
npm run migrate
|
||||
|
||||
# Production
|
||||
npm run migrate:prod
|
||||
```
|
||||
|
||||
## Note on Duplicate Migration Numbers
|
||||
|
||||
The legacy directory contains renamed duplicates:
|
||||
- `014_add_host_name_to_events_duplicate.js` (was duplicate of 014)
|
||||
- `027_add_rate_limit_settings_duplicate.js` (was duplicate of 027)
|
||||
|
||||
These have been renamed to avoid conflicts while preserving the migration history.
|
||||
@@ -1,33 +1,38 @@
|
||||
const bcrypt = require('bcrypt');
|
||||
const { db, initializeDatabase } = require('../src/database/db');
|
||||
const { generateReadablePassword } = require('../src/utils/passwordGenerator');
|
||||
const { initializeDatabase } = require('../../src/database/db');
|
||||
const { generateReadablePassword } = require('../../src/utils/passwordGenerator');
|
||||
const fs = require('fs').promises;
|
||||
const path = require('path');
|
||||
|
||||
async function runMigrations() {
|
||||
console.log('Running database migrations...');
|
||||
exports.up = async function(knex) {
|
||||
console.log('Initializing database schema...');
|
||||
|
||||
try {
|
||||
// Initialize tables
|
||||
await initializeDatabase();
|
||||
|
||||
// Create default admin user if none exists
|
||||
const adminExists = await db('admin_users').first();
|
||||
const adminExists = await knex('admin_users').first();
|
||||
if (!adminExists) {
|
||||
// Generate a secure random password
|
||||
const generatedPassword = generateReadablePassword();
|
||||
const passwordHash = await bcrypt.hash(generatedPassword, 12); // Increased rounds for better security
|
||||
|
||||
await db('admin_users').insert({
|
||||
username: 'admin',
|
||||
email: 'admin@example.com',
|
||||
// Get admin credentials from environment or use defaults
|
||||
const adminUsername = process.env.ADMIN_USERNAME || 'admin';
|
||||
const adminEmail = process.env.ADMIN_EMAIL || 'admin@example.com';
|
||||
|
||||
await knex('admin_users').insert({
|
||||
username: adminUsername,
|
||||
email: adminEmail,
|
||||
password_hash: passwordHash,
|
||||
must_change_password: true, // Flag for forcing password change
|
||||
created_at: new Date()
|
||||
});
|
||||
|
||||
// Save the generated password to a file for the user to retrieve
|
||||
const setupInfoPath = path.join(__dirname, '..', '..', 'ADMIN_CREDENTIALS.txt');
|
||||
// Try to save credentials to file, but don't fail if we can't
|
||||
const dataDir = path.join(__dirname, '..', '..', 'data');
|
||||
const setupInfoPath = path.join(dataDir, 'ADMIN_CREDENTIALS.txt');
|
||||
|
||||
const setupInfo = `
|
||||
========================================
|
||||
PicPeak Admin Credentials
|
||||
@@ -35,39 +40,49 @@ PicPeak Admin Credentials
|
||||
|
||||
Your admin account has been created with these credentials:
|
||||
|
||||
Username: admin
|
||||
Email: ${adminEmail}
|
||||
Password: ${generatedPassword}
|
||||
|
||||
IMPORTANT SECURITY NOTES:
|
||||
1. You MUST change this password on first login
|
||||
1. Please change this password after first login
|
||||
2. This file will be created only once
|
||||
3. Store these credentials securely
|
||||
4. Delete this file after noting the password
|
||||
|
||||
Login URL: ${process.env.ADMIN_URL || 'http://localhost:3001'}/admin
|
||||
|
||||
Login with the email address shown above
|
||||
|
||||
Generated on: ${new Date().toISOString()}
|
||||
========================================
|
||||
`;
|
||||
|
||||
await fs.writeFile(setupInfoPath, setupInfo, 'utf8');
|
||||
try {
|
||||
// Try to create directory and write file
|
||||
await fs.mkdir(dataDir, { recursive: true });
|
||||
await fs.writeFile(setupInfoPath, setupInfo, 'utf8');
|
||||
console.log(`📁 Credentials also saved to: data/ADMIN_CREDENTIALS.txt`);
|
||||
} catch (error) {
|
||||
// If we can't write the file, that's okay - credentials are shown in console
|
||||
console.log('⚠️ Could not save credentials to file (permission denied)');
|
||||
console.log(' Please copy the credentials shown above');
|
||||
}
|
||||
|
||||
console.log('\n========================================');
|
||||
console.log('✅ Admin user created successfully!');
|
||||
console.log('========================================');
|
||||
console.log('Username: admin');
|
||||
console.log(`Email: ${adminEmail}`);
|
||||
console.log(`Password: ${generatedPassword}`);
|
||||
console.log('\n⚠️ IMPORTANT:');
|
||||
console.log('1. Save these credentials securely');
|
||||
console.log('2. You will be required to change the password on first login');
|
||||
console.log('3. Credentials are also saved in: ADMIN_CREDENTIALS.txt');
|
||||
console.log('2. Please change the password after first login');
|
||||
console.log('========================================\n');
|
||||
}
|
||||
|
||||
// Create default email templates if none exist
|
||||
const templateExists = await db('email_templates').first();
|
||||
const templateExists = await knex('email_templates').first();
|
||||
if (!templateExists) {
|
||||
await db('email_templates').insert([
|
||||
await knex('email_templates').insert([
|
||||
{
|
||||
template_key: 'gallery_created',
|
||||
subject: 'Your Photo Gallery is Ready!',
|
||||
@@ -101,9 +116,9 @@ Generated on: ${new Date().toISOString()}
|
||||
}
|
||||
|
||||
// Create default email config if none exists
|
||||
const emailConfig = await db('email_configs').first();
|
||||
const emailConfig = await knex('email_configs').first();
|
||||
if (!emailConfig) {
|
||||
await db('email_configs').insert({
|
||||
await knex('email_configs').insert({
|
||||
smtp_host: process.env.SMTP_HOST || 'mailhog',
|
||||
smtp_port: process.env.SMTP_PORT || 1025,
|
||||
smtp_secure: process.env.SMTP_SECURE === 'true',
|
||||
@@ -116,11 +131,13 @@ Generated on: ${new Date().toISOString()}
|
||||
}
|
||||
|
||||
console.log('Migrations completed successfully');
|
||||
process.exit(0);
|
||||
} catch (error) {
|
||||
console.error('Migration failed:', error);
|
||||
process.exit(1);
|
||||
console.error('Initial setup failed:', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
runMigrations();
|
||||
exports.down = async function(knex) {
|
||||
// This migration cannot be rolled back as it creates the initial schema
|
||||
console.log('Initial setup cannot be rolled back');
|
||||
};
|
||||
@@ -0,0 +1,220 @@
|
||||
const { db } = require('../../src/database/db');
|
||||
|
||||
async function up() {
|
||||
console.log('Adding backup service tables and settings...');
|
||||
|
||||
// Create backup_runs table to track backup history
|
||||
const hasBackupRunsTable = await db.schema.hasTable('backup_runs');
|
||||
if (!hasBackupRunsTable) {
|
||||
await db.schema.createTable('backup_runs', (table) => {
|
||||
table.increments('id').primary();
|
||||
table.datetime('started_at').notNullable();
|
||||
table.datetime('completed_at');
|
||||
table.string('status').defaultTo('running'); // running, completed, failed
|
||||
table.string('backup_type'); // full, incremental
|
||||
table.integer('files_backed_up').defaultTo(0);
|
||||
table.bigInteger('total_size_bytes').defaultTo(0);
|
||||
table.integer('duration_seconds');
|
||||
table.text('error_message');
|
||||
table.json('statistics'); // Detailed stats about the backup
|
||||
table.json('file_checksums'); // Store checksums for change detection
|
||||
});
|
||||
}
|
||||
|
||||
// Create backup_file_states table to track individual file states
|
||||
const hasBackupFileStatesTable = await db.schema.hasTable('backup_file_states');
|
||||
if (!hasBackupFileStatesTable) {
|
||||
await db.schema.createTable('backup_file_states', (table) => {
|
||||
table.increments('id').primary();
|
||||
table.string('file_path').notNullable();
|
||||
table.string('checksum').notNullable();
|
||||
table.bigInteger('size_bytes');
|
||||
table.datetime('last_modified');
|
||||
table.datetime('last_backed_up');
|
||||
table.boolean('is_archived').defaultTo(false);
|
||||
table.index(['file_path'], 'idx_backup_file_path');
|
||||
table.index(['checksum'], 'idx_backup_checksum');
|
||||
});
|
||||
}
|
||||
|
||||
// Add backup-related settings to app_settings
|
||||
const backupSettings = [
|
||||
{
|
||||
setting_key: 'backup_enabled',
|
||||
setting_value: JSON.stringify(false),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_schedule',
|
||||
setting_value: JSON.stringify('0 2 * * *'), // Default: 2 AM daily
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_destination_type',
|
||||
setting_value: JSON.stringify('local'), // local, rsync, s3
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_destination_path',
|
||||
setting_value: JSON.stringify('/backup/picpeak'),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_rsync_host',
|
||||
setting_value: JSON.stringify(''),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_rsync_user',
|
||||
setting_value: JSON.stringify(''),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_rsync_path',
|
||||
setting_value: JSON.stringify(''),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_rsync_ssh_key',
|
||||
setting_value: JSON.stringify(''),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_s3_endpoint',
|
||||
setting_value: JSON.stringify(''),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_s3_bucket',
|
||||
setting_value: JSON.stringify(''),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_s3_access_key',
|
||||
setting_value: JSON.stringify(''),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_s3_secret_key',
|
||||
setting_value: JSON.stringify(''),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_s3_region',
|
||||
setting_value: JSON.stringify('us-east-1'),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_retention_days',
|
||||
setting_value: JSON.stringify(30),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_include_archived',
|
||||
setting_value: JSON.stringify(true),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_compression',
|
||||
setting_value: JSON.stringify(true),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_email_on_failure',
|
||||
setting_value: JSON.stringify(true),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_email_on_success',
|
||||
setting_value: JSON.stringify(false),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_max_file_size_mb',
|
||||
setting_value: JSON.stringify(5000), // Skip files larger than 5GB
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_exclude_patterns',
|
||||
setting_value: JSON.stringify(['*.tmp', '.DS_Store', 'Thumbs.db']),
|
||||
setting_type: 'backup'
|
||||
}
|
||||
];
|
||||
|
||||
// Insert backup settings if they don't exist
|
||||
for (const setting of backupSettings) {
|
||||
const exists = await db('app_settings')
|
||||
.where('setting_key', setting.setting_key)
|
||||
.first();
|
||||
|
||||
if (!exists) {
|
||||
await db('app_settings').insert(setting);
|
||||
}
|
||||
}
|
||||
|
||||
// Add backup-related email templates
|
||||
const backupEmailTemplates = [
|
||||
{
|
||||
template_key: 'backup_failed',
|
||||
subject: 'Backup Failed - Immediate Attention Required',
|
||||
body_html: `<h2>Backup Failed</h2>
|
||||
<p>The scheduled backup has failed and requires immediate attention.</p>
|
||||
<p><strong>Error Details:</strong></p>
|
||||
<ul>
|
||||
<li>Start Time: {{start_time}}</li>
|
||||
<li>Backup Type: {{backup_type}}</li>
|
||||
<li>Error: {{error_message}}</li>
|
||||
</ul>
|
||||
<p>Please check the system logs for more details and resolve the issue as soon as possible.</p>`,
|
||||
body_text: 'Backup Failed\n\nThe scheduled backup has failed and requires immediate attention.\n\nStart Time: {{start_time}}\nBackup Type: {{backup_type}}\nError: {{error_message}}\n\nPlease check the system logs for more details.',
|
||||
variables: JSON.stringify(['start_time', 'backup_type', 'error_message'])
|
||||
},
|
||||
{
|
||||
template_key: 'backup_completed',
|
||||
subject: 'Backup Completed Successfully',
|
||||
body_html: `<h2>Backup Completed</h2>
|
||||
<p>The scheduled backup has been completed successfully.</p>
|
||||
<p><strong>Backup Summary:</strong></p>
|
||||
<ul>
|
||||
<li>Start Time: {{start_time}}</li>
|
||||
<li>Duration: {{duration}}</li>
|
||||
<li>Files Backed Up: {{files_count}}</li>
|
||||
<li>Total Size: {{total_size}}</li>
|
||||
<li>Backup Type: {{backup_type}}</li>
|
||||
</ul>`,
|
||||
body_text: 'Backup Completed\n\nThe scheduled backup has been completed successfully.\n\nStart Time: {{start_time}}\nDuration: {{duration}}\nFiles Backed Up: {{files_count}}\nTotal Size: {{total_size}}\nBackup Type: {{backup_type}}',
|
||||
variables: JSON.stringify(['start_time', 'duration', 'files_count', 'total_size', 'backup_type'])
|
||||
}
|
||||
];
|
||||
|
||||
// Insert backup email templates if they don't exist
|
||||
for (const template of backupEmailTemplates) {
|
||||
const exists = await db('email_templates')
|
||||
.where('template_key', template.template_key)
|
||||
.first();
|
||||
|
||||
if (!exists) {
|
||||
await db('email_templates').insert(template);
|
||||
}
|
||||
}
|
||||
|
||||
console.log('Backup service tables and settings added successfully');
|
||||
}
|
||||
|
||||
async function down() {
|
||||
// Remove backup tables
|
||||
await db.schema.dropTableIfExists('backup_file_states');
|
||||
await db.schema.dropTableIfExists('backup_runs');
|
||||
|
||||
// Remove backup settings
|
||||
await db('app_settings')
|
||||
.where('setting_type', 'backup')
|
||||
.delete();
|
||||
|
||||
// Remove backup email templates
|
||||
await db('email_templates')
|
||||
.whereIn('template_key', ['backup_failed', 'backup_completed'])
|
||||
.delete();
|
||||
}
|
||||
|
||||
module.exports = { up, down };
|
||||
@@ -0,0 +1,159 @@
|
||||
const { db } = require('../../src/database/db');
|
||||
|
||||
async function up() {
|
||||
console.log('Adding database backup tables and settings...');
|
||||
|
||||
// Create database_backup_runs table to track database backup history
|
||||
const hasDatabaseBackupRunsTable = await db.schema.hasTable('database_backup_runs');
|
||||
if (!hasDatabaseBackupRunsTable) {
|
||||
await db.schema.createTable('database_backup_runs', (table) => {
|
||||
table.increments('id').primary();
|
||||
table.datetime('started_at').notNullable();
|
||||
table.datetime('completed_at');
|
||||
table.string('status').defaultTo('running'); // running, completed, failed
|
||||
table.string('backup_type'); // sqlite, postgresql
|
||||
table.string('destination_path');
|
||||
table.string('file_path');
|
||||
table.bigInteger('file_size_bytes').defaultTo(0);
|
||||
table.bigInteger('original_size_bytes').defaultTo(0);
|
||||
table.integer('duration_seconds');
|
||||
table.string('checksum'); // SHA256 checksum of backup file
|
||||
table.float('compression_ratio'); // Compression percentage
|
||||
table.json('table_checksums'); // Individual table checksums
|
||||
table.text('error_message');
|
||||
table.json('statistics'); // Detailed stats about the backup
|
||||
table.index(['started_at'], 'idx_db_backup_started');
|
||||
table.index(['status'], 'idx_db_backup_status');
|
||||
});
|
||||
}
|
||||
|
||||
// Add database backup-related settings to app_settings
|
||||
const databaseBackupSettings = [
|
||||
{
|
||||
setting_key: 'database_backup_enabled',
|
||||
setting_value: JSON.stringify(false),
|
||||
setting_type: 'database_backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'database_backup_schedule',
|
||||
setting_value: JSON.stringify('0 3 * * *'), // Default: 3 AM daily
|
||||
setting_type: 'database_backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'database_backup_destination_path',
|
||||
setting_value: JSON.stringify('/backup/database'),
|
||||
setting_type: 'database_backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'database_backup_compress',
|
||||
setting_value: JSON.stringify(true),
|
||||
setting_type: 'database_backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'database_backup_validate_integrity',
|
||||
setting_value: JSON.stringify(true),
|
||||
setting_type: 'database_backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'database_backup_include_checksums',
|
||||
setting_value: JSON.stringify(true),
|
||||
setting_type: 'database_backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'database_backup_retention_days',
|
||||
setting_value: JSON.stringify(30),
|
||||
setting_type: 'database_backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'database_backup_email_on_failure',
|
||||
setting_value: JSON.stringify(true),
|
||||
setting_type: 'database_backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'database_backup_email_on_success',
|
||||
setting_value: JSON.stringify(false),
|
||||
setting_type: 'database_backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'database_backup_max_retries',
|
||||
setting_value: JSON.stringify(3),
|
||||
setting_type: 'database_backup'
|
||||
}
|
||||
];
|
||||
|
||||
// Insert database backup settings if they don't exist
|
||||
for (const setting of databaseBackupSettings) {
|
||||
const exists = await db('app_settings')
|
||||
.where('setting_key', setting.setting_key)
|
||||
.first();
|
||||
|
||||
if (!exists) {
|
||||
await db('app_settings').insert(setting);
|
||||
}
|
||||
}
|
||||
|
||||
// Add database backup-related email templates
|
||||
const databaseBackupEmailTemplates = [
|
||||
{
|
||||
template_key: 'database_backup_failed',
|
||||
subject: 'Database Backup Failed - Critical Alert',
|
||||
body_html: `<h2>Database Backup Failed</h2>
|
||||
<p>The scheduled database backup has failed and requires immediate attention.</p>
|
||||
<p><strong>Error Details:</strong></p>
|
||||
<ul>
|
||||
<li>Backup Type: {{backup_type}}</li>
|
||||
<li>Timestamp: {{timestamp}}</li>
|
||||
<li>Error: {{error_message}}</li>
|
||||
</ul>
|
||||
<p>This is a critical issue that could affect disaster recovery. Please investigate immediately.</p>`,
|
||||
body_text: 'Database Backup Failed\n\nThe scheduled database backup has failed.\n\nBackup Type: {{backup_type}}\nTimestamp: {{timestamp}}\nError: {{error_message}}\n\nThis is critical - please investigate immediately.',
|
||||
variables: JSON.stringify(['backup_type', 'timestamp', 'error_message'])
|
||||
},
|
||||
{
|
||||
template_key: 'database_backup_completed',
|
||||
subject: 'Database Backup Completed Successfully',
|
||||
body_html: `<h2>Database Backup Completed</h2>
|
||||
<p>The scheduled database backup has been completed successfully.</p>
|
||||
<p><strong>Backup Summary:</strong></p>
|
||||
<ul>
|
||||
<li>Backup Type: {{backup_type}}</li>
|
||||
<li>Duration: {{duration}}</li>
|
||||
<li>File Size: {{file_size}}</li>
|
||||
<li>Compression Ratio: {{compression_ratio}}</li>
|
||||
<li>File Path: {{file_path}}</li>
|
||||
</ul>`,
|
||||
body_text: 'Database Backup Completed\n\nThe scheduled database backup has been completed successfully.\n\nBackup Type: {{backup_type}}\nDuration: {{duration}}\nFile Size: {{file_size}}\nCompression Ratio: {{compression_ratio}}\nFile Path: {{file_path}}',
|
||||
variables: JSON.stringify(['backup_type', 'duration', 'file_size', 'compression_ratio', 'file_path'])
|
||||
}
|
||||
];
|
||||
|
||||
// Insert database backup email templates if they don't exist
|
||||
for (const template of databaseBackupEmailTemplates) {
|
||||
const exists = await db('email_templates')
|
||||
.where('template_key', template.template_key)
|
||||
.first();
|
||||
|
||||
if (!exists) {
|
||||
await db('email_templates').insert(template);
|
||||
}
|
||||
}
|
||||
|
||||
console.log('Database backup tables and settings added successfully');
|
||||
}
|
||||
|
||||
async function down() {
|
||||
// Remove database backup tables
|
||||
await db.schema.dropTableIfExists('database_backup_runs');
|
||||
|
||||
// Remove database backup settings
|
||||
await db('app_settings')
|
||||
.where('setting_type', 'database_backup')
|
||||
.delete();
|
||||
|
||||
// Remove database backup email templates
|
||||
await db('email_templates')
|
||||
.whereIn('template_key', ['database_backup_failed', 'database_backup_completed'])
|
||||
.delete();
|
||||
}
|
||||
|
||||
module.exports = { up, down };
|
||||
@@ -0,0 +1,86 @@
|
||||
const { db } = require('../../src/database/db');
|
||||
const logger = require('../../src/utils/logger');
|
||||
|
||||
async function up() {
|
||||
console.log('Adding backup manifest columns...');
|
||||
|
||||
// Add manifest columns to backup_runs table
|
||||
const hasManifestPath = await db.schema.hasColumn('backup_runs', 'manifest_path');
|
||||
if (!hasManifestPath) {
|
||||
await db.schema.alterTable('backup_runs', (table) => {
|
||||
table.string('manifest_path'); // Path to the manifest file
|
||||
table.string('manifest_id'); // Unique manifest ID
|
||||
table.string('manifest_format').defaultTo('json'); // json or yaml
|
||||
});
|
||||
}
|
||||
|
||||
// Add backup manifest-related settings to app_settings
|
||||
const manifestSettings = [
|
||||
{
|
||||
setting_key: 'backup_manifest_enabled',
|
||||
setting_value: JSON.stringify(true),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_manifest_format',
|
||||
setting_value: JSON.stringify('json'), // json or yaml
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_manifest_path',
|
||||
setting_value: JSON.stringify('/backup/manifests'),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_manifest_validate',
|
||||
setting_value: JSON.stringify(true),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_manifest_include_checksums',
|
||||
setting_value: JSON.stringify(true),
|
||||
setting_type: 'backup'
|
||||
}
|
||||
];
|
||||
|
||||
// Insert manifest settings if they don't exist
|
||||
for (const setting of manifestSettings) {
|
||||
const exists = await db('app_settings')
|
||||
.where('setting_key', setting.setting_key)
|
||||
.first();
|
||||
|
||||
if (!exists) {
|
||||
await db('app_settings').insert(setting);
|
||||
}
|
||||
}
|
||||
|
||||
console.log('✓ Backup manifest columns and settings added');
|
||||
}
|
||||
|
||||
async function down() {
|
||||
// Remove manifest columns from backup_runs table
|
||||
const hasManifestPath = await db.schema.hasColumn('backup_runs', 'manifest_path');
|
||||
if (hasManifestPath) {
|
||||
await db.schema.alterTable('backup_runs', (table) => {
|
||||
table.dropColumn('manifest_path');
|
||||
table.dropColumn('manifest_id');
|
||||
table.dropColumn('manifest_format');
|
||||
});
|
||||
}
|
||||
|
||||
// Remove manifest settings
|
||||
await db('app_settings')
|
||||
.where('setting_type', 'backup')
|
||||
.whereIn('setting_key', [
|
||||
'backup_manifest_enabled',
|
||||
'backup_manifest_format',
|
||||
'backup_manifest_path',
|
||||
'backup_manifest_validate',
|
||||
'backup_manifest_include_checksums'
|
||||
])
|
||||
.delete();
|
||||
|
||||
console.log('✓ Backup manifest columns and settings removed');
|
||||
}
|
||||
|
||||
module.exports = { up, down };
|
||||
@@ -0,0 +1,234 @@
|
||||
// No helpers needed for this migration
|
||||
|
||||
/**
|
||||
* Add restore_runs table for tracking restore operations
|
||||
*/
|
||||
exports.up = async function(knex) {
|
||||
// Create restore_runs table
|
||||
const hasRestoreRunsTable = await knex.schema.hasTable('restore_runs');
|
||||
if (!hasRestoreRunsTable) {
|
||||
await knex.schema.createTable('restore_runs', table => {
|
||||
table.increments('id').primary();
|
||||
|
||||
// Timing
|
||||
table.timestamp('started_at').notNullable().defaultTo(knex.fn.now());
|
||||
table.timestamp('completed_at');
|
||||
table.integer('duration_seconds');
|
||||
|
||||
// Status and type
|
||||
table.string('status', 50).notNullable().defaultTo('running');
|
||||
table.string('restore_type', 50).notNullable(); // full, database, files, selective
|
||||
|
||||
// Source information
|
||||
table.string('source', 500).notNullable(); // Backup source path or S3 URL
|
||||
table.string('manifest_path', 500); // Path to manifest file
|
||||
|
||||
// Results
|
||||
table.text('error_message');
|
||||
table.text('statistics'); // JSON object with detailed statistics
|
||||
table.text('restore_log'); // JSON array of log entries
|
||||
|
||||
// Safety backup
|
||||
table.string('pre_restore_backup_path', 500); // Path to pre-restore safety backup
|
||||
|
||||
// Flags
|
||||
table.boolean('is_dry_run').defaultTo(false);
|
||||
table.boolean('was_rollback_attempted').defaultTo(false);
|
||||
table.boolean('was_successful').defaultTo(false);
|
||||
|
||||
// Operator information
|
||||
table.string('operator_type', 50).defaultTo('manual'); // manual, scheduled, api
|
||||
table.integer('operator_user_id').references('id').inTable('admin_users').onDelete('SET NULL');
|
||||
table.string('operator_ip', 50);
|
||||
|
||||
// Metadata
|
||||
table.text('metadata'); // JSON object for additional data
|
||||
|
||||
table.index(['status', 'started_at']);
|
||||
table.index(['restore_type', 'started_at']);
|
||||
});
|
||||
}
|
||||
|
||||
// Create restore_file_operations table for tracking individual file operations
|
||||
const hasRestoreFileOperationsTable = await knex.schema.hasTable('restore_file_operations');
|
||||
if (!hasRestoreFileOperationsTable) {
|
||||
await knex.schema.createTable('restore_file_operations', table => {
|
||||
table.increments('id').primary();
|
||||
|
||||
table.integer('restore_run_id').notNullable()
|
||||
.references('id').inTable('restore_runs').onDelete('CASCADE');
|
||||
|
||||
table.string('file_path', 500).notNullable();
|
||||
table.string('operation', 50).notNullable(); // restore, skip, error
|
||||
table.string('status', 50).notNullable(); // pending, in_progress, completed, failed
|
||||
|
||||
table.bigInteger('file_size');
|
||||
table.string('checksum', 64);
|
||||
table.boolean('checksum_verified').defaultTo(false);
|
||||
|
||||
table.text('error_message');
|
||||
table.timestamp('started_at');
|
||||
table.timestamp('completed_at');
|
||||
|
||||
table.index(['restore_run_id', 'status']);
|
||||
table.index(['file_path']);
|
||||
});
|
||||
}
|
||||
|
||||
// Create restore_validation_results table
|
||||
const hasRestoreValidationResultsTable = await knex.schema.hasTable('restore_validation_results');
|
||||
if (!hasRestoreValidationResultsTable) {
|
||||
await knex.schema.createTable('restore_validation_results', table => {
|
||||
table.increments('id').primary();
|
||||
|
||||
table.integer('restore_run_id').notNullable()
|
||||
.references('id').inTable('restore_runs').onDelete('CASCADE');
|
||||
|
||||
table.string('validation_type', 50).notNullable(); // pre-restore, post-restore
|
||||
table.boolean('is_valid').notNullable();
|
||||
|
||||
table.text('errors'); // JSON array of errors
|
||||
table.text('warnings'); // JSON array of warnings
|
||||
table.text('checksums'); // JSON object with checksum comparisons
|
||||
|
||||
table.timestamp('validated_at').notNullable().defaultTo(knex.fn.now());
|
||||
|
||||
table.index(['restore_run_id', 'validation_type']);
|
||||
});
|
||||
}
|
||||
|
||||
// Add restore-related settings to app_settings
|
||||
const restoreSettings = [
|
||||
{
|
||||
setting_key: 'restore_allow_force',
|
||||
setting_value: JSON.stringify(false),
|
||||
setting_type: 'restore'
|
||||
},
|
||||
{
|
||||
setting_key: 'restore_require_pre_backup',
|
||||
setting_value: JSON.stringify(true),
|
||||
setting_type: 'restore'
|
||||
},
|
||||
{
|
||||
setting_key: 'restore_max_file_size_mb',
|
||||
setting_value: '5000',
|
||||
setting_type: 'restore'
|
||||
},
|
||||
{
|
||||
setting_key: 'restore_verify_checksums',
|
||||
setting_value: JSON.stringify(true),
|
||||
setting_type: 'restore'
|
||||
},
|
||||
{
|
||||
setting_key: 'restore_email_on_completion',
|
||||
setting_value: JSON.stringify(true),
|
||||
setting_type: 'restore'
|
||||
},
|
||||
{
|
||||
setting_key: 'restore_retention_days',
|
||||
setting_value: '30',
|
||||
setting_type: 'restore'
|
||||
}
|
||||
];
|
||||
|
||||
for (const setting of restoreSettings) {
|
||||
const exists = await knex('app_settings')
|
||||
.where('setting_key', setting.setting_key)
|
||||
.first();
|
||||
|
||||
if (!exists) {
|
||||
await knex('app_settings').insert(setting);
|
||||
}
|
||||
}
|
||||
|
||||
// Add new email templates for restore notifications
|
||||
const emailTemplates = [
|
||||
{
|
||||
template_key: 'restore_completed',
|
||||
subject: '✅ Restore Completed Successfully',
|
||||
body_html: `<h2>Restore Operation Completed</h2>
|
||||
<p>A restore operation has completed successfully.</p>
|
||||
|
||||
<h3>Details:</h3>
|
||||
<ul>
|
||||
<li><strong>Restore Type:</strong> {{restore_type}}</li>
|
||||
<li><strong>Duration:</strong> {{duration}}</li>
|
||||
<li><strong>Files Restored:</strong> {{files_restored}}</li>
|
||||
<li><strong>Backup ID:</strong> {{backup_id}}</li>
|
||||
<li><strong>Timestamp:</strong> {{timestamp}}</li>
|
||||
</ul>
|
||||
|
||||
<p>Please verify that all systems are functioning correctly after the restore.</p>`,
|
||||
body_text: `Restore Operation Completed
|
||||
|
||||
A restore operation has completed successfully.
|
||||
|
||||
Details:
|
||||
- Restore Type: {{restore_type}}
|
||||
- Duration: {{duration}}
|
||||
- Files Restored: {{files_restored}}
|
||||
- Backup ID: {{backup_id}}
|
||||
- Timestamp: {{timestamp}}
|
||||
|
||||
Please verify that all systems are functioning correctly after the restore.`,
|
||||
variables: JSON.stringify(['restore_type', 'duration', 'files_restored', 'backup_id', 'timestamp'])
|
||||
},
|
||||
{
|
||||
template_key: 'restore_failed',
|
||||
subject: '❌ Restore Operation Failed',
|
||||
body_html: `<h2>Restore Operation Failed</h2>
|
||||
<p>A restore operation has failed and requires attention.</p>
|
||||
|
||||
<h3>Details:</h3>
|
||||
<ul>
|
||||
<li><strong>Restore Type:</strong> {{restore_type}}</li>
|
||||
<li><strong>Error:</strong> {{error_message}}</li>
|
||||
<li><strong>Timestamp:</strong> {{timestamp}}</li>
|
||||
</ul>
|
||||
|
||||
<p>Please check the system logs for more details and take appropriate action.</p>
|
||||
|
||||
<p><strong>Important:</strong> If a pre-restore backup was created, it may be used for recovery.</p>`,
|
||||
body_text: `Restore Operation Failed
|
||||
|
||||
A restore operation has failed and requires attention.
|
||||
|
||||
Details:
|
||||
- Restore Type: {{restore_type}}
|
||||
- Error: {{error_message}}
|
||||
- Timestamp: {{timestamp}}
|
||||
|
||||
Please check the system logs for more details and take appropriate action.
|
||||
|
||||
Important: If a pre-restore backup was created, it may be used for recovery.`,
|
||||
variables: JSON.stringify(['restore_type', 'error_message', 'timestamp'])
|
||||
}
|
||||
];
|
||||
|
||||
for (const template of emailTemplates) {
|
||||
const exists = await knex('email_templates')
|
||||
.where('template_key', template.template_key)
|
||||
.first();
|
||||
|
||||
if (!exists) {
|
||||
await knex('email_templates').insert(template);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
exports.down = async function(knex) {
|
||||
// Remove email templates
|
||||
await knex('email_templates')
|
||||
.whereIn('template_key', ['restore_completed', 'restore_failed'])
|
||||
.delete();
|
||||
|
||||
// Remove settings
|
||||
await knex('app_settings')
|
||||
.where('setting_type', 'restore')
|
||||
.delete();
|
||||
|
||||
// Drop tables
|
||||
await knex.schema.dropTableIfExists('restore_validation_results');
|
||||
await knex.schema.dropTableIfExists('restore_file_operations');
|
||||
await knex.schema.dropTableIfExists('restore_runs');
|
||||
};
|
||||
@@ -0,0 +1,129 @@
|
||||
// No helpers needed for boolean values
|
||||
|
||||
exports.up = async function(knex) {
|
||||
console.log('Adding gallery feedback tables...');
|
||||
|
||||
// Create event_feedback_settings table
|
||||
await knex.schema.createTable('event_feedback_settings', (table) => {
|
||||
table.increments('id').primary();
|
||||
table.integer('event_id').references('id').inTable('events').onDelete('CASCADE');
|
||||
table.boolean('feedback_enabled').defaultTo(false);
|
||||
table.boolean('allow_ratings').defaultTo(true);
|
||||
table.boolean('allow_likes').defaultTo(true);
|
||||
table.boolean('allow_comments').defaultTo(false);
|
||||
table.boolean('allow_favorites').defaultTo(true);
|
||||
table.boolean('require_name_email').defaultTo(false);
|
||||
table.boolean('moderate_comments').defaultTo(true);
|
||||
table.boolean('show_feedback_to_guests').defaultTo(true);
|
||||
table.timestamp('created_at').defaultTo(knex.fn.now());
|
||||
table.timestamp('updated_at').defaultTo(knex.fn.now());
|
||||
table.unique(['event_id']);
|
||||
});
|
||||
|
||||
// Create photo_feedback table
|
||||
await knex.schema.createTable('photo_feedback', (table) => {
|
||||
table.increments('id').primary();
|
||||
table.integer('photo_id').references('id').inTable('photos').onDelete('CASCADE');
|
||||
table.integer('event_id').references('id').inTable('events').onDelete('CASCADE');
|
||||
table.string('feedback_type', 20).notNullable();
|
||||
table.integer('rating');
|
||||
table.text('comment_text');
|
||||
table.string('guest_name', 100);
|
||||
table.string('guest_email', 255);
|
||||
table.string('guest_identifier', 64);
|
||||
table.string('ip_address', 45);
|
||||
table.text('user_agent');
|
||||
table.boolean('is_approved').defaultTo(true);
|
||||
table.boolean('is_hidden').defaultTo(false);
|
||||
table.timestamp('created_at').defaultTo(knex.fn.now());
|
||||
table.timestamp('updated_at').defaultTo(knex.fn.now());
|
||||
|
||||
// Add indexes
|
||||
table.index(['photo_id']);
|
||||
table.index(['event_id']);
|
||||
table.index(['feedback_type']);
|
||||
table.index(['guest_identifier']);
|
||||
|
||||
// Add check constraint for rating (PostgreSQL)
|
||||
if (knex.client.config.client === 'pg') {
|
||||
table.check('?? >= 1 AND ?? <= 5', ['rating', 'rating']);
|
||||
}
|
||||
});
|
||||
|
||||
// Create feedback_rate_limits table
|
||||
await knex.schema.createTable('feedback_rate_limits', (table) => {
|
||||
table.increments('id').primary();
|
||||
table.string('identifier', 64).notNullable();
|
||||
table.integer('event_id').references('id').inTable('events').onDelete('CASCADE');
|
||||
table.string('action_type', 20).notNullable();
|
||||
table.integer('action_count').defaultTo(1);
|
||||
table.timestamp('window_start').defaultTo(knex.fn.now());
|
||||
|
||||
// Add indexes
|
||||
table.index(['identifier', 'event_id', 'action_type']);
|
||||
table.index(['window_start']);
|
||||
});
|
||||
|
||||
// Create feedback_word_filters table
|
||||
await knex.schema.createTable('feedback_word_filters', (table) => {
|
||||
table.increments('id').primary();
|
||||
table.string('word', 100).notNullable();
|
||||
table.string('severity', 20).defaultTo('moderate');
|
||||
table.boolean('is_active').defaultTo(true);
|
||||
table.timestamp('created_at').defaultTo(knex.fn.now());
|
||||
table.unique(['word']);
|
||||
});
|
||||
|
||||
// Add feedback summary columns to photos table
|
||||
await knex.schema.alterTable('photos', (table) => {
|
||||
table.integer('feedback_count').defaultTo(0);
|
||||
table.integer('like_count').defaultTo(0);
|
||||
table.decimal('average_rating', 3, 2).defaultTo(0);
|
||||
table.integer('favorite_count').defaultTo(0);
|
||||
});
|
||||
|
||||
// Add feedback notification settings to app_settings
|
||||
await knex('app_settings').insert([
|
||||
{
|
||||
setting_key: 'feedback_notification_email',
|
||||
setting_value: JSON.stringify(''),
|
||||
setting_type: 'feedback'
|
||||
},
|
||||
{
|
||||
setting_key: 'feedback_rate_limits',
|
||||
setting_value: JSON.stringify({
|
||||
rating: { max: 100, window: 3600 }, // 100 ratings per hour
|
||||
comment: { max: 20, window: 3600 }, // 20 comments per hour
|
||||
like: { max: 200, window: 3600 } // 200 likes per hour
|
||||
}),
|
||||
setting_type: 'feedback'
|
||||
}
|
||||
]);
|
||||
|
||||
console.log('Gallery feedback tables created successfully');
|
||||
};
|
||||
|
||||
exports.down = async function(knex) {
|
||||
console.log('Removing gallery feedback tables...');
|
||||
|
||||
// Remove feedback settings from app_settings
|
||||
await knex('app_settings')
|
||||
.whereIn('setting_key', ['feedback_notification_email', 'feedback_rate_limits'])
|
||||
.delete();
|
||||
|
||||
// Remove feedback columns from photos table
|
||||
await knex.schema.alterTable('photos', (table) => {
|
||||
table.dropColumn('feedback_count');
|
||||
table.dropColumn('like_count');
|
||||
table.dropColumn('average_rating');
|
||||
table.dropColumn('favorite_count');
|
||||
});
|
||||
|
||||
// Drop tables in reverse order
|
||||
await knex.schema.dropTableIfExists('feedback_word_filters');
|
||||
await knex.schema.dropTableIfExists('feedback_rate_limits');
|
||||
await knex.schema.dropTableIfExists('photo_feedback');
|
||||
await knex.schema.dropTableIfExists('event_feedback_settings');
|
||||
|
||||
console.log('Gallery feedback tables removed');
|
||||
};
|
||||
@@ -0,0 +1,139 @@
|
||||
const { db } = require('../../src/database/db');
|
||||
|
||||
async function up() {
|
||||
console.log('Adding version tracking to backup tables...');
|
||||
|
||||
// Add version columns to database_backup_runs table
|
||||
const hasDatabaseBackupRunsTable = await db.schema.hasTable('database_backup_runs');
|
||||
if (hasDatabaseBackupRunsTable) {
|
||||
const hasAppVersion = await db.schema.hasColumn('database_backup_runs', 'app_version');
|
||||
if (!hasAppVersion) {
|
||||
await db.schema.alterTable('database_backup_runs', (table) => {
|
||||
table.string('app_version'); // Application version
|
||||
table.string('node_version'); // Node.js version
|
||||
table.string('db_schema_version'); // Database schema version (migration name)
|
||||
table.json('environment_info'); // Additional environment information
|
||||
});
|
||||
console.log('Added version columns to database_backup_runs table');
|
||||
}
|
||||
}
|
||||
|
||||
// Add version columns to backup_runs table (file backups)
|
||||
const hasBackupRunsTable = await db.schema.hasTable('backup_runs');
|
||||
if (hasBackupRunsTable) {
|
||||
const hasAppVersion = await db.schema.hasColumn('backup_runs', 'app_version');
|
||||
if (!hasAppVersion) {
|
||||
await db.schema.alterTable('backup_runs', (table) => {
|
||||
table.string('app_version'); // Application version
|
||||
table.string('node_version'); // Node.js version
|
||||
table.string('db_schema_version'); // Database schema version
|
||||
table.json('manifest_info'); // Manifest summary information
|
||||
});
|
||||
console.log('Added version columns to backup_runs table');
|
||||
}
|
||||
}
|
||||
|
||||
// Add restore tracking table
|
||||
const hasRestoreHistoryTable = await db.schema.hasTable('restore_history');
|
||||
if (!hasRestoreHistoryTable) {
|
||||
await db.schema.createTable('restore_history', (table) => {
|
||||
table.increments('id').primary();
|
||||
table.datetime('started_at').notNullable();
|
||||
table.datetime('completed_at');
|
||||
table.string('status').defaultTo('running'); // running, completed, failed, partial
|
||||
table.string('restore_type'); // database, files, full
|
||||
table.string('backup_id'); // Reference to the backup that was restored
|
||||
table.string('backup_app_version'); // Version of app that created the backup
|
||||
table.string('restore_app_version'); // Version of app performing the restore
|
||||
table.string('backup_node_version'); // Node version that created the backup
|
||||
table.string('restore_node_version'); // Node version performing the restore
|
||||
table.string('backup_schema_version'); // Schema version in the backup
|
||||
table.string('restore_schema_version'); // Current schema version
|
||||
table.json('version_compatibility'); // Compatibility check results
|
||||
table.json('restore_options'); // Options used during restore
|
||||
table.json('statistics'); // Restore statistics
|
||||
table.text('warnings'); // Any warnings during restore
|
||||
table.text('error_message'); // Error details if failed
|
||||
table.string('restored_by'); // User who initiated the restore
|
||||
table.index(['started_at'], 'idx_restore_started');
|
||||
table.index(['backup_id'], 'idx_restore_backup_id');
|
||||
});
|
||||
console.log('Created restore_history table');
|
||||
}
|
||||
|
||||
// Add version compatibility settings
|
||||
const versionSettings = [
|
||||
{
|
||||
setting_key: 'backup_require_version_match',
|
||||
setting_value: JSON.stringify(false), // If true, exact version match required for restore
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_allow_minor_version_mismatch',
|
||||
setting_value: JSON.stringify(true), // Allow restoring from same major version
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_warn_on_version_mismatch',
|
||||
setting_value: JSON.stringify(true), // Show warning when versions don't match
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_check_schema_compatibility',
|
||||
setting_value: JSON.stringify(true), // Check if migrations are compatible
|
||||
setting_type: 'backup'
|
||||
}
|
||||
];
|
||||
|
||||
// Insert version settings if they don't exist
|
||||
for (const setting of versionSettings) {
|
||||
const exists = await db('app_settings')
|
||||
.where('setting_key', setting.setting_key)
|
||||
.first();
|
||||
|
||||
if (!exists) {
|
||||
await db('app_settings').insert(setting);
|
||||
}
|
||||
}
|
||||
|
||||
console.log('Version tracking for backups added successfully');
|
||||
}
|
||||
|
||||
async function down() {
|
||||
// Remove version columns from database_backup_runs
|
||||
const hasDatabaseBackupRunsTable = await db.schema.hasTable('database_backup_runs');
|
||||
if (hasDatabaseBackupRunsTable) {
|
||||
await db.schema.alterTable('database_backup_runs', (table) => {
|
||||
table.dropColumn('app_version');
|
||||
table.dropColumn('node_version');
|
||||
table.dropColumn('db_schema_version');
|
||||
table.dropColumn('environment_info');
|
||||
});
|
||||
}
|
||||
|
||||
// Remove version columns from backup_runs
|
||||
const hasBackupRunsTable = await db.schema.hasTable('backup_runs');
|
||||
if (hasBackupRunsTable) {
|
||||
await db.schema.alterTable('backup_runs', (table) => {
|
||||
table.dropColumn('app_version');
|
||||
table.dropColumn('node_version');
|
||||
table.dropColumn('db_schema_version');
|
||||
table.dropColumn('manifest_info');
|
||||
});
|
||||
}
|
||||
|
||||
// Drop restore_history table
|
||||
await db.schema.dropTableIfExists('restore_history');
|
||||
|
||||
// Remove version settings
|
||||
await db('app_settings')
|
||||
.whereIn('setting_key', [
|
||||
'backup_require_version_match',
|
||||
'backup_allow_minor_version_mismatch',
|
||||
'backup_warn_on_version_mismatch',
|
||||
'backup_check_schema_compatibility'
|
||||
])
|
||||
.delete();
|
||||
}
|
||||
|
||||
module.exports = { up, down };
|
||||
@@ -0,0 +1,221 @@
|
||||
const { db } = require('../../src/database/db');
|
||||
|
||||
async function up() {
|
||||
console.log('Enhancing backup system...');
|
||||
|
||||
// Add new settings to app_settings table if they don't exist
|
||||
const backupSettings = [
|
||||
{
|
||||
setting_key: 'backup_s3_force_path_style',
|
||||
setting_value: JSON.stringify(false),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_s3_ssl_enabled',
|
||||
setting_value: JSON.stringify(true),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_s3_prefix',
|
||||
setting_value: JSON.stringify(''),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_incremental',
|
||||
setting_value: JSON.stringify(false),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_include_database',
|
||||
setting_value: JSON.stringify(true),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_encryption_enabled',
|
||||
setting_value: JSON.stringify(false),
|
||||
setting_type: 'backup'
|
||||
},
|
||||
{
|
||||
setting_key: 'backup_database_schedule',
|
||||
setting_value: JSON.stringify(''),
|
||||
setting_type: 'backup'
|
||||
}
|
||||
];
|
||||
|
||||
// Insert settings if they don't exist
|
||||
for (const setting of backupSettings) {
|
||||
const exists = await db('app_settings')
|
||||
.where('setting_key', setting.setting_key)
|
||||
.first();
|
||||
|
||||
if (!exists) {
|
||||
await db('app_settings').insert(setting);
|
||||
}
|
||||
}
|
||||
|
||||
// Check and add columns to backup_runs table
|
||||
const hasBackupRunsTable = await db.schema.hasTable('backup_runs');
|
||||
if (hasBackupRunsTable) {
|
||||
// Check for existing columns before adding
|
||||
const hasManifestPath = await db.schema.hasColumn('backup_runs', 'manifest_path');
|
||||
const hasManifestId = await db.schema.hasColumn('backup_runs', 'manifest_id');
|
||||
const hasManifestFormat = await db.schema.hasColumn('backup_runs', 'manifest_format');
|
||||
const hasParentBackupId = await db.schema.hasColumn('backup_runs', 'parent_backup_id');
|
||||
const hasBackupMode = await db.schema.hasColumn('backup_runs', 'backup_mode');
|
||||
|
||||
if (!hasManifestPath || !hasManifestId || !hasManifestFormat || !hasParentBackupId || !hasBackupMode) {
|
||||
await db.schema.alterTable('backup_runs', (table) => {
|
||||
if (!hasManifestPath) {
|
||||
table.string('manifest_path', 500).comment('Path to backup manifest file');
|
||||
}
|
||||
if (!hasManifestId) {
|
||||
table.uuid('manifest_id').comment('Unique identifier for the manifest');
|
||||
}
|
||||
if (!hasManifestFormat) {
|
||||
table.enum('manifest_format', ['json', 'yaml']).comment('Format of the manifest file');
|
||||
}
|
||||
if (!hasParentBackupId) {
|
||||
table.integer('parent_backup_id').unsigned().references('id').inTable('backup_runs').onDelete('SET NULL').comment('Parent backup for incremental backups');
|
||||
}
|
||||
if (!hasBackupMode) {
|
||||
table.enum('backup_mode', ['full', 'incremental', 'database']).defaultTo('full').comment('Type of backup performed');
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// Add indexes if they don't exist
|
||||
try {
|
||||
await db.raw('CREATE INDEX IF NOT EXISTS idx_backup_runs_mode_status ON backup_runs(backup_mode, status)');
|
||||
await db.raw('CREATE INDEX IF NOT EXISTS idx_backup_runs_parent ON backup_runs(parent_backup_id)');
|
||||
await db.raw('CREATE INDEX IF NOT EXISTS idx_backup_runs_created_mode ON backup_runs(created_at, backup_mode)');
|
||||
} catch (error) {
|
||||
console.log('Note: Some indexes may already exist, continuing...');
|
||||
}
|
||||
}
|
||||
|
||||
// Create backup_manifest table if it doesn't exist
|
||||
const hasManifestTable = await db.schema.hasTable('backup_manifest');
|
||||
if (!hasManifestTable) {
|
||||
await db.schema.createTable('backup_manifest', (table) => {
|
||||
table.increments('id').primary();
|
||||
table.integer('backup_run_id').unsigned().notNullable().references('id').inTable('backup_runs').onDelete('CASCADE');
|
||||
table.uuid('manifest_id').notNullable().unique().comment('Unique identifier matching backup_runs.manifest_id');
|
||||
table.string('version', 20).notNullable().defaultTo('1.0.0').comment('Manifest schema version');
|
||||
table.enum('format', ['json', 'yaml']).notNullable().defaultTo('json');
|
||||
|
||||
// Backup metadata
|
||||
table.timestamp('backup_start').notNullable();
|
||||
table.timestamp('backup_end').notNullable();
|
||||
table.bigInteger('total_size').unsigned().comment('Total size of backup in bytes');
|
||||
table.integer('file_count').unsigned().comment('Number of files in backup');
|
||||
table.integer('photo_count').unsigned().comment('Number of photos backed up');
|
||||
table.integer('event_count').unsigned().comment('Number of events backed up');
|
||||
|
||||
// Incremental backup metadata
|
||||
table.boolean('is_incremental').defaultTo(false);
|
||||
table.uuid('parent_manifest_id').comment('Parent manifest ID for incremental backups');
|
||||
table.timestamp('incremental_since').comment('Timestamp for incremental backup baseline');
|
||||
|
||||
// Content checksums
|
||||
table.string('checksum_algorithm', 50).defaultTo('sha256').comment('Algorithm used for checksums');
|
||||
table.text('manifest_checksum').comment('Checksum of the manifest file itself');
|
||||
|
||||
// Storage information
|
||||
table.string('storage_location', 500).comment('Primary storage location (local path or S3 URI)');
|
||||
table.string('storage_provider', 50).comment('Storage provider (local, s3, etc.)');
|
||||
|
||||
// Encryption metadata
|
||||
table.boolean('is_encrypted').defaultTo(false);
|
||||
table.string('encryption_algorithm', 100).comment('Encryption algorithm used');
|
||||
table.string('encryption_key_id', 255).comment('ID of encryption key used');
|
||||
|
||||
// Additional metadata as JSON
|
||||
table.json('metadata').comment('Additional metadata as JSON');
|
||||
|
||||
// Timestamps
|
||||
table.timestamps(true, true);
|
||||
|
||||
// Indexes
|
||||
table.index(['backup_run_id'], 'idx_manifest_backup_run');
|
||||
table.index(['manifest_id'], 'idx_manifest_uuid');
|
||||
table.index(['parent_manifest_id'], 'idx_manifest_parent');
|
||||
table.index(['backup_start', 'backup_end'], 'idx_manifest_time_range');
|
||||
table.index(['is_incremental', 'created_at'], 'idx_manifest_incremental_created');
|
||||
});
|
||||
}
|
||||
|
||||
// Add composite indexes for common query patterns
|
||||
try {
|
||||
await db.raw(`
|
||||
CREATE INDEX IF NOT EXISTS idx_backup_runs_recent_successful
|
||||
ON backup_runs(created_at DESC)
|
||||
WHERE status = 'completed' AND backup_mode = 'full';
|
||||
`);
|
||||
await db.raw(`
|
||||
CREATE INDEX IF NOT EXISTS idx_backup_runs_incremental_chain
|
||||
ON backup_runs(parent_backup_id, created_at)
|
||||
WHERE backup_mode = 'incremental';
|
||||
`);
|
||||
} catch (error) {
|
||||
console.log('Note: Some composite indexes may already exist, continuing...');
|
||||
}
|
||||
|
||||
console.log('Backup system enhancements completed');
|
||||
}
|
||||
|
||||
async function down() {
|
||||
// Drop indexes first
|
||||
try {
|
||||
await db.raw('DROP INDEX IF EXISTS idx_backup_runs_incremental_chain;');
|
||||
await db.raw('DROP INDEX IF EXISTS idx_backup_runs_recent_successful;');
|
||||
} catch (error) {
|
||||
// Ignore errors if indexes don't exist
|
||||
}
|
||||
|
||||
// Drop backup_manifest table
|
||||
await db.schema.dropTableIfExists('backup_manifest');
|
||||
|
||||
// Remove columns from backup_runs if they exist
|
||||
const hasBackupRunsTable = await db.schema.hasTable('backup_runs');
|
||||
if (hasBackupRunsTable) {
|
||||
const hasBackupMode = await db.schema.hasColumn('backup_runs', 'backup_mode');
|
||||
const hasParentBackupId = await db.schema.hasColumn('backup_runs', 'parent_backup_id');
|
||||
const hasManifestFormat = await db.schema.hasColumn('backup_runs', 'manifest_format');
|
||||
const hasManifestId = await db.schema.hasColumn('backup_runs', 'manifest_id');
|
||||
const hasManifestPath = await db.schema.hasColumn('backup_runs', 'manifest_path');
|
||||
|
||||
if (hasBackupMode || hasParentBackupId || hasManifestFormat || hasManifestId || hasManifestPath) {
|
||||
await db.schema.alterTable('backup_runs', (table) => {
|
||||
if (hasBackupMode) table.dropColumn('backup_mode');
|
||||
if (hasParentBackupId) table.dropColumn('parent_backup_id');
|
||||
if (hasManifestFormat) table.dropColumn('manifest_format');
|
||||
if (hasManifestId) table.dropColumn('manifest_id');
|
||||
if (hasManifestPath) table.dropColumn('manifest_path');
|
||||
});
|
||||
}
|
||||
|
||||
// Drop indexes
|
||||
try {
|
||||
await db.raw('DROP INDEX IF EXISTS idx_backup_runs_mode_status');
|
||||
await db.raw('DROP INDEX IF EXISTS idx_backup_runs_parent');
|
||||
await db.raw('DROP INDEX IF EXISTS idx_backup_runs_created_mode');
|
||||
} catch (error) {
|
||||
// Ignore errors if indexes don't exist
|
||||
}
|
||||
}
|
||||
|
||||
// Remove settings
|
||||
await db('app_settings')
|
||||
.whereIn('setting_key', [
|
||||
'backup_s3_force_path_style',
|
||||
'backup_s3_ssl_enabled',
|
||||
'backup_s3_prefix',
|
||||
'backup_incremental',
|
||||
'backup_include_database',
|
||||
'backup_encryption_enabled',
|
||||
'backup_database_schedule'
|
||||
])
|
||||
.delete();
|
||||
}
|
||||
|
||||
module.exports = { up, down };
|
||||
+2
-2
@@ -1,4 +1,4 @@
|
||||
const { db } = require('../src/database/db');
|
||||
const { db } = require('../../src/database/db');
|
||||
|
||||
async function up() {
|
||||
console.log('Adding photo categories and CMS tables...');
|
||||
@@ -40,7 +40,7 @@ async function up() {
|
||||
// Add language preference to app_settings for global default
|
||||
await db('app_settings').insert({
|
||||
setting_key: 'default_language',
|
||||
setting_value: 'en',
|
||||
setting_value: JSON.stringify('en'),
|
||||
setting_type: 'general',
|
||||
updated_at: new Date()
|
||||
});
|
||||
+1
-2
@@ -8,8 +8,7 @@ exports.up = async function(knex) {
|
||||
await knex('app_settings').insert({
|
||||
setting_key: 'general_default_welcome_message',
|
||||
setting_value: JSON.stringify('Thank you for using our photo sharing service! We hope you enjoy your photos.'),
|
||||
setting_type: 'general',
|
||||
updated_at: new Date()
|
||||
setting_type: 'general'
|
||||
});
|
||||
}
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
const { db } = require('../src/database/db');
|
||||
const { db } = require('../../src/database/db');
|
||||
|
||||
async function up() {
|
||||
// Check if host_name column already exists
|
||||
@@ -0,0 +1,33 @@
|
||||
/**
|
||||
* Fix email_queue table by ensuring it doesn't have updated_at column
|
||||
* This migration addresses the PostgreSQL error where queries are trying to update
|
||||
* a non-existent updated_at column
|
||||
*/
|
||||
|
||||
exports.up = async function(knex) {
|
||||
// First, check if the column exists
|
||||
const hasUpdatedAt = await knex.schema.hasColumn('email_queue', 'updated_at');
|
||||
|
||||
if (hasUpdatedAt) {
|
||||
console.log('Found updated_at column in email_queue table, removing it...');
|
||||
await knex.schema.table('email_queue', (table) => {
|
||||
table.dropColumn('updated_at');
|
||||
});
|
||||
}
|
||||
|
||||
// Also ensure the table has all required columns
|
||||
const hasCreatedAt = await knex.schema.hasColumn('email_queue', 'created_at');
|
||||
if (!hasCreatedAt) {
|
||||
console.log('Adding missing created_at column to email_queue table...');
|
||||
await knex.schema.table('email_queue', (table) => {
|
||||
table.datetime('created_at').defaultTo(knex.fn.now());
|
||||
});
|
||||
}
|
||||
|
||||
console.log('email_queue table schema fixed');
|
||||
};
|
||||
|
||||
exports.down = async function(knex) {
|
||||
// In the down migration, we don't add back updated_at since it shouldn't exist
|
||||
// This is intentionally left minimal
|
||||
};
|
||||
@@ -0,0 +1,234 @@
|
||||
exports.up = async function(knex) {
|
||||
// Update gallery_created template with proper German translation
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'gallery_created')
|
||||
.update({
|
||||
subject_de: 'Ihre Fotogalerie ist bereit!',
|
||||
body_html_de: `<h2>Galerie erfolgreich erstellt</h2>
|
||||
<p>Liebe(r) {{host_name}},</p>
|
||||
<p>Ihre Fotogalerie "{{event_name}}" wurde erfolgreich erstellt!</p>
|
||||
{{#if welcome_message}}
|
||||
<div style="background-color: #f3f4f6; padding: 20px; border-radius: 8px; margin: 20px 0;">
|
||||
<p style="margin: 0 0 10px 0; font-weight: 600; color: #374151;">Persönliche Nachricht:</p>
|
||||
<p style="margin: 0; color: #4b5563;">{{welcome_message}}</p>
|
||||
</div>
|
||||
{{/if}}
|
||||
<p><strong>Galerie-Details:</strong></p>
|
||||
<ul>
|
||||
<li>Veranstaltungsdatum: {{event_date}}</li>
|
||||
<li>Galerie-Link: <a href="{{gallery_link}}" style="color: #5C8762;">{{gallery_link}}</a></li>
|
||||
<li>Passwort: {{gallery_password}}</li>
|
||||
<li>Ablaufdatum: {{expiry_date}}</li>
|
||||
</ul>
|
||||
<p>Teilen Sie diesen Link und das Passwort mit Ihren Gästen, damit diese die Fotos ansehen und herunterladen können.</p>
|
||||
<p style="background-color: #FEF3C7; padding: 15px; border-radius: 5px; border-left: 4px solid #F59E0B;">
|
||||
<strong>Wichtig:</strong> Diese Galerie läuft am {{expiry_date}} ab. Nach diesem Datum werden die Fotos archiviert und sind nicht mehr zugänglich.
|
||||
</p>
|
||||
<a href="{{gallery_link}}" style="display: inline-block; padding: 12px 30px; background-color: #5C8762; color: white; text-decoration: none; border-radius: 5px; font-weight: 500; margin: 20px 0;">Galerie anzeigen</a>
|
||||
<p>Mit freundlichen Grüßen,<br>Ihr Foto-Sharing-Team</p>`,
|
||||
body_text_de: `Galerie erfolgreich erstellt
|
||||
|
||||
Liebe(r) {{host_name}},
|
||||
|
||||
Ihre Fotogalerie "{{event_name}}" wurde erfolgreich erstellt!
|
||||
|
||||
{{#if welcome_message}}
|
||||
Persönliche Nachricht:
|
||||
{{welcome_message}}
|
||||
|
||||
{{/if}}
|
||||
Galerie-Details:
|
||||
- Veranstaltungsdatum: {{event_date}}
|
||||
- Galerie-Link: {{gallery_link}}
|
||||
- Passwort: {{gallery_password}}
|
||||
- Ablaufdatum: {{expiry_date}}
|
||||
|
||||
Teilen Sie diesen Link und das Passwort mit Ihren Gästen, damit diese die Fotos ansehen und herunterladen können.
|
||||
|
||||
WICHTIG: Diese Galerie läuft am {{expiry_date}} ab. Nach diesem Datum werden die Fotos archiviert und sind nicht mehr zugänglich.
|
||||
|
||||
Mit freundlichen Grüßen,
|
||||
Ihr Foto-Sharing-Team`
|
||||
});
|
||||
|
||||
// Update expiration_warning template with proper German translation
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'expiration_warning')
|
||||
.update({
|
||||
subject_de: 'Ihre Fotogalerie läuft bald ab',
|
||||
body_html_de: `<h2>Galerie läuft bald ab</h2>
|
||||
<p>Liebe(r) {{host_name}},</p>
|
||||
<p>Ihre Fotogalerie "{{event_name}}" läuft in <strong>{{days_remaining}} Tagen</strong> ab.</p>
|
||||
<p>Nach Ablauf wird die Galerie archiviert und ist für Gäste nicht mehr zugänglich. Bitte stellen Sie sicher, dass alle gewünschten Fotos heruntergeladen wurden.</p>
|
||||
<p><strong>Ablaufdatum:</strong> {{expiry_date}}</p>
|
||||
<a href="{{gallery_link}}" style="display: inline-block; padding: 12px 30px; background-color: #5C8762; color: white; text-decoration: none; border-radius: 5px; font-weight: 500; margin: 20px 0;">Galerie jetzt besuchen</a>
|
||||
<p style="background-color: #FEE2E2; padding: 15px; border-radius: 5px; border-left: 4px solid #EF4444;">
|
||||
<strong>Erinnerung:</strong> Nach dem {{expiry_date}} können Ihre Gäste nicht mehr auf die Galerie zugreifen.
|
||||
</p>
|
||||
<p>Mit freundlichen Grüßen,<br>Ihr Foto-Sharing-Team</p>`,
|
||||
body_text_de: `Galerie läuft bald ab
|
||||
|
||||
Liebe(r) {{host_name}},
|
||||
|
||||
Ihre Fotogalerie "{{event_name}}" läuft in {{days_remaining}} Tagen ab.
|
||||
|
||||
Nach Ablauf wird die Galerie archiviert und ist für Gäste nicht mehr zugänglich. Bitte stellen Sie sicher, dass alle gewünschten Fotos heruntergeladen wurden.
|
||||
|
||||
Ablaufdatum: {{expiry_date}}
|
||||
|
||||
Galerie-Link: {{gallery_link}}
|
||||
|
||||
ERINNERUNG: Nach dem {{expiry_date}} können Ihre Gäste nicht mehr auf die Galerie zugreifen.
|
||||
|
||||
Mit freundlichen Grüßen,
|
||||
Ihr Foto-Sharing-Team`
|
||||
});
|
||||
|
||||
// Update gallery_expired template with proper German translation
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'gallery_expired')
|
||||
.update({
|
||||
subject_de: 'Ihre Fotogalerie {{event_name}} ist abgelaufen',
|
||||
body_html_de: `<h2>Galerie abgelaufen</h2>
|
||||
<p>Liebe(r) {{host_name}},</p>
|
||||
<p>Ihre Fotogalerie "{{event_name}}" ist am {{expiry_date}} abgelaufen und wurde archiviert.</p>
|
||||
<p>Die Galerie ist nicht mehr für Gäste zugänglich. Alle Fotos wurden sicher in unserem Archivsystem gespeichert.</p>
|
||||
<p>Wenn Sie wieder Zugriff auf die archivierten Fotos benötigen, wenden Sie sich bitte an unseren Support:</p>
|
||||
<p style="background-color: #F3F4F6; padding: 15px; border-radius: 5px;">
|
||||
<strong>Kontakt:</strong><br>
|
||||
E-Mail: <a href="mailto:{{admin_email}}" style="color: #5C8762;">{{admin_email}}</a><br>
|
||||
{{#if support_phone}}Telefon: {{support_phone}}{{/if}}
|
||||
</p>
|
||||
<p>Vielen Dank für die Nutzung unseres Foto-Sharing-Services!</p>
|
||||
<p>Mit freundlichen Grüßen,<br>Ihr Foto-Sharing-Team</p>`,
|
||||
body_text_de: `Galerie abgelaufen
|
||||
|
||||
Liebe(r) {{host_name}},
|
||||
|
||||
Ihre Fotogalerie "{{event_name}}" ist am {{expiry_date}} abgelaufen und wurde archiviert.
|
||||
|
||||
Die Galerie ist nicht mehr für Gäste zugänglich. Alle Fotos wurden sicher in unserem Archivsystem gespeichert.
|
||||
|
||||
Wenn Sie wieder Zugriff auf die archivierten Fotos benötigen, wenden Sie sich bitte an unseren Support:
|
||||
|
||||
E-Mail: {{admin_email}}
|
||||
{{#if support_phone}}Telefon: {{support_phone}}{{/if}}
|
||||
|
||||
Vielen Dank für die Nutzung unseres Foto-Sharing-Services!
|
||||
|
||||
Mit freundlichen Grüßen,
|
||||
Ihr Foto-Sharing-Team`
|
||||
});
|
||||
|
||||
// Update archive_complete template with proper German translation
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'archive_complete')
|
||||
.update({
|
||||
subject_de: 'Archivierung abgeschlossen: {{event_name}}',
|
||||
body_html_de: `<h2>Archivierung abgeschlossen</h2>
|
||||
<p>Liebe(r) {{host_name}},</p>
|
||||
<p>Die Fotogalerie "{{event_name}}" wurde erfolgreich archiviert.</p>
|
||||
<p><strong>Archiv-Details:</strong></p>
|
||||
<ul>
|
||||
<li>Archivgröße: {{archive_size}}</li>
|
||||
<li>Archivierungsdatum: {{archive_date}}</li>
|
||||
<li>Anzahl der Fotos: {{photo_count}}</li>
|
||||
</ul>
|
||||
<p>Das Archiv wird sicher in unserem System aufbewahrt. Bei Bedarf können Sie sich an unseren Support wenden, um Zugriff auf die archivierten Fotos zu erhalten.</p>
|
||||
<p style="background-color: #F0FDF4; padding: 15px; border-radius: 5px; border-left: 4px solid #22C55E;">
|
||||
<strong>✓ Erfolgreich archiviert:</strong> Ihre Fotos sind sicher gespeichert und können bei Bedarf wiederhergestellt werden.
|
||||
</p>
|
||||
<p>Kontakt für Archivzugriff:<br>
|
||||
E-Mail: <a href="mailto:{{admin_email}}" style="color: #5C8762;">{{admin_email}}</a></p>
|
||||
<p>Mit freundlichen Grüßen,<br>Ihr Foto-Sharing-Team</p>`,
|
||||
body_text_de: `Archivierung abgeschlossen
|
||||
|
||||
Liebe(r) {{host_name}},
|
||||
|
||||
Die Fotogalerie "{{event_name}}" wurde erfolgreich archiviert.
|
||||
|
||||
Archiv-Details:
|
||||
- Archivgröße: {{archive_size}}
|
||||
- Archivierungsdatum: {{archive_date}}
|
||||
- Anzahl der Fotos: {{photo_count}}
|
||||
|
||||
Das Archiv wird sicher in unserem System aufbewahrt. Bei Bedarf können Sie sich an unseren Support wenden, um Zugriff auf die archivierten Fotos zu erhalten.
|
||||
|
||||
✓ ERFOLGREICH ARCHIVIERT: Ihre Fotos sind sicher gespeichert und können bei Bedarf wiederhergestellt werden.
|
||||
|
||||
Kontakt für Archivzugriff:
|
||||
E-Mail: {{admin_email}}
|
||||
|
||||
Mit freundlichen Grüßen,
|
||||
Ihr Foto-Sharing-Team`
|
||||
});
|
||||
|
||||
// Also update the non-language-specific fields to match German for consistency
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'gallery_created')
|
||||
.update({
|
||||
subject: knex.raw('subject_de'),
|
||||
body_html: knex.raw('body_html_de'),
|
||||
body_text: knex.raw('body_text_de')
|
||||
});
|
||||
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'expiration_warning')
|
||||
.update({
|
||||
subject: knex.raw('subject_de'),
|
||||
body_html: knex.raw('body_html_de'),
|
||||
body_text: knex.raw('body_text_de')
|
||||
});
|
||||
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'gallery_expired')
|
||||
.update({
|
||||
subject: knex.raw('subject_de'),
|
||||
body_html: knex.raw('body_html_de'),
|
||||
body_text: knex.raw('body_text_de')
|
||||
});
|
||||
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'archive_complete')
|
||||
.update({
|
||||
subject: knex.raw('subject_de'),
|
||||
body_html: knex.raw('body_html_de'),
|
||||
body_text: knex.raw('body_text_de')
|
||||
});
|
||||
};
|
||||
|
||||
exports.down = async function(knex) {
|
||||
// Revert to previous German translations
|
||||
// This is a simplified rollback - in production you might want to store the old values
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'gallery_created')
|
||||
.update({
|
||||
subject: knex.raw('subject_en'),
|
||||
body_html: knex.raw('body_html_en'),
|
||||
body_text: knex.raw('body_text_en')
|
||||
});
|
||||
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'expiration_warning')
|
||||
.update({
|
||||
subject: knex.raw('subject_en'),
|
||||
body_html: knex.raw('body_html_en'),
|
||||
body_text: knex.raw('body_text_en')
|
||||
});
|
||||
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'gallery_expired')
|
||||
.update({
|
||||
subject: knex.raw('subject_en'),
|
||||
body_html: knex.raw('body_html_en'),
|
||||
body_text: knex.raw('body_text_en')
|
||||
});
|
||||
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'archive_complete')
|
||||
.update({
|
||||
subject: knex.raw('subject_en'),
|
||||
body_html: knex.raw('body_html_en'),
|
||||
body_text: knex.raw('body_text_en')
|
||||
});
|
||||
};
|
||||
@@ -0,0 +1,41 @@
|
||||
exports.up = async function(knex) {
|
||||
// Add language column to events table if it doesn't exist
|
||||
const hasLanguageInEvents = await knex.schema.hasColumn('events', 'language');
|
||||
if (!hasLanguageInEvents) {
|
||||
await knex.schema.alterTable('events', function(table) {
|
||||
table.string('language', 5).defaultTo('en');
|
||||
});
|
||||
}
|
||||
|
||||
// Add default_language to email_configs if it doesn't exist
|
||||
const hasDefaultLanguage = await knex.schema.hasColumn('email_configs', 'default_language');
|
||||
if (!hasDefaultLanguage) {
|
||||
await knex.schema.alterTable('email_configs', function(table) {
|
||||
table.string('default_language', 5).defaultTo('en');
|
||||
});
|
||||
}
|
||||
|
||||
// Set default language to German for the existing email config
|
||||
await knex('email_configs')
|
||||
.update({
|
||||
default_language: 'de'
|
||||
});
|
||||
};
|
||||
|
||||
exports.down = async function(knex) {
|
||||
// Remove language column from events table
|
||||
const hasLanguageInEvents = await knex.schema.hasColumn('events', 'language');
|
||||
if (hasLanguageInEvents) {
|
||||
await knex.schema.alterTable('events', function(table) {
|
||||
table.dropColumn('language');
|
||||
});
|
||||
}
|
||||
|
||||
// Remove default_language from email_configs
|
||||
const hasDefaultLanguage = await knex.schema.hasColumn('email_configs', 'default_language');
|
||||
if (hasDefaultLanguage) {
|
||||
await knex.schema.alterTable('email_configs', function(table) {
|
||||
table.dropColumn('default_language');
|
||||
});
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,62 @@
|
||||
exports.up = async function(knex) {
|
||||
// Add rate limit settings to app_settings
|
||||
const rateLimitSettings = [
|
||||
{
|
||||
setting_key: 'rate_limit_enabled',
|
||||
setting_value: JSON.stringify(true),
|
||||
setting_type: 'security'
|
||||
},
|
||||
{
|
||||
setting_key: 'rate_limit_window_minutes',
|
||||
setting_value: JSON.stringify(15),
|
||||
setting_type: 'security'
|
||||
},
|
||||
{
|
||||
setting_key: 'rate_limit_max_requests',
|
||||
setting_value: JSON.stringify(1000),
|
||||
setting_type: 'security'
|
||||
},
|
||||
{
|
||||
setting_key: 'rate_limit_auth_max_requests',
|
||||
setting_value: JSON.stringify(5),
|
||||
setting_type: 'security'
|
||||
},
|
||||
{
|
||||
setting_key: 'rate_limit_skip_authenticated',
|
||||
setting_value: JSON.stringify(true),
|
||||
setting_type: 'security'
|
||||
},
|
||||
{
|
||||
setting_key: 'rate_limit_public_endpoints_only',
|
||||
setting_value: JSON.stringify(false),
|
||||
setting_type: 'security'
|
||||
}
|
||||
];
|
||||
|
||||
// Insert settings if they don't exist
|
||||
for (const setting of rateLimitSettings) {
|
||||
const exists = await knex('app_settings')
|
||||
.where('setting_key', setting.setting_key)
|
||||
.first();
|
||||
|
||||
if (!exists) {
|
||||
await knex('app_settings').insert({
|
||||
...setting
|
||||
});
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
exports.down = async function(knex) {
|
||||
// Remove rate limit settings
|
||||
await knex('app_settings')
|
||||
.whereIn('setting_key', [
|
||||
'rate_limit_enabled',
|
||||
'rate_limit_window_minutes',
|
||||
'rate_limit_max_requests',
|
||||
'rate_limit_auth_max_requests',
|
||||
'rate_limit_skip_authenticated',
|
||||
'rate_limit_public_endpoints_only'
|
||||
])
|
||||
.del();
|
||||
};
|
||||
@@ -0,0 +1,307 @@
|
||||
exports.up = async function(knex) {
|
||||
// Update English templates to match the quality and content of German templates
|
||||
|
||||
// 1. Gallery Created - Match German version with proper styling and conditionals
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'gallery_created')
|
||||
.update({
|
||||
subject_en: 'Your photo gallery is ready',
|
||||
body_html_en: `
|
||||
<h2>Hello {{host_name}},</h2>
|
||||
|
||||
<p>Your photo gallery <strong>{{event_name}}</strong> for {{event_date}} has been successfully created and is now online!</p>
|
||||
|
||||
{{#if welcome_message}}
|
||||
<div style="background-color: #f0f8ff; border-left: 4px solid #5C8762; padding: 15px; margin: 20px 0; border-radius: 4px;">
|
||||
<p style="margin: 0;"><strong>Personal message from your photographer:</strong></p>
|
||||
<p style="margin: 10px 0 0 0;">{{welcome_message}}</p>
|
||||
</div>
|
||||
{{/if}}
|
||||
|
||||
<div style="background-color: #f9f9f9; padding: 20px; border-radius: 8px; margin: 20px 0;">
|
||||
<h3 style="margin-top: 0;">Your access data:</h3>
|
||||
<ul style="list-style: none; padding: 0;">
|
||||
<li style="margin-bottom: 10px;"><strong>Gallery link:</strong> <a href="{{gallery_link}}" style="color: #5C8762;">{{gallery_link}}</a></li>
|
||||
<li style="margin-bottom: 10px;"><strong>Password:</strong> {{gallery_password}}</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
<div style="text-align: center; margin: 30px 0;">
|
||||
<a href="{{gallery_link}}" style="display: inline-block; padding: 12px 30px; background-color: #5C8762; color: white; text-decoration: none; border-radius: 5px; font-weight: 500;">View Gallery</a>
|
||||
</div>
|
||||
|
||||
<div style="background-color: #fff3cd; border: 1px solid #ffeaa7; color: #856404; padding: 15px; border-radius: 4px; margin: 20px 0;">
|
||||
<p style="margin: 0;"><strong>Important:</strong> Your gallery will be available until <strong>{{expiry_date}}</strong>. After this date, the photos will be archived and will only be available upon request.</p>
|
||||
</div>
|
||||
|
||||
<p>We hope you enjoy your photos!</p>
|
||||
|
||||
<p>Best regards,<br>
|
||||
Your Photo Sharing Team</p>`,
|
||||
body_text_en: `Hello {{host_name}},
|
||||
|
||||
Your photo gallery "{{event_name}}" for {{event_date}} has been successfully created and is now online!
|
||||
|
||||
{{#if welcome_message}}
|
||||
Personal message from your photographer:
|
||||
{{welcome_message}}
|
||||
{{/if}}
|
||||
|
||||
Your access data:
|
||||
- Gallery link: {{gallery_link}}
|
||||
- Password: {{gallery_password}}
|
||||
|
||||
Important: Your gallery will be available until {{expiry_date}}. After this date, the photos will be archived and will only be available upon request.
|
||||
|
||||
We hope you enjoy your photos!
|
||||
|
||||
Best regards,
|
||||
Your Photo Sharing Team`
|
||||
});
|
||||
|
||||
// 2. Expiration Warning - Match German version with urgency and styling
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'expiration_warning')
|
||||
.update({
|
||||
subject_en: 'Your photo gallery expires soon',
|
||||
body_html_en: `
|
||||
<h2>Hello {{host_name}},</h2>
|
||||
|
||||
<p>Your photo gallery <strong>{{event_name}}</strong> will expire in <strong style="color: #e74c3c; font-size: 18px;">{{days_remaining}} days</strong>!</p>
|
||||
|
||||
<div style="background-color: #fee; border: 1px solid #fcc; color: #c33; padding: 20px; border-radius: 8px; margin: 20px 0;">
|
||||
<p style="margin: 0; font-weight: bold; font-size: 16px;">⚠️ Important Notice</p>
|
||||
<p style="margin: 10px 0 0 0;">After {{expiry_date}}, your gallery will no longer be accessible online. The photos will be archived and will only be available upon special request.</p>
|
||||
</div>
|
||||
|
||||
<p><strong>Don't miss out – download your photos now!</strong></p>
|
||||
|
||||
<div style="text-align: center; margin: 30px 0;">
|
||||
<a href="{{gallery_link}}" style="display: inline-block; padding: 14px 35px; background-color: #e74c3c; color: white; text-decoration: none; border-radius: 5px; font-weight: 600; font-size: 16px;">Visit Gallery Now</a>
|
||||
</div>
|
||||
|
||||
<div style="background-color: #f9f9f9; padding: 15px; border-radius: 8px; margin: 20px 0;">
|
||||
<p style="margin: 0;"><strong>Quick reminder of your access data:</strong></p>
|
||||
<ul style="list-style: none; padding: 0; margin: 10px 0 0 0;">
|
||||
<li>Gallery link: <a href="{{gallery_link}}" style="color: #5C8762;">{{gallery_link}}</a></li>
|
||||
<li>Password: {{gallery_password}}</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
<p>If you have any questions, please don't hesitate to contact us.</p>
|
||||
|
||||
<p>Best regards,<br>
|
||||
Your Photo Sharing Team</p>`,
|
||||
body_text_en: `Hello {{host_name}},
|
||||
|
||||
Your photo gallery "{{event_name}}" will expire in {{days_remaining}} days!
|
||||
|
||||
⚠️ Important Notice
|
||||
After {{expiry_date}}, your gallery will no longer be accessible online. The photos will be archived and will only be available upon special request.
|
||||
|
||||
Don't miss out – download your photos now!
|
||||
|
||||
Quick reminder of your access data:
|
||||
- Gallery link: {{gallery_link}}
|
||||
- Password: {{gallery_password}}
|
||||
|
||||
If you have any questions, please don't hesitate to contact us.
|
||||
|
||||
Best regards,
|
||||
Your Photo Sharing Team`
|
||||
});
|
||||
|
||||
// 3. Gallery Expired - Match German version with contact information
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'gallery_expired')
|
||||
.update({
|
||||
subject_en: 'Your photo gallery has expired',
|
||||
body_html_en: `
|
||||
<h2>Hello {{host_name}},</h2>
|
||||
|
||||
<p>Your photo gallery <strong>{{event_name}}</strong> expired on {{expiry_date}} and is no longer accessible online.</p>
|
||||
|
||||
<div style="background-color: #f9f9f9; border-left: 4px solid #5C8762; padding: 20px; margin: 20px 0; border-radius: 4px;">
|
||||
<h3 style="margin-top: 0;">Your photos are safely archived</h3>
|
||||
<p>Don't worry – your photos have been securely archived and are not lost. If you need access to your photos, please contact us:</p>
|
||||
<ul style="list-style: none; padding: 0;">
|
||||
<li style="margin-bottom: 8px;">📧 Email: <a href="mailto:{{support_email}}" style="color: #5C8762;">{{support_email}}</a></li>
|
||||
{{#if support_phone}}
|
||||
<li>📞 Phone: {{support_phone}}</li>
|
||||
{{/if}}
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
<p>Please have the following information ready when contacting us:</p>
|
||||
<ul>
|
||||
<li>Event name: {{event_name}}</li>
|
||||
<li>Event date: {{event_date}}</li>
|
||||
<li>Expiry date: {{expiry_date}}</li>
|
||||
</ul>
|
||||
|
||||
<p>We'll be happy to help you access your archived photos.</p>
|
||||
|
||||
<p>Best regards,<br>
|
||||
Your Photo Sharing Team</p>`,
|
||||
body_text_en: `Hello {{host_name}},
|
||||
|
||||
Your photo gallery "{{event_name}}" expired on {{expiry_date}} and is no longer accessible online.
|
||||
|
||||
Your photos are safely archived
|
||||
Don't worry – your photos have been securely archived and are not lost. If you need access to your photos, please contact us:
|
||||
|
||||
📧 Email: {{support_email}}
|
||||
{{#if support_phone}}📞 Phone: {{support_phone}}{{/if}}
|
||||
|
||||
Please have the following information ready when contacting us:
|
||||
- Event name: {{event_name}}
|
||||
- Event date: {{event_date}}
|
||||
- Expiry date: {{expiry_date}}
|
||||
|
||||
We'll be happy to help you access your archived photos.
|
||||
|
||||
Best regards,
|
||||
Your Photo Sharing Team`
|
||||
});
|
||||
|
||||
// 4. Archive Complete - Match German version with success message and details
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'archive_complete')
|
||||
.update({
|
||||
subject_en: 'Your photo gallery has been successfully archived',
|
||||
body_html_en: `
|
||||
<h2>Hello {{host_name}},</h2>
|
||||
|
||||
<p>Your photo gallery <strong>{{event_name}}</strong> has been successfully archived.</p>
|
||||
|
||||
<div style="background-color: #d4edda; border: 1px solid #c3e6cb; color: #155724; padding: 20px; border-radius: 8px; margin: 20px 0;">
|
||||
<p style="margin: 0; font-weight: bold;">✅ Archive successfully created</p>
|
||||
<p style="margin: 10px 0 0 0;">Your photos are now safely stored in our archive.</p>
|
||||
</div>
|
||||
|
||||
<div style="background-color: #f9f9f9; padding: 20px; border-radius: 8px; margin: 20px 0;">
|
||||
<h3 style="margin-top: 0;">Archive details:</h3>
|
||||
<ul style="list-style: none; padding: 0;">
|
||||
<li style="margin-bottom: 8px;"><strong>Event:</strong> {{event_name}}</li>
|
||||
<li style="margin-bottom: 8px;"><strong>Archive date:</strong> {{archive_date}}</li>
|
||||
<li style="margin-bottom: 8px;"><strong>Number of photos:</strong> {{photo_count}}</li>
|
||||
<li><strong>Archive size:</strong> {{archive_size}}</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
<p>If you need access to your archived photos in the future, please contact us at:</p>
|
||||
<p style="margin-left: 20px;">
|
||||
📧 <a href="mailto:{{support_email}}" style="color: #5C8762;">{{support_email}}</a><br>
|
||||
{{#if support_phone}}📞 {{support_phone}}{{/if}}
|
||||
</p>
|
||||
|
||||
<p>Thank you for using our photo sharing service!</p>
|
||||
|
||||
<p>Best regards,<br>
|
||||
Your Photo Sharing Team</p>`,
|
||||
body_text_en: `Hello {{host_name}},
|
||||
|
||||
Your photo gallery "{{event_name}}" has been successfully archived.
|
||||
|
||||
✅ Archive successfully created
|
||||
Your photos are now safely stored in our archive.
|
||||
|
||||
Archive details:
|
||||
- Event: {{event_name}}
|
||||
- Archive date: {{archive_date}}
|
||||
- Number of photos: {{photo_count}}
|
||||
- Archive size: {{archive_size}}
|
||||
|
||||
If you need access to your archived photos in the future, please contact us at:
|
||||
📧 {{support_email}}
|
||||
{{#if support_phone}}📞 {{support_phone}}{{/if}}
|
||||
|
||||
Thank you for using our photo sharing service!
|
||||
|
||||
Best regards,
|
||||
Your Photo Sharing Team`
|
||||
});
|
||||
|
||||
// 5. Test Email - Update to match German style
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'test_email')
|
||||
.update({
|
||||
subject_en: 'Test Email - Photo Sharing Platform',
|
||||
body_html_en: `
|
||||
<h2>Test Email</h2>
|
||||
|
||||
<p>This is a test email from your photo sharing platform.</p>
|
||||
|
||||
<div style="background-color: #d4edda; border: 1px solid #c3e6cb; color: #155724; padding: 15px; border-radius: 4px; margin: 20px 0;">
|
||||
<p style="margin: 0;"><strong>✅ Email configuration successful!</strong></p>
|
||||
<p style="margin: 10px 0 0 0;">Your email settings have been configured correctly and emails can be sent.</p>
|
||||
</div>
|
||||
|
||||
<div style="background-color: #f9f9f9; padding: 15px; border-radius: 8px; margin: 20px 0;">
|
||||
<p style="margin: 0;"><strong>Configuration details:</strong></p>
|
||||
<ul style="margin: 10px 0 0 0;">
|
||||
<li>Timestamp: {{timestamp}}</li>
|
||||
<li>Sender: {{from_email}}</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
<p>Best regards,<br>
|
||||
Your Photo Sharing Team</p>`,
|
||||
body_text_en: `Test Email
|
||||
|
||||
This is a test email from your photo sharing platform.
|
||||
|
||||
✅ Email configuration successful!
|
||||
Your email settings have been configured correctly and emails can be sent.
|
||||
|
||||
Configuration details:
|
||||
- Timestamp: {{timestamp}}
|
||||
- Sender: {{from_email}}
|
||||
|
||||
Best regards,
|
||||
Your Photo Sharing Team`
|
||||
});
|
||||
};
|
||||
|
||||
exports.down = async function(knex) {
|
||||
// Revert to previous simpler English templates
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'gallery_created')
|
||||
.update({
|
||||
subject_en: 'Your Photo Gallery is Ready',
|
||||
body_html_en: '<h2>Hello,</h2><p>Your photo gallery "{{event_name}}" has been created.</p><p><strong>Access Link:</strong> <a href="{{gallery_link}}">{{gallery_link}}</a></p><p><strong>Password:</strong> {{gallery_password}}</p><p>The gallery will be available until {{expiry_date}}.</p>',
|
||||
body_text_en: 'Your photo gallery "{{event_name}}" has been created. Access Link: {{gallery_link}} Password: {{gallery_password}} The gallery will be available until {{expiry_date}}.'
|
||||
});
|
||||
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'expiration_warning')
|
||||
.update({
|
||||
subject_en: 'Gallery Expires in {{days_remaining}} Days',
|
||||
body_html_en: '<h2>Reminder</h2><p>Your photo gallery "{{event_name}}" will expire in {{days_remaining}} days.</p><p>Please download your photos before {{expiry_date}}.</p><p><a href="{{gallery_link}}">Access Gallery</a></p>',
|
||||
body_text_en: 'Your photo gallery "{{event_name}}" will expire in {{days_remaining}} days. Please download your photos before {{expiry_date}}. Access Gallery: {{gallery_link}}'
|
||||
});
|
||||
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'gallery_expired')
|
||||
.update({
|
||||
subject_en: 'Gallery Expired',
|
||||
body_html_en: '<h2>Gallery Expired</h2><p>Your photo gallery "{{event_name}}" has expired and is no longer accessible.</p><p>If you need access to your photos, please contact support.</p>',
|
||||
body_text_en: 'Your photo gallery "{{event_name}}" has expired and is no longer accessible. If you need access to your photos, please contact support.'
|
||||
});
|
||||
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'archive_complete')
|
||||
.update({
|
||||
subject_en: 'Gallery Archived',
|
||||
body_html_en: '<h2>Archive Complete</h2><p>Your gallery "{{event_name}}" has been archived.</p><p>Archive size: {{archive_size}}</p>',
|
||||
body_text_en: 'Your gallery "{{event_name}}" has been archived. Archive size: {{archive_size}}'
|
||||
});
|
||||
|
||||
await knex('email_templates')
|
||||
.where('template_key', 'test_email')
|
||||
.update({
|
||||
subject_en: 'Test Email',
|
||||
body_html_en: '<p>This is a test email sent at {{timestamp}}.</p>',
|
||||
body_text_en: 'This is a test email sent at {{timestamp}}.'
|
||||
});
|
||||
};
|
||||
@@ -27,8 +27,12 @@ async function isMigrationApplied(filename) {
|
||||
|
||||
// Mark migration as applied without running it (for existing schema)
|
||||
async function markMigrationAsApplied(filename) {
|
||||
await db('migrations').insert({ filename });
|
||||
console.log(`Marked migration ${filename} as applied`);
|
||||
// Check if already marked to avoid duplicate key error
|
||||
const isApplied = await isMigrationApplied(filename);
|
||||
if (!isApplied) {
|
||||
await db('migrations').insert({ filename });
|
||||
console.log(`Marked migration ${filename} as applied`);
|
||||
}
|
||||
}
|
||||
|
||||
// Detect existing schema and mark migrations as applied
|
||||
@@ -36,12 +40,14 @@ async function detectExistingSchema() {
|
||||
console.log('Detecting existing schema...');
|
||||
|
||||
const tableChecks = [
|
||||
{ table: 'events', migration: 'init.js' },
|
||||
{ table: 'photos', migration: 'init.js' },
|
||||
{ table: 'events', migration: '001_init.js' },
|
||||
{ table: 'photos', migration: '001_init.js' },
|
||||
{ table: 'photo_categories', migration: '004_add_categories_and_cms.js' },
|
||||
{ table: 'cms_pages', migration: '004_add_categories_and_cms.js' },
|
||||
{ table: 'login_attempts', migration: '015_add_login_attempts_table.js' },
|
||||
{ table: 'token_blacklist', migration: '017_add_token_revocation_tables.js' },
|
||||
{ table: 'backup_runs', migration: '029_add_backup_service_tables.js' },
|
||||
{ table: 'gallery_feedback', migration: '033_add_gallery_feedback.js' },
|
||||
];
|
||||
|
||||
for (const check of tableChecks) {
|
||||
@@ -56,13 +62,14 @@ async function detectExistingSchema() {
|
||||
}
|
||||
|
||||
// Run a single migration safely
|
||||
async function runMigrationSafely(filename) {
|
||||
async function runMigrationSafely(filepath) {
|
||||
try {
|
||||
const migrationPath = path.join(__dirname, filename);
|
||||
const migrationPath = path.join(__dirname, filepath);
|
||||
const migration = require(migrationPath);
|
||||
const filename = path.basename(filepath);
|
||||
|
||||
if (migration.up) {
|
||||
console.log(`Running migration: ${filename}`);
|
||||
console.log(`Running migration: ${filepath}`);
|
||||
|
||||
// Run migration in a transaction if possible
|
||||
if (db.client.config.client === 'pg') {
|
||||
@@ -74,14 +81,14 @@ async function runMigrationSafely(filename) {
|
||||
}
|
||||
|
||||
await db('migrations').insert({ filename });
|
||||
console.log(`Migration ${filename} completed successfully`);
|
||||
console.log(`Migration ${filepath} completed successfully`);
|
||||
}
|
||||
} catch (error) {
|
||||
// Check if error is because schema already exists
|
||||
if (error.code === '42P07' || // PostgreSQL: relation already exists
|
||||
error.code === 'SQLITE_ERROR' && error.message.includes('already exists')) {
|
||||
console.log(`Migration ${filename} - schema already exists, marking as applied`);
|
||||
await markMigrationAsApplied(filename);
|
||||
console.log(`Migration ${filepath} - schema already exists, marking as applied`);
|
||||
await markMigrationAsApplied(path.basename(filepath));
|
||||
} else {
|
||||
throw error;
|
||||
}
|
||||
@@ -101,26 +108,79 @@ async function runMigrations() {
|
||||
// Create migrations tracking table
|
||||
await ensureMigrationsTable();
|
||||
|
||||
// Detect and mark existing schema
|
||||
await detectExistingSchema();
|
||||
// Check if essential tables exist to determine if this is truly a new deployment
|
||||
const hasEventsTable = await db.schema.hasTable('events');
|
||||
const hasPhotosTable = await db.schema.hasTable('photos');
|
||||
const hasAdminTable = await db.schema.hasTable('admin_users');
|
||||
const hasActivityLogsTable = await db.schema.hasTable('activity_logs');
|
||||
|
||||
// Get all migration files
|
||||
const files = await fs.readdir(__dirname);
|
||||
const migrationFiles = files
|
||||
.filter(f => f.match(/^\d{3}_.*\.js$/) || f === 'init.js')
|
||||
.sort((a, b) => {
|
||||
// Ensure init.js runs first
|
||||
if (a === 'init.js') return -1;
|
||||
if (b === 'init.js') return 1;
|
||||
return a.localeCompare(b);
|
||||
});
|
||||
// Get applied migrations
|
||||
const appliedMigrations = await db('migrations').select('filename');
|
||||
const appliedFilenames = appliedMigrations.map(m => m.filename);
|
||||
|
||||
// Check if this is a new deployment
|
||||
// It's new if no essential tables exist OR no migrations have been applied
|
||||
const isNewDeployment = (!hasEventsTable || !hasPhotosTable || !hasAdminTable || !hasActivityLogsTable) || appliedFilenames.length === 0;
|
||||
|
||||
// Only detect existing schema for truly existing deployments
|
||||
if (!isNewDeployment) {
|
||||
await detectExistingSchema();
|
||||
}
|
||||
|
||||
// Get migration files from appropriate directories
|
||||
let migrationFiles = [];
|
||||
|
||||
if (isNewDeployment) {
|
||||
// For new deployments, only run core migrations
|
||||
console.log('New deployment detected - running core migrations only');
|
||||
const coreDir = path.join(__dirname, 'core');
|
||||
const coreFiles = await fs.readdir(coreDir);
|
||||
migrationFiles = coreFiles
|
||||
.filter(f => f.match(/^\d{3}_.*\.js$/))
|
||||
.map(f => path.join('core', f))
|
||||
.sort((a, b) => {
|
||||
const baseA = path.basename(a);
|
||||
const baseB = path.basename(b);
|
||||
const numA = parseInt(baseA.split('_')[0]);
|
||||
const numB = parseInt(baseB.split('_')[0]);
|
||||
return numA - numB;
|
||||
});
|
||||
} else {
|
||||
// For existing deployments, run all migrations (legacy + core)
|
||||
console.log('Existing deployment detected - checking all migrations');
|
||||
|
||||
// Get legacy migrations
|
||||
const legacyDir = path.join(__dirname, 'legacy');
|
||||
const legacyFiles = await fs.readdir(legacyDir);
|
||||
const legacyMigrations = legacyFiles
|
||||
.filter(f => f.match(/^\d{3}_.*\.js$/))
|
||||
.map(f => path.join('legacy', f));
|
||||
|
||||
// Get core migrations
|
||||
const coreDir = path.join(__dirname, 'core');
|
||||
const coreFiles = await fs.readdir(coreDir);
|
||||
const coreMigrations = coreFiles
|
||||
.filter(f => f.match(/^\d{3}_.*\.js$/))
|
||||
.map(f => path.join('core', f));
|
||||
|
||||
// Combine and sort by number
|
||||
migrationFiles = [...legacyMigrations, ...coreMigrations]
|
||||
.sort((a, b) => {
|
||||
const baseA = path.basename(a);
|
||||
const baseB = path.basename(b);
|
||||
const numA = parseInt(baseA.split('_')[0]);
|
||||
const numB = parseInt(baseB.split('_')[0]);
|
||||
return numA - numB;
|
||||
});
|
||||
}
|
||||
|
||||
// Run pending migrations
|
||||
let pendingCount = 0;
|
||||
let skippedCount = 0;
|
||||
|
||||
for (const file of migrationFiles) {
|
||||
const isApplied = await isMigrationApplied(file);
|
||||
const filename = path.basename(file);
|
||||
const isApplied = appliedFilenames.includes(filename);
|
||||
if (!isApplied) {
|
||||
await runMigrationSafely(file);
|
||||
pendingCount++;
|
||||
|
||||
@@ -22,15 +22,16 @@ async function getAppliedMigrations() {
|
||||
}
|
||||
|
||||
// Run a single migration
|
||||
async function runMigration(filename) {
|
||||
const migrationPath = path.join(__dirname, filename);
|
||||
async function runMigration(filepath) {
|
||||
const migrationPath = path.join(__dirname, filepath);
|
||||
const migration = require(migrationPath);
|
||||
const filename = path.basename(filepath);
|
||||
|
||||
if (migration.up) {
|
||||
console.log(`Running migration: ${filename}`);
|
||||
console.log(`Running migration: ${filepath}`);
|
||||
await migration.up(db);
|
||||
await db('migrations').insert({ filename });
|
||||
console.log(`Migration ${filename} completed`);
|
||||
console.log(`Migration ${filepath} completed`);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -50,19 +51,62 @@ async function runMigrations() {
|
||||
// Create migrations table
|
||||
await createMigrationsTable();
|
||||
|
||||
// Get all migration files
|
||||
const files = await fs.readdir(__dirname);
|
||||
const migrationFiles = files
|
||||
.filter(f => f.match(/^\d{3}_.*\.js$/))
|
||||
.sort();
|
||||
|
||||
// Get applied migrations
|
||||
const appliedMigrations = await getAppliedMigrations();
|
||||
|
||||
// Check if this is a new deployment (no migrations have been applied)
|
||||
const isNewDeployment = appliedMigrations.length === 0;
|
||||
|
||||
// Get migration files from appropriate directories
|
||||
let migrationFiles = [];
|
||||
|
||||
if (isNewDeployment) {
|
||||
// For new deployments, only run core migrations
|
||||
console.log('New deployment detected - running core migrations only');
|
||||
const coreDir = path.join(__dirname, 'core');
|
||||
const coreFiles = await fs.readdir(coreDir);
|
||||
migrationFiles = coreFiles
|
||||
.filter(f => f.match(/^\d{3}_.*\.js$/))
|
||||
.map(f => path.join('core', f))
|
||||
.sort((a, b) => {
|
||||
const baseA = path.basename(a);
|
||||
const baseB = path.basename(b);
|
||||
const numA = parseInt(baseA.split('_')[0]);
|
||||
const numB = parseInt(baseB.split('_')[0]);
|
||||
return numA - numB;
|
||||
});
|
||||
} else {
|
||||
// For existing deployments, run all migrations (legacy + core)
|
||||
console.log('Existing deployment detected - checking all migrations');
|
||||
|
||||
// Get legacy migrations
|
||||
const legacyDir = path.join(__dirname, 'legacy');
|
||||
const legacyFiles = await fs.readdir(legacyDir);
|
||||
const legacyMigrations = legacyFiles
|
||||
.filter(f => f.match(/^\d{3}_.*\.js$/))
|
||||
.map(f => path.join('legacy', f));
|
||||
|
||||
// Get core migrations
|
||||
const coreDir = path.join(__dirname, 'core');
|
||||
const coreFiles = await fs.readdir(coreDir);
|
||||
const coreMigrations = coreFiles
|
||||
.filter(f => f.match(/^\d{3}_.*\.js$/))
|
||||
.map(f => path.join('core', f));
|
||||
|
||||
// Combine and sort by number
|
||||
migrationFiles = [...legacyMigrations, ...coreMigrations]
|
||||
.sort((a, b) => {
|
||||
const numA = parseInt(path.basename(a).split('_')[0]);
|
||||
const numB = parseInt(path.basename(b).split('_')[0]);
|
||||
return numA - numB;
|
||||
});
|
||||
}
|
||||
|
||||
// Run pending migrations
|
||||
let pendingCount = 0;
|
||||
for (const file of migrationFiles) {
|
||||
if (!appliedMigrations.includes(file)) {
|
||||
const filename = path.basename(file);
|
||||
if (!appliedMigrations.includes(filename)) {
|
||||
await runMigration(file);
|
||||
pendingCount++;
|
||||
}
|
||||
|
||||
Generated
+2448
-308
File diff suppressed because it is too large
Load Diff
+18
-9
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "picpeak-backend",
|
||||
"version": "1.0.41",
|
||||
"version": "1.0.99",
|
||||
"description": "Backend for PicPeak event photo sharing platform",
|
||||
"main": "server.js",
|
||||
"scripts": {
|
||||
@@ -8,34 +8,43 @@
|
||||
"dev": "nodemon server.js",
|
||||
"migrate": "node migrations/run-migrations.js",
|
||||
"migrate:safe": "node migrations/run-migrations-safe.js",
|
||||
"fix-temp-photos": "node scripts/fix-temp-photos.js",
|
||||
"test": "jest",
|
||||
"lint": "eslint src/"
|
||||
"lint": "eslint src/",
|
||||
"test-backup": "node scripts/test-backup-service.js",
|
||||
"test-restore": "node scripts/test-restore-service.js"
|
||||
},
|
||||
"dependencies": {
|
||||
"@aws-sdk/client-s3": "^3.850.0",
|
||||
"@aws-sdk/lib-storage": "^3.850.0",
|
||||
"@aws-sdk/s3-request-presigner": "^3.850.0",
|
||||
"adm-zip": "^0.5.16",
|
||||
"archiver": "^5.3.1",
|
||||
"axios": "^1.10.0",
|
||||
"bcrypt": "^5.1.0",
|
||||
"chokidar": "^3.5.3",
|
||||
"bcrypt": "6.0.0",
|
||||
"chokidar": "4.0.3",
|
||||
"cors": "^2.8.5",
|
||||
"dotenv": "^16.0.3",
|
||||
"express": "^4.18.2",
|
||||
"express-rate-limit": "^6.7.0",
|
||||
"express-validator": "^7.0.1",
|
||||
"form-data": "^4.0.3",
|
||||
"form-data": "^4.0.4",
|
||||
"handlebars": "^4.7.8",
|
||||
"helmet": "^7.0.0",
|
||||
"i18next": "^25.3.1",
|
||||
"i18next": "25.3.2",
|
||||
"i18next-browser-languagedetector": "^8.2.0",
|
||||
"i18next-http-backend": "^3.0.2",
|
||||
"joi": "^17.9.1",
|
||||
"js-yaml": "^4.1.0",
|
||||
"jsonwebtoken": "^9.0.0",
|
||||
"knex": "^2.4.2",
|
||||
"multer": "^2.0.1",
|
||||
"mime-types": "^3.0.1",
|
||||
"multer": "^2.0.2",
|
||||
"node-cron": "^3.0.2",
|
||||
"nodemailer": "^6.9.1",
|
||||
"nodemailer": "7.0.5",
|
||||
"pg": "^8.16.3",
|
||||
"react-i18next": "^15.6.0",
|
||||
"sharp": "^0.32.0",
|
||||
"sharp": "0.34.3",
|
||||
"sqlite3": "^5.1.6",
|
||||
"uuid": "^11.1.0",
|
||||
"winston": "^3.8.2",
|
||||
|
||||
@@ -1,60 +0,0 @@
|
||||
require('dotenv').config();
|
||||
const { db } = require('../src/database/db');
|
||||
|
||||
async function checkDatabaseIssues() {
|
||||
console.log('Checking database issues...\n');
|
||||
|
||||
try {
|
||||
// Check email_templates table structure
|
||||
console.log('1. Checking email_templates table structure:');
|
||||
const emailTemplateColumns = await db('email_templates').columnInfo();
|
||||
console.log('Columns:', Object.keys(emailTemplateColumns));
|
||||
|
||||
// Check if any templates exist
|
||||
const templateCount = await db('email_templates').count('* as count');
|
||||
console.log('Template count:', templateCount[0].count);
|
||||
|
||||
// Check for specific template
|
||||
const galleryCreatedTemplate = await db('email_templates')
|
||||
.where('template_key', 'gallery_created')
|
||||
.first();
|
||||
console.log('gallery_created template exists:', !!galleryCreatedTemplate);
|
||||
|
||||
// Check activity_logs table
|
||||
console.log('\n2. Checking activity_logs table:');
|
||||
const activityLogColumns = await db('activity_logs').columnInfo();
|
||||
console.log('Columns:', Object.keys(activityLogColumns));
|
||||
|
||||
// Check migrations table
|
||||
console.log('\n3. Checking migrations status:');
|
||||
const migrations = await db('migrations')
|
||||
.orderBy('id', 'desc')
|
||||
.limit(10);
|
||||
console.log('Latest migrations:');
|
||||
migrations.forEach(m => console.log(` - ${m.filename}`));
|
||||
|
||||
// Test a simple query from notifications route
|
||||
console.log('\n4. Testing notifications query:');
|
||||
try {
|
||||
const notifications = await db('activity_logs')
|
||||
.select(
|
||||
'activity_logs.*',
|
||||
'events.event_name'
|
||||
)
|
||||
.leftJoin('events', 'activity_logs.event_id', 'events.id')
|
||||
.orderBy('activity_logs.created_at', 'desc')
|
||||
.limit(5);
|
||||
console.log(`Found ${notifications.length} notifications`);
|
||||
} catch (error) {
|
||||
console.error('Notifications query failed:', error.message);
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
console.error('Error:', error);
|
||||
} finally {
|
||||
await db.destroy();
|
||||
process.exit(0);
|
||||
}
|
||||
}
|
||||
|
||||
checkDatabaseIssues();
|
||||
@@ -1,42 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
const sqlite3 = require('sqlite3').verbose();
|
||||
|
||||
// Connect to the database
|
||||
const dbPath = '/app/data/photo_sharing.db';
|
||||
console.log(`Connecting to database at: ${dbPath}`);
|
||||
|
||||
const db = new sqlite3.Database(dbPath, sqlite3.OPEN_READONLY, (err) => {
|
||||
if (err) {
|
||||
console.error('Error opening database:', err.message);
|
||||
process.exit(1);
|
||||
}
|
||||
console.log('Connected to the SQLite database.\n');
|
||||
});
|
||||
|
||||
// Get schema for events table
|
||||
console.log('=== EVENTS TABLE SCHEMA ===');
|
||||
db.all("PRAGMA table_info(events)", [], (err, rows) => {
|
||||
if (err) {
|
||||
console.error('Error getting events schema:', err.message);
|
||||
} else {
|
||||
rows.forEach(row => {
|
||||
console.log(`${row.name} (${row.type})`);
|
||||
});
|
||||
}
|
||||
|
||||
console.log('\n=== PHOTOS TABLE SCHEMA ===');
|
||||
// Get schema for photos table
|
||||
db.all("PRAGMA table_info(photos)", [], (err, rows) => {
|
||||
if (err) {
|
||||
console.error('Error getting photos schema:', err.message);
|
||||
} else {
|
||||
rows.forEach(row => {
|
||||
console.log(`${row.name} (${row.type})`);
|
||||
});
|
||||
}
|
||||
|
||||
// Close the database
|
||||
db.close();
|
||||
});
|
||||
});
|
||||
@@ -1,58 +0,0 @@
|
||||
const path = require('path');
|
||||
require('dotenv').config({ path: path.join(__dirname, '../.env') });
|
||||
const { db } = require('../src/database/db');
|
||||
const bcrypt = require('bcrypt');
|
||||
const jwt = require('jsonwebtoken');
|
||||
const { v4: uuidv4 } = require('uuid');
|
||||
|
||||
async function createTestEvent() {
|
||||
try {
|
||||
console.log('Creating test event...');
|
||||
|
||||
// Hash a simple password
|
||||
const passwordHash = await bcrypt.hash('test123', 10);
|
||||
|
||||
// Generate share token
|
||||
const shareToken = uuidv4().replace(/-/g, '');
|
||||
const shareLink = `http://localhost:3005/gallery/wedding-test123-2025-07-07/${shareToken}`;
|
||||
|
||||
// Create event
|
||||
const eventData = {
|
||||
slug: 'wedding-test123-2025-07-07',
|
||||
event_type: 'wedding',
|
||||
event_name: 'Test Wedding',
|
||||
event_date: '2025-07-07',
|
||||
host_email: 'host@example.com',
|
||||
admin_email: 'admin@example.com',
|
||||
password_hash: passwordHash,
|
||||
welcome_message: 'Welcome to our test wedding gallery!',
|
||||
color_theme: null, // Use global theme
|
||||
is_active: 1,
|
||||
expires_at: new Date(Date.now() + 30 * 24 * 60 * 60 * 1000).toISOString(),
|
||||
share_link: shareLink
|
||||
};
|
||||
|
||||
// Delete existing event if it exists
|
||||
await db('events').where('slug', eventData.slug).delete();
|
||||
|
||||
// Insert new event
|
||||
const insertResult = await db('events').insert(eventData).returning('id');
|
||||
const eventId = insertResult[0]?.id || insertResult[0];
|
||||
console.log('Event created with ID:', eventId);
|
||||
|
||||
console.log('\nTest event created successfully!');
|
||||
console.log('Event details:');
|
||||
console.log('- Name:', eventData.event_name);
|
||||
console.log('- Slug:', eventData.slug);
|
||||
console.log('- Password:', 'test123');
|
||||
console.log('- Share link:', shareLink);
|
||||
console.log('\nYou can now access the gallery at the share link above');
|
||||
|
||||
process.exit(0);
|
||||
} catch (error) {
|
||||
console.error('Error creating test event:', error);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
createTestEvent();
|
||||
@@ -1,92 +0,0 @@
|
||||
require('dotenv').config();
|
||||
const { db } = require('../src/database/db');
|
||||
|
||||
async function debugEndpoints() {
|
||||
console.log('Debugging 500 errors...\n');
|
||||
|
||||
try {
|
||||
// Test email templates query
|
||||
console.log('1. Testing email templates query:');
|
||||
try {
|
||||
const templates = await db('email_templates')
|
||||
.select('*')
|
||||
.orderBy('template_key');
|
||||
|
||||
console.log(`Found ${templates.length} templates`);
|
||||
if (templates.length > 0) {
|
||||
console.log('First template columns:', Object.keys(templates[0]));
|
||||
console.log('Template keys:', templates.map(t => t.template_key));
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Email templates query failed:', error.message);
|
||||
console.error('Error code:', error.code);
|
||||
}
|
||||
|
||||
// Test notifications query
|
||||
console.log('\n2. Testing notifications query:');
|
||||
try {
|
||||
const notifications = await db('activity_logs')
|
||||
.select(
|
||||
'activity_logs.*',
|
||||
'events.event_name'
|
||||
)
|
||||
.leftJoin('events', 'activity_logs.event_id', 'events.id')
|
||||
.whereNull('activity_logs.read_at')
|
||||
.orderBy('activity_logs.created_at', 'desc')
|
||||
.limit(5);
|
||||
|
||||
console.log(`Found ${notifications.length} unread notifications`);
|
||||
} catch (error) {
|
||||
console.error('Notifications query failed:', error.message);
|
||||
console.error('Error code:', error.code);
|
||||
|
||||
// Check if it's a column issue
|
||||
if (error.message.includes('column')) {
|
||||
console.log('\nChecking activity_logs columns:');
|
||||
const columns = await db('activity_logs').columnInfo();
|
||||
console.log('Columns:', Object.keys(columns));
|
||||
}
|
||||
}
|
||||
|
||||
// Test specific template query
|
||||
console.log('\n3. Testing specific template query (gallery_created):');
|
||||
try {
|
||||
const template = await db('email_templates')
|
||||
.where('template_key', 'gallery_created')
|
||||
.first();
|
||||
|
||||
if (template) {
|
||||
console.log('Template found:', template.template_key);
|
||||
console.log('Has subject_en?', template.subject_en !== undefined);
|
||||
console.log('Has subject?', template.subject !== undefined);
|
||||
} else {
|
||||
console.log('Template not found');
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Template query failed:', error.message);
|
||||
}
|
||||
|
||||
// Check CMS pages
|
||||
console.log('\n4. Checking CMS pages:');
|
||||
try {
|
||||
const pages = await db('cms_pages')
|
||||
.select('slug', 'title', 'is_published')
|
||||
.orderBy('slug');
|
||||
|
||||
console.log(`Found ${pages.length} CMS pages:`);
|
||||
pages.forEach(page => {
|
||||
console.log(` - ${page.slug}: ${page.title} (published: ${page.is_published})`);
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('CMS pages query failed:', error.message);
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
console.error('General error:', error);
|
||||
} finally {
|
||||
await db.destroy();
|
||||
process.exit(0);
|
||||
}
|
||||
}
|
||||
|
||||
debugEndpoints();
|
||||
@@ -1,132 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* Script to diagnose thumbnail serving issues
|
||||
* Usage: node scripts/diagnose-thumbnails.js <eventId>
|
||||
*/
|
||||
|
||||
const path = require('path');
|
||||
const fs = require('fs').promises;
|
||||
const { db } = require('../src/database/db');
|
||||
|
||||
const STORAGE_PATH = process.env.STORAGE_PATH || path.join(__dirname, '../../storage');
|
||||
const THUMBNAILS_DIR = path.join(STORAGE_PATH, 'thumbnails');
|
||||
|
||||
async function diagnoseThumbnails(eventId) {
|
||||
if (!eventId) {
|
||||
console.error('Usage: node scripts/diagnose-thumbnails.js <eventId>');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
console.log(`Diagnosing thumbnails for event ID: ${eventId}`);
|
||||
console.log(`Storage path: ${STORAGE_PATH}`);
|
||||
console.log(`Thumbnails directory: ${THUMBNAILS_DIR}\n`);
|
||||
|
||||
try {
|
||||
// Get event info
|
||||
const event = await db('events').where('id', eventId).first();
|
||||
if (!event) {
|
||||
console.error(`Event not found with ID: ${eventId}`);
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(`Event: ${event.event_name} (${event.slug})`);
|
||||
console.log(`Active: ${event.is_active}, Archived: ${event.is_archived}\n`);
|
||||
|
||||
// Get photos for this event
|
||||
const photos = await db('photos')
|
||||
.where('event_id', eventId)
|
||||
.select('id', 'filename', 'path', 'thumbnail_path');
|
||||
|
||||
console.log(`Found ${photos.length} photos in database\n`);
|
||||
|
||||
let missingThumbnails = 0;
|
||||
let existingThumbnails = 0;
|
||||
let pathIssues = [];
|
||||
|
||||
for (const photo of photos.slice(0, 10)) { // Check first 10 photos
|
||||
console.log(`Photo ID ${photo.id}: ${photo.filename}`);
|
||||
console.log(` Photo path: ${photo.path}`);
|
||||
console.log(` Thumbnail path in DB: ${photo.thumbnail_path}`);
|
||||
|
||||
if (photo.thumbnail_path) {
|
||||
// Expected thumbnail filename
|
||||
const expectedThumbName = `thumb_${photo.filename}`;
|
||||
const expectedThumbPath = path.join(THUMBNAILS_DIR, expectedThumbName);
|
||||
|
||||
// Check if thumbnail exists
|
||||
try {
|
||||
await fs.access(expectedThumbPath);
|
||||
console.log(` ✓ Thumbnail exists at: ${expectedThumbName}`);
|
||||
existingThumbnails++;
|
||||
|
||||
// Check if DB path matches expected path
|
||||
const dbThumbName = path.basename(photo.thumbnail_path);
|
||||
if (dbThumbName !== expectedThumbName) {
|
||||
console.log(` ⚠ Path mismatch! DB has: ${dbThumbName}, Expected: ${expectedThumbName}`);
|
||||
pathIssues.push({
|
||||
photoId: photo.id,
|
||||
dbPath: photo.thumbnail_path,
|
||||
expectedPath: `thumbnails/${expectedThumbName}`
|
||||
});
|
||||
}
|
||||
} catch {
|
||||
console.log(` ✗ Thumbnail missing: ${expectedThumbName}`);
|
||||
missingThumbnails++;
|
||||
}
|
||||
} else {
|
||||
console.log(` ✗ No thumbnail path in database`);
|
||||
missingThumbnails++;
|
||||
}
|
||||
console.log('');
|
||||
}
|
||||
|
||||
console.log('--- Summary ---');
|
||||
console.log(`Existing thumbnails: ${existingThumbnails}`);
|
||||
console.log(`Missing thumbnails: ${missingThumbnails}`);
|
||||
console.log(`Path issues: ${pathIssues.length}`);
|
||||
|
||||
if (pathIssues.length > 0) {
|
||||
console.log('\n--- Path Issues ---');
|
||||
console.log('The following photos have incorrect thumbnail paths in the database:');
|
||||
for (const issue of pathIssues) {
|
||||
console.log(`Photo ID ${issue.photoId}:`);
|
||||
console.log(` Current: ${issue.dbPath}`);
|
||||
console.log(` Should be: ${issue.expectedPath}`);
|
||||
}
|
||||
|
||||
console.log('\nTo fix path issues, run:');
|
||||
console.log(`UPDATE photos SET thumbnail_path = 'thumbnails/thumb_' || filename WHERE event_id = ${eventId};`);
|
||||
}
|
||||
|
||||
// Check for any thumbnails in the directory that match this event
|
||||
const files = await fs.readdir(THUMBNAILS_DIR);
|
||||
const eventThumbnails = files.filter(f => {
|
||||
// Try to match thumbnails for this event
|
||||
for (const photo of photos) {
|
||||
if (f === `thumb_${photo.filename}`) return true;
|
||||
}
|
||||
return false;
|
||||
});
|
||||
|
||||
console.log(`\n--- Filesystem Check ---`);
|
||||
console.log(`Found ${eventThumbnails.length} thumbnails in directory for this event`);
|
||||
|
||||
} catch (error) {
|
||||
console.error('Error during diagnosis:', error);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
// Parse command line arguments
|
||||
const eventId = process.argv[2] ? parseInt(process.argv[2]) : null;
|
||||
|
||||
// Run the diagnosis
|
||||
diagnoseThumbnails(eventId).then(async () => {
|
||||
await db.destroy();
|
||||
console.log('\nDiagnosis complete');
|
||||
}).catch(async error => {
|
||||
console.error('Diagnosis failed:', error);
|
||||
await db.destroy();
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -0,0 +1,61 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* Fix migration state by marking migrations as applied if their tables already exist
|
||||
*/
|
||||
|
||||
const { db } = require('../src/database/db');
|
||||
|
||||
async function fixMigrationState() {
|
||||
try {
|
||||
console.log('Checking migration state...');
|
||||
|
||||
// Ensure migrations table exists
|
||||
const hasMigrationsTable = await db.schema.hasTable('migrations');
|
||||
if (!hasMigrationsTable) {
|
||||
await db.schema.createTable('migrations', (table) => {
|
||||
table.increments('id').primary();
|
||||
table.string('filename').unique().notNullable();
|
||||
table.timestamp('applied_at').defaultTo(db.fn.now());
|
||||
});
|
||||
console.log('Created migrations tracking table');
|
||||
}
|
||||
|
||||
// Check for specific tables and mark their migrations as applied
|
||||
const tableChecks = [
|
||||
{ table: 'restore_runs', migration: '032_add_restore_runs_table.js' },
|
||||
{ table: 'restore_file_operations', migration: '032_add_restore_runs_table.js' },
|
||||
{ table: 'restore_validation_results', migration: '032_add_restore_runs_table.js' },
|
||||
{ table: 'gallery_feedback', migration: '033_add_gallery_feedback.js' },
|
||||
{ table: 'feedback_photos', migration: '033_add_gallery_feedback.js' },
|
||||
];
|
||||
|
||||
for (const check of tableChecks) {
|
||||
const tableExists = await db.schema.hasTable(check.table);
|
||||
if (tableExists) {
|
||||
const migrationApplied = await db('migrations')
|
||||
.where('filename', check.migration)
|
||||
.first();
|
||||
|
||||
if (!migrationApplied) {
|
||||
await db('migrations').insert({
|
||||
filename: check.migration,
|
||||
applied_at: new Date()
|
||||
});
|
||||
console.log(`✅ Marked ${check.migration} as applied (table ${check.table} exists)`);
|
||||
} else {
|
||||
console.log(`ℹ️ ${check.migration} already marked as applied`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
console.log('\nMigration state fixed successfully!');
|
||||
} catch (error) {
|
||||
console.error('Error fixing migration state:', error.message);
|
||||
process.exit(1);
|
||||
} finally {
|
||||
await db.destroy();
|
||||
}
|
||||
}
|
||||
|
||||
fixMigrationState();
|
||||
@@ -1,145 +0,0 @@
|
||||
require('dotenv').config();
|
||||
const { db } = require('../src/database/db');
|
||||
|
||||
async function fixProductionIssues() {
|
||||
console.log('Fixing production database issues...\n');
|
||||
|
||||
try {
|
||||
// 1. Check and fix email_templates structure
|
||||
console.log('1. Checking email_templates structure:');
|
||||
const emailColumns = await db('email_templates').columnInfo();
|
||||
console.log('Current columns:', Object.keys(emailColumns));
|
||||
|
||||
// Check if we need to add basic columns back
|
||||
const hasSubject = 'subject' in emailColumns;
|
||||
const hasSubjectEn = 'subject_en' in emailColumns;
|
||||
|
||||
if (hasSubjectEn && !hasSubject) {
|
||||
console.log('Adding basic columns back to email_templates...');
|
||||
await db.schema.alterTable('email_templates', (table) => {
|
||||
table.string('subject');
|
||||
table.text('body_html');
|
||||
table.text('body_text');
|
||||
});
|
||||
|
||||
// Copy values from _en columns
|
||||
await db('email_templates').update({
|
||||
subject: db.raw('subject_en'),
|
||||
body_html: db.raw('body_html_en'),
|
||||
body_text: db.raw('body_text_en')
|
||||
});
|
||||
console.log('Basic columns added successfully');
|
||||
}
|
||||
|
||||
// 2. Ensure default templates exist
|
||||
console.log('\n2. Checking email templates:');
|
||||
const templateCount = await db('email_templates').count('* as count');
|
||||
console.log('Template count:', templateCount[0].count);
|
||||
|
||||
if (templateCount[0].count === 0) {
|
||||
console.log('No templates found, inserting defaults...');
|
||||
const defaultTemplates = [
|
||||
{
|
||||
template_key: 'gallery_created',
|
||||
subject: 'Your Photo Gallery is Ready!',
|
||||
body_html: '<h2>Gallery Created Successfully</h2>...',
|
||||
body_text: 'Gallery Created Successfully...',
|
||||
variables: JSON.stringify(['host_name', 'event_name', 'event_date', 'gallery_link', 'gallery_password', 'expiry_date'])
|
||||
},
|
||||
{
|
||||
template_key: 'expiration_warning',
|
||||
subject: 'Your Photo Gallery Expires Soon',
|
||||
body_html: '<h2>Gallery Expiring Soon</h2>...',
|
||||
body_text: 'Gallery Expiring Soon...',
|
||||
variables: JSON.stringify(['host_name', 'event_name', 'days_remaining', 'gallery_link'])
|
||||
},
|
||||
{
|
||||
template_key: 'gallery_expired',
|
||||
subject: 'Your Photo Gallery Has Expired',
|
||||
body_html: '<h2>Gallery Expired</h2>...',
|
||||
body_text: 'Gallery Expired...',
|
||||
variables: JSON.stringify(['host_name', 'event_name'])
|
||||
},
|
||||
{
|
||||
template_key: 'archive_complete',
|
||||
subject: 'Gallery Archive Complete',
|
||||
body_html: '<h2>Archive Complete</h2>...',
|
||||
body_text: 'Archive Complete...',
|
||||
variables: JSON.stringify(['host_name', 'event_name', 'archive_size'])
|
||||
}
|
||||
];
|
||||
|
||||
for (const template of defaultTemplates) {
|
||||
// Add language columns if they exist
|
||||
if (hasSubjectEn) {
|
||||
template.subject_en = template.subject;
|
||||
template.body_html_en = template.body_html;
|
||||
template.body_text_en = template.body_text;
|
||||
template.subject_de = template.subject;
|
||||
template.body_html_de = template.body_html;
|
||||
template.body_text_de = template.body_text;
|
||||
}
|
||||
|
||||
await db('email_templates').insert(template);
|
||||
}
|
||||
console.log('Default templates inserted');
|
||||
}
|
||||
|
||||
// 3. Check activity_logs structure
|
||||
console.log('\n3. Checking activity_logs structure:');
|
||||
const activityColumns = await db('activity_logs').columnInfo();
|
||||
console.log('Columns:', Object.keys(activityColumns));
|
||||
|
||||
// Check if read_at exists
|
||||
if (!('read_at' in activityColumns)) {
|
||||
console.log('Adding read_at column to activity_logs...');
|
||||
await db.schema.alterTable('activity_logs', (table) => {
|
||||
table.datetime('read_at').nullable();
|
||||
});
|
||||
console.log('read_at column added');
|
||||
}
|
||||
|
||||
// 4. Check and add CMS pages
|
||||
console.log('\n4. Checking CMS pages:');
|
||||
const cmsColumns = await db('cms_pages').columnInfo();
|
||||
console.log('CMS columns:', Object.keys(cmsColumns));
|
||||
|
||||
const impressum = await db('cms_pages').where('slug', 'impressum').first();
|
||||
const datenschutz = await db('cms_pages').where('slug', 'datenschutz').first();
|
||||
|
||||
if (!impressum) {
|
||||
console.log('Adding Impressum page...');
|
||||
await db('cms_pages').insert({
|
||||
slug: 'impressum',
|
||||
title_en: 'Legal Notice',
|
||||
title_de: 'Impressum',
|
||||
content_en: '<h1>Legal Notice</h1><p>Your legal information here...</p>',
|
||||
content_de: '<h1>Impressum</h1><p>Ihre rechtlichen Informationen hier...</p>',
|
||||
updated_at: new Date()
|
||||
});
|
||||
}
|
||||
|
||||
if (!datenschutz) {
|
||||
console.log('Adding Datenschutz page...');
|
||||
await db('cms_pages').insert({
|
||||
slug: 'datenschutz',
|
||||
title_en: 'Privacy Policy',
|
||||
title_de: 'Datenschutzerklärung',
|
||||
content_en: '<h1>Privacy Policy</h1><p>Your privacy policy here...</p>',
|
||||
content_de: '<h1>Datenschutzerklärung</h1><p>Ihre Datenschutzerklärung hier...</p>',
|
||||
updated_at: new Date()
|
||||
});
|
||||
}
|
||||
|
||||
console.log('\n✅ All fixes applied successfully!');
|
||||
|
||||
} catch (error) {
|
||||
console.error('Error fixing issues:', error);
|
||||
console.error('Stack:', error.stack);
|
||||
} finally {
|
||||
await db.destroy();
|
||||
process.exit(0);
|
||||
}
|
||||
}
|
||||
|
||||
fixProductionIssues();
|
||||
@@ -0,0 +1,171 @@
|
||||
require('dotenv').config({ path: '../.env' });
|
||||
const path = require('path');
|
||||
const fs = require('fs').promises;
|
||||
const { db } = require('../src/database/db');
|
||||
const { generatePhotoFilename } = require('../src/utils/filenameSanitizer');
|
||||
|
||||
async function fixTempPhotos() {
|
||||
console.log('Starting to fix temporary photo files...\n');
|
||||
|
||||
try {
|
||||
// Find all photos with temp_ filenames
|
||||
const tempPhotos = await db('photos')
|
||||
.where('filename', 'like', 'temp_%')
|
||||
.orderBy('event_id', 'asc')
|
||||
.orderBy('category_id', 'asc')
|
||||
.orderBy('id', 'asc');
|
||||
|
||||
console.log(`Found ${tempPhotos.length} photos with temporary filenames\n`);
|
||||
|
||||
if (tempPhotos.length === 0) {
|
||||
console.log('No temporary photos found. Exiting.');
|
||||
return;
|
||||
}
|
||||
|
||||
// Group photos by event and category
|
||||
const grouped = {};
|
||||
for (const photo of tempPhotos) {
|
||||
const key = `${photo.event_id}_${photo.category_id || 'null'}`;
|
||||
if (!grouped[key]) {
|
||||
grouped[key] = [];
|
||||
}
|
||||
grouped[key].push(photo);
|
||||
}
|
||||
|
||||
console.log(`Processing ${Object.keys(grouped).length} event/category groups...\n`);
|
||||
|
||||
// Process each group
|
||||
for (const [key, photos] of Object.entries(grouped)) {
|
||||
const [eventId, categoryIdStr] = key.split('_');
|
||||
const categoryId = categoryIdStr === 'null' ? null : parseInt(categoryIdStr);
|
||||
|
||||
console.log(`\nProcessing Event ID: ${eventId}, Category ID: ${categoryId || 'uncategorized'}`);
|
||||
console.log(`Photos in group: ${photos.length}`);
|
||||
|
||||
// Get event details
|
||||
const event = await db('events').where({ id: eventId }).first();
|
||||
if (!event) {
|
||||
console.error(`Event ${eventId} not found! Skipping...`);
|
||||
continue;
|
||||
}
|
||||
|
||||
// Get category details if applicable
|
||||
let category = null;
|
||||
let startCounter = 1;
|
||||
|
||||
if (categoryId) {
|
||||
category = await db('photo_categories').where({ id: categoryId }).first();
|
||||
if (!category) {
|
||||
console.error(`Category ${categoryId} not found! Treating as uncategorized...`);
|
||||
} else {
|
||||
// Get the highest counter for this category
|
||||
const maxPhoto = await db('photos')
|
||||
.where({ event_id: eventId, category_id: categoryId })
|
||||
.whereNot('filename', 'like', 'temp_%')
|
||||
.orderBy('id', 'desc')
|
||||
.first();
|
||||
|
||||
if (maxPhoto && maxPhoto.filename) {
|
||||
// Extract counter from filename
|
||||
const match = maxPhoto.filename.match(/_(\d+)\.[^.]+$/);
|
||||
if (match) {
|
||||
startCounter = parseInt(match[1]) + 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// For uncategorized, get the highest counter
|
||||
const maxPhoto = await db('photos')
|
||||
.where({ event_id: eventId })
|
||||
.whereNull('category_id')
|
||||
.whereNot('filename', 'like', 'temp_%')
|
||||
.orderBy('id', 'desc')
|
||||
.first();
|
||||
|
||||
if (maxPhoto && maxPhoto.filename) {
|
||||
const match = maxPhoto.filename.match(/_(\d+)\.[^.]+$/);
|
||||
if (match) {
|
||||
startCounter = parseInt(match[1]) + 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
console.log(`Starting counter: ${startCounter}`);
|
||||
|
||||
// Process each photo in the group
|
||||
let successCount = 0;
|
||||
let errorCount = 0;
|
||||
|
||||
for (let i = 0; i < photos.length; i++) {
|
||||
const photo = photos[i];
|
||||
const counter = startCounter + i;
|
||||
|
||||
try {
|
||||
// Generate new filename
|
||||
const extension = path.extname(photo.filename);
|
||||
const newFilename = generatePhotoFilename(
|
||||
event.event_name,
|
||||
category ? category.name : 'uncategorized',
|
||||
counter,
|
||||
extension
|
||||
);
|
||||
|
||||
// Build full paths
|
||||
const storagePath = process.env.STORAGE_PATH || path.join(__dirname, '../../storage');
|
||||
const oldPath = path.join(storagePath, 'events/active', photo.path);
|
||||
const newPath = path.join(path.dirname(oldPath), newFilename);
|
||||
|
||||
// Check if old file exists
|
||||
try {
|
||||
await fs.access(oldPath);
|
||||
} catch (e) {
|
||||
console.error(`File not found: ${oldPath}`);
|
||||
errorCount++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Rename the file
|
||||
await fs.rename(oldPath, newPath);
|
||||
|
||||
// Update database
|
||||
const newRelativePath = path.relative(path.join(storagePath, 'events/active'), newPath);
|
||||
await db('photos')
|
||||
.where({ id: photo.id })
|
||||
.update({
|
||||
filename: newFilename,
|
||||
path: newRelativePath
|
||||
});
|
||||
|
||||
console.log(`✓ Renamed: ${photo.filename} → ${newFilename}`);
|
||||
successCount++;
|
||||
|
||||
} catch (error) {
|
||||
console.error(`✗ Failed to process photo ${photo.id}: ${error.message}`);
|
||||
errorCount++;
|
||||
}
|
||||
}
|
||||
|
||||
// Update category counter if needed
|
||||
if (category && successCount > 0) {
|
||||
const newCounter = startCounter + photos.length - 1;
|
||||
await db('photo_categories')
|
||||
.where({ id: categoryId })
|
||||
.update({ photo_counter: newCounter });
|
||||
console.log(`Updated category counter to ${newCounter}`);
|
||||
}
|
||||
|
||||
console.log(`\nGroup summary: ${successCount} successful, ${errorCount} errors`);
|
||||
}
|
||||
|
||||
console.log('\n=== COMPLETE ===');
|
||||
console.log('All temporary photos have been processed.');
|
||||
|
||||
} catch (error) {
|
||||
console.error('Fatal error:', error);
|
||||
} finally {
|
||||
await db.destroy();
|
||||
}
|
||||
}
|
||||
|
||||
// Run the script
|
||||
fixTempPhotos().catch(console.error);
|
||||
@@ -0,0 +1,46 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* Mark a specific migration as applied without running it
|
||||
* Usage: node scripts/mark-migration-applied.js <migration-filename>
|
||||
*/
|
||||
|
||||
const { db } = require('../src/database/db');
|
||||
|
||||
async function markMigrationAsApplied(filename) {
|
||||
try {
|
||||
// Check if migration is already marked
|
||||
const existing = await db('migrations')
|
||||
.where('filename', filename)
|
||||
.first();
|
||||
|
||||
if (existing) {
|
||||
console.log(`Migration ${filename} is already marked as applied`);
|
||||
return;
|
||||
}
|
||||
|
||||
// Mark as applied
|
||||
await db('migrations').insert({
|
||||
filename,
|
||||
applied_at: new Date()
|
||||
});
|
||||
|
||||
console.log(`✅ Migration ${filename} marked as applied`);
|
||||
} catch (error) {
|
||||
console.error('Error marking migration:', error.message);
|
||||
process.exit(1);
|
||||
} finally {
|
||||
await db.destroy();
|
||||
}
|
||||
}
|
||||
|
||||
// Get migration filename from command line
|
||||
const migrationFile = process.argv[2];
|
||||
|
||||
if (!migrationFile) {
|
||||
console.error('Usage: node scripts/mark-migration-applied.js <migration-filename>');
|
||||
console.error('Example: node scripts/mark-migration-applied.js 032_add_restore_runs_table.js');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
markMigrationAsApplied(migrationFile);
|
||||
@@ -1,40 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* Run database migrations using existing db connection
|
||||
*/
|
||||
|
||||
const { db } = require('../src/database/db');
|
||||
|
||||
async function runMigrations() {
|
||||
console.log('Running database migrations...\n');
|
||||
|
||||
try {
|
||||
// Run all pending migrations
|
||||
const result = await db.migrate.latest({
|
||||
directory: './migrations'
|
||||
});
|
||||
|
||||
if (result[1].length === 0) {
|
||||
console.log('✓ Database is already up to date');
|
||||
} else {
|
||||
console.log(`✓ Ran ${result[1].length} migrations:`);
|
||||
result[1].forEach(migration => {
|
||||
console.log(` - ${migration}`);
|
||||
});
|
||||
}
|
||||
|
||||
// Show current migration status
|
||||
const list = await db.migrate.list();
|
||||
console.log(`\nCurrent status: ${list[0].length} completed migrations`);
|
||||
|
||||
await db.destroy();
|
||||
process.exit(0);
|
||||
} catch (error) {
|
||||
console.error('Migration error:', error);
|
||||
await db.destroy();
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
runMigrations();
|
||||
@@ -0,0 +1,66 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* Show or reset admin credentials
|
||||
*/
|
||||
|
||||
const bcrypt = require('bcrypt');
|
||||
const { db } = require('../src/database/db');
|
||||
const { generateReadablePassword } = require('../src/utils/passwordGenerator');
|
||||
|
||||
async function showAdminCredentials(resetPassword = false) {
|
||||
try {
|
||||
// Get admin user
|
||||
const admin = await db('admin_users')
|
||||
.where('username', 'admin')
|
||||
.first();
|
||||
|
||||
if (!admin) {
|
||||
console.error('❌ No admin user found in database');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
console.log('\n========================================');
|
||||
console.log('PicPeak Admin Credentials');
|
||||
console.log('========================================');
|
||||
console.log(`Username: ${admin.username}`);
|
||||
console.log(`Email: ${admin.email}`);
|
||||
|
||||
if (resetPassword) {
|
||||
// Generate new password
|
||||
const newPassword = generateReadablePassword();
|
||||
const passwordHash = await bcrypt.hash(newPassword, 12);
|
||||
|
||||
// Update password
|
||||
await db('admin_users')
|
||||
.where('id', admin.id)
|
||||
.update({
|
||||
password_hash: passwordHash,
|
||||
updated_at: new Date()
|
||||
});
|
||||
|
||||
console.log(`Password: ${newPassword} (NEWLY RESET)`);
|
||||
console.log('\n⚠️ IMPORTANT: Please save this password securely!');
|
||||
} else {
|
||||
console.log('Password: [hidden - use --reset flag to generate new password]');
|
||||
}
|
||||
|
||||
console.log('\nLogin URL: http://localhost:3001/admin');
|
||||
console.log('========================================\n');
|
||||
|
||||
} catch (error) {
|
||||
console.error('Error:', error.message);
|
||||
process.exit(1);
|
||||
} finally {
|
||||
await db.destroy();
|
||||
}
|
||||
}
|
||||
|
||||
// Check for reset flag
|
||||
const resetPassword = process.argv.includes('--reset');
|
||||
|
||||
if (resetPassword) {
|
||||
console.log('🔄 Resetting admin password...');
|
||||
}
|
||||
|
||||
showAdminCredentials(resetPassword);
|
||||
Executable
+577
@@ -0,0 +1,577 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* Manual Integration Test Script for Enhanced Backup System
|
||||
*
|
||||
* This script provides a comprehensive test of the backup system with real services.
|
||||
* It can be used to test against MinIO, AWS S3, or other S3-compatible services.
|
||||
*
|
||||
* Usage:
|
||||
* node scripts/test-backup-integration.js [options]
|
||||
*
|
||||
* Options:
|
||||
* --endpoint <url> S3 endpoint URL (default: http://localhost:9000)
|
||||
* --access-key <key> S3 access key (default: minioadmin)
|
||||
* --secret-key <key> S3 secret key (default: minioadmin)
|
||||
* --bucket <name> S3 bucket name (default: test-backup-<timestamp>)
|
||||
* --type <type> Backup type: s3, local, rsync (default: s3)
|
||||
* --cleanup Clean up test data after completion
|
||||
* --verbose Enable verbose logging
|
||||
* --help Show this help message
|
||||
*
|
||||
* Examples:
|
||||
* # Test with local MinIO
|
||||
* node scripts/test-backup-integration.js
|
||||
*
|
||||
* # Test with AWS S3
|
||||
* node scripts/test-backup-integration.js \
|
||||
* --endpoint https://s3.amazonaws.com \
|
||||
* --access-key AKIAIOSFODNN7EXAMPLE \
|
||||
* --secret-key wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY \
|
||||
* --bucket my-test-bucket
|
||||
*
|
||||
* # Test local backup
|
||||
* node scripts/test-backup-integration.js --type local
|
||||
*/
|
||||
|
||||
const path = require('path');
|
||||
const fs = require('fs').promises;
|
||||
const crypto = require('crypto');
|
||||
const { S3Client, CreateBucketCommand, HeadBucketCommand, ListObjectsV2Command, GetObjectCommand, DeleteObjectsCommand, DeleteBucketCommand } = require('@aws-sdk/client-s3');
|
||||
|
||||
// Parse command line arguments
|
||||
const args = process.argv.slice(2);
|
||||
const options = {
|
||||
endpoint: 'http://localhost:9000',
|
||||
accessKey: 'minioadmin',
|
||||
secretKey: 'minioadmin',
|
||||
bucket: `test-backup-${Date.now()}`,
|
||||
type: 's3',
|
||||
cleanup: false,
|
||||
verbose: false
|
||||
};
|
||||
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
switch (args[i]) {
|
||||
case '--endpoint':
|
||||
options.endpoint = args[++i];
|
||||
break;
|
||||
case '--access-key':
|
||||
options.accessKey = args[++i];
|
||||
break;
|
||||
case '--secret-key':
|
||||
options.secretKey = args[++i];
|
||||
break;
|
||||
case '--bucket':
|
||||
options.bucket = args[++i];
|
||||
break;
|
||||
case '--type':
|
||||
options.type = args[++i];
|
||||
break;
|
||||
case '--cleanup':
|
||||
options.cleanup = true;
|
||||
break;
|
||||
case '--verbose':
|
||||
options.verbose = true;
|
||||
break;
|
||||
case '--help':
|
||||
console.log(module.exports.description || 'Manual Integration Test Script');
|
||||
process.exit(0);
|
||||
}
|
||||
}
|
||||
|
||||
// Load environment and services
|
||||
require('dotenv').config();
|
||||
const { db, initialize: initDb } = require('../src/database/db');
|
||||
const backupService = require('../src/services/backupService');
|
||||
const S3StorageAdapter = require('../src/services/storage/s3Storage');
|
||||
const logger = require('../src/utils/logger');
|
||||
|
||||
// Configure logger based on verbose flag
|
||||
if (!options.verbose) {
|
||||
logger.info = () => {};
|
||||
logger.debug = () => {};
|
||||
}
|
||||
|
||||
// Test results
|
||||
const results = {
|
||||
passed: 0,
|
||||
failed: 0,
|
||||
skipped: 0,
|
||||
tests: []
|
||||
};
|
||||
|
||||
// Test utilities
|
||||
async function runTest(name, testFn) {
|
||||
console.log(`\n📋 Running: ${name}`);
|
||||
try {
|
||||
const startTime = Date.now();
|
||||
await testFn();
|
||||
const duration = Date.now() - startTime;
|
||||
console.log(`✅ PASSED: ${name} (${duration}ms)`);
|
||||
results.passed++;
|
||||
results.tests.push({ name, status: 'passed', duration });
|
||||
} catch (error) {
|
||||
console.error(`❌ FAILED: ${name}`);
|
||||
console.error(` Error: ${error.message}`);
|
||||
if (options.verbose) {
|
||||
console.error(error.stack);
|
||||
}
|
||||
results.failed++;
|
||||
results.tests.push({ name, status: 'failed', error: error.message });
|
||||
}
|
||||
}
|
||||
|
||||
async function skipTest(name, reason) {
|
||||
console.log(`\n⏭️ Skipping: ${name}`);
|
||||
console.log(` Reason: ${reason}`);
|
||||
results.skipped++;
|
||||
results.tests.push({ name, status: 'skipped', reason });
|
||||
}
|
||||
|
||||
// Test functions
|
||||
async function testS3Connection() {
|
||||
const s3Adapter = new S3StorageAdapter({
|
||||
bucket: options.bucket,
|
||||
endpoint: options.endpoint,
|
||||
accessKeyId: options.accessKey,
|
||||
secretAccessKey: options.secretKey,
|
||||
region: 'us-east-1',
|
||||
forcePathStyle: true,
|
||||
sslEnabled: options.endpoint.startsWith('https')
|
||||
});
|
||||
|
||||
await s3Adapter.testConnection();
|
||||
console.log(` ✓ Connected to S3 endpoint: ${options.endpoint}`);
|
||||
console.log(` ✓ Bucket accessible: ${options.bucket}`);
|
||||
}
|
||||
|
||||
async function setupTestData() {
|
||||
const storagePath = path.join(__dirname, '../test-storage');
|
||||
process.env.STORAGE_PATH = storagePath;
|
||||
|
||||
// Create directory structure
|
||||
const dirs = [
|
||||
'events/active/wedding-2024',
|
||||
'events/active/birthday-2024',
|
||||
'events/archived',
|
||||
'thumbnails',
|
||||
'uploads',
|
||||
'backups'
|
||||
];
|
||||
|
||||
for (const dir of dirs) {
|
||||
await fs.mkdir(path.join(storagePath, dir), { recursive: true });
|
||||
}
|
||||
|
||||
// Create test files with various sizes
|
||||
const files = [
|
||||
{ path: 'events/active/wedding-2024/photo1.jpg', size: 1024 * 1024 }, // 1MB
|
||||
{ path: 'events/active/wedding-2024/photo2.jpg', size: 512 * 1024 }, // 512KB
|
||||
{ path: 'events/active/birthday-2024/photo1.jpg', size: 2 * 1024 * 1024 }, // 2MB
|
||||
{ path: 'events/archived/old-event.zip', size: 5 * 1024 * 1024 }, // 5MB
|
||||
{ path: 'thumbnails/thumb1.jpg', size: 50 * 1024 }, // 50KB
|
||||
{ path: 'uploads/logo.png', size: 100 * 1024 } // 100KB
|
||||
];
|
||||
|
||||
let totalSize = 0;
|
||||
for (const file of files) {
|
||||
const content = crypto.randomBytes(file.size);
|
||||
await fs.writeFile(path.join(storagePath, file.path), content);
|
||||
totalSize += file.size;
|
||||
}
|
||||
|
||||
console.log(` ✓ Created ${files.length} test files`);
|
||||
console.log(` ✓ Total size: ${(totalSize / 1024 / 1024).toFixed(2)} MB`);
|
||||
|
||||
return { storagePath, fileCount: files.length, totalSize };
|
||||
}
|
||||
|
||||
async function configureBackup(type) {
|
||||
const baseSettings = [
|
||||
{ setting_key: 'backup_enabled', setting_value: 'true' },
|
||||
{ setting_key: 'backup_destination_type', setting_value: `"${type}"` },
|
||||
{ setting_key: 'backup_include_archived', setting_value: 'true' },
|
||||
{ setting_key: 'backup_include_database', setting_value: 'true' },
|
||||
{ setting_key: 'backup_incremental', setting_value: 'true' },
|
||||
{ setting_key: 'backup_manifest_format', setting_value: '"json"' },
|
||||
{ setting_key: 'backup_max_file_size_mb', setting_value: '100' }
|
||||
];
|
||||
|
||||
const typeSpecificSettings = {
|
||||
s3: [
|
||||
{ setting_key: 'backup_s3_bucket', setting_value: `"${options.bucket}"` },
|
||||
{ setting_key: 'backup_s3_endpoint', setting_value: `"${options.endpoint}"` },
|
||||
{ setting_key: 'backup_s3_access_key', setting_value: `"${options.accessKey}"` },
|
||||
{ setting_key: 'backup_s3_secret_key', setting_value: `"${options.secretKey}"` },
|
||||
{ setting_key: 'backup_s3_region', setting_value: '"us-east-1"' },
|
||||
{ setting_key: 'backup_s3_force_path_style', setting_value: 'true' },
|
||||
{ setting_key: 'backup_s3_ssl_enabled', setting_value: options.endpoint.startsWith('https') ? 'true' : 'false' }
|
||||
],
|
||||
local: [
|
||||
{ setting_key: 'backup_destination_path', setting_value: `"${path.join(__dirname, '../test-backup')}"` }
|
||||
],
|
||||
rsync: [
|
||||
{ setting_key: 'backup_rsync_host', setting_value: '"localhost"' },
|
||||
{ setting_key: 'backup_rsync_path', setting_value: `"${path.join(__dirname, '../test-backup-rsync')}"` }
|
||||
]
|
||||
};
|
||||
|
||||
const settings = [...baseSettings, ...(typeSpecificSettings[type] || [])];
|
||||
|
||||
// Clear existing settings
|
||||
await db('app_settings').where('setting_type', 'backup').del();
|
||||
|
||||
// Insert new settings
|
||||
for (const setting of settings) {
|
||||
await db('app_settings').insert({
|
||||
setting_type: 'backup',
|
||||
...setting,
|
||||
created_at: new Date(),
|
||||
updated_at: new Date()
|
||||
});
|
||||
}
|
||||
|
||||
console.log(` ✓ Configured ${type} backup with ${settings.length} settings`);
|
||||
}
|
||||
|
||||
async function performBackup() {
|
||||
const startTime = Date.now();
|
||||
|
||||
// Run the backup
|
||||
await backupService.runBackup();
|
||||
|
||||
// Get backup results
|
||||
const backupRun = await db('backup_runs')
|
||||
.orderBy('started_at', 'desc')
|
||||
.first();
|
||||
|
||||
if (!backupRun) {
|
||||
throw new Error('No backup run found');
|
||||
}
|
||||
|
||||
if (backupRun.status !== 'completed') {
|
||||
throw new Error(`Backup failed with status: ${backupRun.status}, error: ${backupRun.error_message}`);
|
||||
}
|
||||
|
||||
const duration = Date.now() - startTime;
|
||||
|
||||
console.log(` ✓ Backup completed in ${duration}ms`);
|
||||
console.log(` ✓ Files backed up: ${backupRun.files_backed_up}`);
|
||||
console.log(` ✓ Total size: ${(backupRun.total_size_bytes / 1024 / 1024).toFixed(2)} MB`);
|
||||
console.log(` ✓ Manifest: ${backupRun.manifest_path ? 'Generated' : 'Not generated'}`);
|
||||
|
||||
return backupRun;
|
||||
}
|
||||
|
||||
async function verifyS3Backup(backupRun) {
|
||||
const s3Client = new S3Client({
|
||||
endpoint: options.endpoint,
|
||||
region: 'us-east-1',
|
||||
credentials: {
|
||||
accessKeyId: options.accessKey,
|
||||
secretAccessKey: options.secretKey
|
||||
},
|
||||
forcePathStyle: true
|
||||
});
|
||||
|
||||
// List objects in bucket
|
||||
const listResponse = await s3Client.send(new ListObjectsV2Command({
|
||||
Bucket: options.bucket
|
||||
}));
|
||||
|
||||
const objects = listResponse.Contents || [];
|
||||
console.log(` ✓ Objects in S3: ${objects.length}`);
|
||||
|
||||
// Verify key components
|
||||
const hasBackupFolder = objects.some(obj => obj.Key.includes('backup-'));
|
||||
const hasManifest = objects.some(obj => obj.Key.includes('backup-manifest'));
|
||||
const hasSummary = objects.some(obj => obj.Key.includes('backup-summary.json'));
|
||||
const hasPhotos = objects.some(obj => obj.Key.includes('events/active'));
|
||||
|
||||
if (!hasBackupFolder) throw new Error('No backup folder found in S3');
|
||||
if (!hasManifest) throw new Error('No manifest found in S3');
|
||||
if (!hasSummary) throw new Error('No summary found in S3');
|
||||
if (!hasPhotos) throw new Error('No photos found in S3');
|
||||
|
||||
console.log(` ✓ Backup structure verified`);
|
||||
|
||||
// Download and verify a file
|
||||
const photoObject = objects.find(obj => obj.Key.includes('photo1.jpg'));
|
||||
if (photoObject) {
|
||||
const getResponse = await s3Client.send(new GetObjectCommand({
|
||||
Bucket: options.bucket,
|
||||
Key: photoObject.Key
|
||||
}));
|
||||
|
||||
const chunks = [];
|
||||
for await (const chunk of getResponse.Body) {
|
||||
chunks.push(chunk);
|
||||
}
|
||||
const content = Buffer.concat(chunks);
|
||||
|
||||
console.log(` ✓ Downloaded test file: ${photoObject.Key} (${content.length} bytes)`);
|
||||
}
|
||||
}
|
||||
|
||||
async function testIncrementalBackup(testData) {
|
||||
// Modify a file
|
||||
const modifiedFile = path.join(testData.storagePath, 'events/active/wedding-2024/photo1.jpg');
|
||||
const newContent = crypto.randomBytes(1024 * 1024 + 100); // Slightly larger
|
||||
await fs.writeFile(modifiedFile, newContent);
|
||||
|
||||
console.log(` ✓ Modified test file`);
|
||||
|
||||
// Perform incremental backup
|
||||
const backupRun = await performBackup();
|
||||
|
||||
if (backupRun.files_backed_up !== 1) {
|
||||
throw new Error(`Expected 1 file in incremental backup, got ${backupRun.files_backed_up}`);
|
||||
}
|
||||
|
||||
console.log(` ✓ Incremental backup correctly identified changed file`);
|
||||
|
||||
// Verify manifest indicates incremental
|
||||
if (backupRun.manifest_path) {
|
||||
const { manifest } = await backupService.getBackupManifest(backupRun.id);
|
||||
if (!manifest.incremental) {
|
||||
throw new Error('Manifest does not indicate incremental backup');
|
||||
}
|
||||
console.log(` ✓ Manifest correctly marked as incremental`);
|
||||
}
|
||||
|
||||
return backupRun;
|
||||
}
|
||||
|
||||
async function testManifestValidation(backupRun) {
|
||||
if (!backupRun.manifest_path) {
|
||||
throw new Error('No manifest path in backup run');
|
||||
}
|
||||
|
||||
const result = await backupService.validateBackupManifest(backupRun.manifest_path);
|
||||
|
||||
if (!result.valid) {
|
||||
throw new Error(`Manifest validation failed: ${result.error}`);
|
||||
}
|
||||
|
||||
console.log(` ✓ Manifest validation passed`);
|
||||
console.log(` ✓ Manifest version: ${result.manifest.manifest.version}`);
|
||||
console.log(` ✓ Files in manifest: ${result.manifest.files.count}`);
|
||||
}
|
||||
|
||||
async function testBackupStatus() {
|
||||
const status = await backupService.getBackupStatus(5);
|
||||
|
||||
console.log(` ✓ Backup service running: ${status.isRunning}`);
|
||||
console.log(` ✓ Backup service healthy: ${status.isHealthy}`);
|
||||
console.log(` ✓ Recent runs: ${status.recentRuns.length}`);
|
||||
|
||||
if (status.lastRun) {
|
||||
console.log(` ✓ Last run status: ${status.lastRun.status}`);
|
||||
console.log(` ✓ Manifest valid: ${status.lastRun.manifestValid}`);
|
||||
}
|
||||
}
|
||||
|
||||
async function cleanupTestData() {
|
||||
if (!options.cleanup) {
|
||||
console.log('\n📌 Test data retained for inspection');
|
||||
console.log(` Storage: ${process.env.STORAGE_PATH}`);
|
||||
if (options.type === 's3') {
|
||||
console.log(` S3 Bucket: ${options.bucket}`);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
console.log('\n🧹 Cleaning up test data...');
|
||||
|
||||
// Clean storage directory
|
||||
if (process.env.STORAGE_PATH) {
|
||||
await fs.rm(process.env.STORAGE_PATH, { recursive: true, force: true });
|
||||
console.log(' ✓ Removed test storage directory');
|
||||
}
|
||||
|
||||
// Clean S3 bucket if used
|
||||
if (options.type === 's3') {
|
||||
const s3Client = new S3Client({
|
||||
endpoint: options.endpoint,
|
||||
region: 'us-east-1',
|
||||
credentials: {
|
||||
accessKeyId: options.accessKey,
|
||||
secretAccessKey: options.secretKey
|
||||
},
|
||||
forcePathStyle: true
|
||||
});
|
||||
|
||||
try {
|
||||
// List and delete all objects
|
||||
const listResponse = await s3Client.send(new ListObjectsV2Command({
|
||||
Bucket: options.bucket
|
||||
}));
|
||||
|
||||
if (listResponse.Contents && listResponse.Contents.length > 0) {
|
||||
await s3Client.send(new DeleteObjectsCommand({
|
||||
Bucket: options.bucket,
|
||||
Delete: {
|
||||
Objects: listResponse.Contents.map(obj => ({ Key: obj.Key }))
|
||||
}
|
||||
}));
|
||||
console.log(` ✓ Deleted ${listResponse.Contents.length} objects from S3`);
|
||||
}
|
||||
|
||||
// Delete bucket
|
||||
await s3Client.send(new DeleteBucketCommand({
|
||||
Bucket: options.bucket
|
||||
}));
|
||||
console.log(` ✓ Deleted S3 bucket: ${options.bucket}`);
|
||||
} catch (error) {
|
||||
console.error(` ⚠️ Failed to cleanup S3: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Clean backup directories
|
||||
const backupDirs = [
|
||||
path.join(__dirname, '../test-backup'),
|
||||
path.join(__dirname, '../test-backup-rsync')
|
||||
];
|
||||
|
||||
for (const dir of backupDirs) {
|
||||
await fs.rm(dir, { recursive: true, force: true }).catch(() => {});
|
||||
}
|
||||
console.log(' ✓ Removed backup directories');
|
||||
}
|
||||
|
||||
// Main test runner
|
||||
async function main() {
|
||||
console.log('🚀 Enhanced Backup System Integration Test');
|
||||
console.log('==========================================');
|
||||
console.log(`Type: ${options.type}`);
|
||||
console.log(`Endpoint: ${options.endpoint}`);
|
||||
console.log(`Bucket: ${options.bucket}`);
|
||||
console.log('');
|
||||
|
||||
let s3Client;
|
||||
let testData;
|
||||
|
||||
try {
|
||||
// Initialize database
|
||||
console.log('📦 Initializing database...');
|
||||
await initDb();
|
||||
await db.migrate.latest();
|
||||
console.log(' ✓ Database initialized');
|
||||
|
||||
// S3-specific setup
|
||||
if (options.type === 's3') {
|
||||
// Test S3 connection
|
||||
await runTest('S3 Connection Test', testS3Connection);
|
||||
|
||||
// Create S3 bucket if needed
|
||||
s3Client = new S3Client({
|
||||
endpoint: options.endpoint,
|
||||
region: 'us-east-1',
|
||||
credentials: {
|
||||
accessKeyId: options.accessKey,
|
||||
secretAccessKey: options.secretKey
|
||||
},
|
||||
forcePathStyle: true
|
||||
});
|
||||
|
||||
try {
|
||||
await s3Client.send(new HeadBucketCommand({ Bucket: options.bucket }));
|
||||
console.log(`\n📦 Using existing bucket: ${options.bucket}`);
|
||||
} catch (error) {
|
||||
if (error.name === 'NotFound') {
|
||||
await s3Client.send(new CreateBucketCommand({ Bucket: options.bucket }));
|
||||
console.log(`\n📦 Created new bucket: ${options.bucket}`);
|
||||
} else {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Setup test data
|
||||
console.log('\n📁 Setting up test data...');
|
||||
testData = await setupTestData();
|
||||
|
||||
// Configure backup
|
||||
console.log(`\n⚙️ Configuring ${options.type} backup...`);
|
||||
await configureBackup(options.type);
|
||||
|
||||
// Run tests based on backup type
|
||||
await runTest('Initial Full Backup', performBackup);
|
||||
|
||||
if (options.type === 's3') {
|
||||
await runTest('Verify S3 Backup Contents', async () => {
|
||||
const lastRun = await db('backup_runs').orderBy('started_at', 'desc').first();
|
||||
await verifyS3Backup(lastRun);
|
||||
});
|
||||
}
|
||||
|
||||
await runTest('Incremental Backup', () => testIncrementalBackup(testData));
|
||||
|
||||
await runTest('Manifest Validation', async () => {
|
||||
const lastRun = await db('backup_runs').orderBy('started_at', 'desc').first();
|
||||
await testManifestValidation(lastRun);
|
||||
});
|
||||
|
||||
await runTest('Backup Status Check', testBackupStatus);
|
||||
|
||||
// Performance test with larger files
|
||||
if (options.type === 's3') {
|
||||
await runTest('Large File Backup (10MB)', async () => {
|
||||
const largeFile = path.join(testData.storagePath, 'events/active/large.jpg');
|
||||
await fs.writeFile(largeFile, crypto.randomBytes(10 * 1024 * 1024));
|
||||
await performBackup();
|
||||
});
|
||||
}
|
||||
|
||||
// Test backup service lifecycle
|
||||
await runTest('Backup Service Start/Stop', async () => {
|
||||
await backupService.startBackupService();
|
||||
console.log(' ✓ Service started');
|
||||
|
||||
backupService.stopBackupService();
|
||||
console.log(' ✓ Service stopped');
|
||||
});
|
||||
|
||||
// Print results summary
|
||||
console.log('\n📊 Test Results Summary');
|
||||
console.log('======================');
|
||||
console.log(`✅ Passed: ${results.passed}`);
|
||||
console.log(`❌ Failed: ${results.failed}`);
|
||||
console.log(`⏭️ Skipped: ${results.skipped}`);
|
||||
console.log(`📋 Total: ${results.tests.length}`);
|
||||
|
||||
if (results.failed > 0) {
|
||||
console.log('\nFailed Tests:');
|
||||
results.tests
|
||||
.filter(t => t.status === 'failed')
|
||||
.forEach(t => console.log(` - ${t.name}: ${t.error}`));
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
console.error('\n💥 Fatal error:', error.message);
|
||||
if (options.verbose) {
|
||||
console.error(error.stack);
|
||||
}
|
||||
results.failed++;
|
||||
} finally {
|
||||
// Cleanup
|
||||
await cleanupTestData();
|
||||
|
||||
// Close database
|
||||
await db.destroy();
|
||||
|
||||
// Exit with appropriate code
|
||||
process.exit(results.failed > 0 ? 1 : 0);
|
||||
}
|
||||
}
|
||||
|
||||
// Run if called directly
|
||||
if (require.main === module) {
|
||||
main().catch(error => {
|
||||
console.error('Unhandled error:', error);
|
||||
process.exit(1);
|
||||
});
|
||||
}
|
||||
|
||||
module.exports = { runTest, skipTest };
|
||||
@@ -0,0 +1,46 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
require('dotenv').config();
|
||||
const { initializeDatabase } = require('../src/database/db');
|
||||
const { runBackup, getBackupStatus } = require('../src/services/backupService');
|
||||
const logger = require('../src/utils/logger');
|
||||
|
||||
async function testBackupService() {
|
||||
try {
|
||||
console.log('Testing backup service...\n');
|
||||
|
||||
// Initialize database
|
||||
await initializeDatabase();
|
||||
|
||||
// Get current backup status
|
||||
console.log('Getting backup status...');
|
||||
const statusBefore = await getBackupStatus();
|
||||
console.log('Last run:', statusBefore.lastRun ? statusBefore.lastRun.started_at : 'Never');
|
||||
console.log('Is healthy:', statusBefore.isHealthy);
|
||||
console.log('');
|
||||
|
||||
// Run backup
|
||||
console.log('Running backup...');
|
||||
await runBackup();
|
||||
|
||||
// Get status after backup
|
||||
console.log('\nGetting status after backup...');
|
||||
const statusAfter = await getBackupStatus();
|
||||
console.log('Last run:', statusAfter.lastRun ? statusAfter.lastRun.started_at : 'Never');
|
||||
console.log('Status:', statusAfter.lastRun ? statusAfter.lastRun.status : 'Unknown');
|
||||
console.log('Files backed up:', statusAfter.lastRun ? statusAfter.lastRun.files_backed_up : 0);
|
||||
console.log('Total size:', statusAfter.lastRun ? `${(statusAfter.lastRun.total_size_bytes / 1024 / 1024).toFixed(2)} MB` : '0 MB');
|
||||
|
||||
if (statusAfter.lastRun && statusAfter.lastRun.error_message) {
|
||||
console.log('Error:', statusAfter.lastRun.error_message);
|
||||
}
|
||||
|
||||
console.log('\nBackup test completed!');
|
||||
process.exit(0);
|
||||
} catch (error) {
|
||||
console.error('Test failed:', error);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
testBackupService();
|
||||
@@ -1,86 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* Script to test photo authentication
|
||||
* Usage: node scripts/test-photo-auth.js <jwt-token>
|
||||
*/
|
||||
|
||||
const axios = require('axios');
|
||||
|
||||
async function testPhotoAuth(token) {
|
||||
if (!token) {
|
||||
console.error('Usage: node scripts/test-photo-auth.js <jwt-token>');
|
||||
console.error('\nTo get a token, login to a gallery and check localStorage for gallery_token_<slug>');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const baseUrl = process.env.API_URL || 'http://localhost:3001';
|
||||
|
||||
console.log(`Testing photo authentication with token: ${token.substring(0, 20)}...`);
|
||||
console.log(`Base URL: ${baseUrl}\n`);
|
||||
|
||||
// Test URLs
|
||||
const tests = [
|
||||
{
|
||||
name: 'Thumbnail via static route',
|
||||
url: `${baseUrl}/thumbnails/thumb_Test_Gallery_uncategorized_5210.jpg`,
|
||||
headers: { 'Authorization': `Bearer ${token}` }
|
||||
},
|
||||
{
|
||||
name: 'Photo via static route',
|
||||
url: `${baseUrl}/photos/wedding-test-gallery-2025-07-14-1/Test_Gallery_uncategorized_5210.jpg`,
|
||||
headers: { 'Authorization': `Bearer ${token}` }
|
||||
},
|
||||
{
|
||||
name: 'Gallery photos API',
|
||||
url: `${baseUrl}/api/gallery/wedding-test-gallery-2025-07-14-1/photos`,
|
||||
headers: { 'Authorization': `Bearer ${token}` }
|
||||
}
|
||||
];
|
||||
|
||||
for (const test of tests) {
|
||||
console.log(`Testing: ${test.name}`);
|
||||
console.log(`URL: ${test.url}`);
|
||||
|
||||
try {
|
||||
const response = await axios.get(test.url, {
|
||||
headers: test.headers,
|
||||
validateStatus: () => true // Don't throw on any status
|
||||
});
|
||||
|
||||
console.log(`Status: ${response.status}`);
|
||||
console.log(`Headers:`, response.headers['content-type']);
|
||||
|
||||
if (response.status === 200) {
|
||||
if (test.name.includes('API')) {
|
||||
console.log(`Photos count: ${response.data.photos?.length || 0}`);
|
||||
} else {
|
||||
console.log(`Content length: ${response.headers['content-length']} bytes`);
|
||||
}
|
||||
} else {
|
||||
console.log(`Error:`, response.data);
|
||||
}
|
||||
} catch (error) {
|
||||
console.log(`Network error:`, error.message);
|
||||
}
|
||||
|
||||
console.log('---\n');
|
||||
}
|
||||
|
||||
// Decode token to show info
|
||||
try {
|
||||
const parts = token.split('.');
|
||||
const payload = JSON.parse(Buffer.from(parts[1], 'base64').toString());
|
||||
console.log('Token payload:', payload);
|
||||
} catch (error) {
|
||||
console.log('Failed to decode token');
|
||||
}
|
||||
}
|
||||
|
||||
// Get token from command line
|
||||
const token = process.argv[2];
|
||||
|
||||
testPhotoAuth(token).catch(error => {
|
||||
console.error('Test failed:', error);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -0,0 +1,325 @@
|
||||
/**
|
||||
* Test script for the restore service
|
||||
*
|
||||
* This script demonstrates the restore service functionality with safety checks
|
||||
*
|
||||
* Usage:
|
||||
* node scripts/test-restore-service.js [options]
|
||||
*
|
||||
* Options:
|
||||
* --dry-run Perform validation only without actual restore
|
||||
* --force Force restore even with warnings
|
||||
* --type Restore type: full, database, files, selective (default: full)
|
||||
* --source Backup source path or S3 URL
|
||||
* --manifest Path to backup manifest
|
||||
*/
|
||||
|
||||
require('dotenv').config();
|
||||
const { restoreService } = require('../src/services/restoreService');
|
||||
const { db } = require('../src/database/db');
|
||||
const logger = require('../src/utils/logger');
|
||||
const path = require('path');
|
||||
const fs = require('fs').promises;
|
||||
|
||||
// Parse command line arguments
|
||||
const args = process.argv.slice(2);
|
||||
const options = {
|
||||
dryRun: args.includes('--dry-run'),
|
||||
force: args.includes('--force'),
|
||||
restoreType: 'full',
|
||||
source: null,
|
||||
manifestPath: null
|
||||
};
|
||||
|
||||
// Parse restore type
|
||||
const typeIndex = args.indexOf('--type');
|
||||
if (typeIndex !== -1 && args[typeIndex + 1]) {
|
||||
options.restoreType = args[typeIndex + 1];
|
||||
}
|
||||
|
||||
// Parse source
|
||||
const sourceIndex = args.indexOf('--source');
|
||||
if (sourceIndex !== -1 && args[sourceIndex + 1]) {
|
||||
options.source = args[sourceIndex + 1];
|
||||
}
|
||||
|
||||
// Parse manifest
|
||||
const manifestIndex = args.indexOf('--manifest');
|
||||
if (manifestIndex !== -1 && args[manifestIndex + 1]) {
|
||||
options.manifestPath = args[manifestIndex + 1];
|
||||
}
|
||||
|
||||
async function testRestore() {
|
||||
console.log('=== PicPeak Restore Service Test ===\n');
|
||||
|
||||
try {
|
||||
// If no source/manifest provided, try to find a recent backup
|
||||
if (!options.source || !options.manifestPath) {
|
||||
console.log('No backup source specified. Looking for recent backups...\n');
|
||||
|
||||
const recentBackup = await db('backup_runs')
|
||||
.where('status', 'completed')
|
||||
.whereNotNull('manifest_path')
|
||||
.orderBy('completed_at', 'desc')
|
||||
.first();
|
||||
|
||||
if (!recentBackup) {
|
||||
console.error('❌ No completed backups found in the database');
|
||||
console.log('\nPlease run a backup first or specify --source and --manifest');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
console.log(`Found recent backup from ${recentBackup.completed_at}`);
|
||||
console.log(`Backup ID: ${recentBackup.manifest_id}`);
|
||||
console.log(`Files backed up: ${recentBackup.files_backed_up}`);
|
||||
console.log(`Total size: ${(recentBackup.total_size_bytes / 1024 / 1024).toFixed(2)} MB`);
|
||||
console.log(`Manifest: ${recentBackup.manifest_path}\n`);
|
||||
|
||||
// For this test, we'll create a mock scenario
|
||||
console.log('⚠️ This is a TEST MODE - using mock data for safety\n');
|
||||
|
||||
// Create test backup directory
|
||||
const testBackupDir = path.join(__dirname, '../temp/test-backup');
|
||||
await fs.mkdir(testBackupDir, { recursive: true });
|
||||
|
||||
// Create test manifest
|
||||
const testManifest = {
|
||||
manifest: {
|
||||
version: '2.0',
|
||||
created: new Date().toISOString(),
|
||||
generator: 'Test Script',
|
||||
format: 'json'
|
||||
},
|
||||
backup: {
|
||||
id: 'test-backup-' + Date.now(),
|
||||
type: 'full',
|
||||
timestamp: new Date().toISOString(),
|
||||
path: testBackupDir,
|
||||
parent_backup_id: null,
|
||||
retention_days: 30
|
||||
},
|
||||
system: {
|
||||
hostname: require('os').hostname(),
|
||||
platform: process.platform,
|
||||
os_release: require('os').release(),
|
||||
architecture: require('os').arch()
|
||||
},
|
||||
application: {
|
||||
name: 'PicPeak',
|
||||
version: require('../package.json').version,
|
||||
node_version: process.version,
|
||||
environment: 'test'
|
||||
},
|
||||
files: {
|
||||
count: 0,
|
||||
total_size: 0,
|
||||
checksums: {},
|
||||
manifest: []
|
||||
},
|
||||
database: {
|
||||
type: process.env.DB_TYPE === 'postgresql' ? 'postgresql' : 'sqlite',
|
||||
backup_file: null,
|
||||
size: 0,
|
||||
checksum: null,
|
||||
tables: {},
|
||||
row_counts: {}
|
||||
},
|
||||
verification: {
|
||||
total_checksum: null,
|
||||
file_count_check: 0,
|
||||
size_check: 0,
|
||||
integrity_timestamp: new Date().toISOString()
|
||||
},
|
||||
metadata: {
|
||||
test_mode: true
|
||||
}
|
||||
};
|
||||
|
||||
// Calculate checksum
|
||||
const crypto = require('crypto');
|
||||
const manifestCopy = JSON.parse(JSON.stringify(testManifest));
|
||||
delete manifestCopy.verification.total_checksum;
|
||||
testManifest.verification.total_checksum = crypto
|
||||
.createHash('sha256')
|
||||
.update(JSON.stringify(manifestCopy, Object.keys(manifestCopy).sort()))
|
||||
.digest('hex');
|
||||
|
||||
// Save test manifest
|
||||
const testManifestPath = path.join(testBackupDir, 'test-manifest.json');
|
||||
await fs.writeFile(testManifestPath, JSON.stringify(testManifest, null, 2));
|
||||
|
||||
options.source = testBackupDir;
|
||||
options.manifestPath = testManifestPath;
|
||||
}
|
||||
|
||||
// Display restore options
|
||||
console.log('Restore Options:');
|
||||
console.log(`- Type: ${options.restoreType}`);
|
||||
console.log(`- Source: ${options.source}`);
|
||||
console.log(`- Manifest: ${options.manifestPath}`);
|
||||
console.log(`- Dry Run: ${options.dryRun ? 'Yes' : 'No'}`);
|
||||
console.log(`- Force: ${options.force ? 'Yes' : 'No'}`);
|
||||
console.log('');
|
||||
|
||||
// Add S3 config if source is S3
|
||||
if (options.source.startsWith('s3://')) {
|
||||
options.s3Config = {
|
||||
accessKeyId: process.env.BACKUP_S3_ACCESS_KEY,
|
||||
secretAccessKey: process.env.BACKUP_S3_SECRET_KEY,
|
||||
region: process.env.BACKUP_S3_REGION || 'us-east-1',
|
||||
endpoint: process.env.BACKUP_S3_ENDPOINT
|
||||
};
|
||||
|
||||
if (!options.s3Config.accessKeyId || !options.s3Config.secretAccessKey) {
|
||||
console.error('❌ S3 credentials not configured in environment');
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
// Confirm before proceeding (unless dry run)
|
||||
if (!options.dryRun) {
|
||||
console.log('⚠️ WARNING: This will restore data from the backup!');
|
||||
console.log('⚠️ Current data may be overwritten!');
|
||||
console.log('');
|
||||
console.log('Press Ctrl+C to cancel, or wait 5 seconds to continue...');
|
||||
await new Promise(resolve => setTimeout(resolve, 5000));
|
||||
}
|
||||
|
||||
console.log('\nStarting restore operation...\n');
|
||||
|
||||
// Perform restore
|
||||
const result = await restoreService.restore(options);
|
||||
|
||||
if (options.dryRun) {
|
||||
console.log('\n=== DRY RUN RESULTS ===\n');
|
||||
|
||||
console.log('Validation:');
|
||||
console.log(`- Valid: ${result.validation.isValid ? '✅ Yes' : '❌ No'}`);
|
||||
|
||||
if (result.validation.errors.length > 0) {
|
||||
console.log('- Errors:');
|
||||
result.validation.errors.forEach(err => console.log(` ❌ ${err}`));
|
||||
}
|
||||
|
||||
if (result.validation.warnings.length > 0) {
|
||||
console.log('- Warnings:');
|
||||
result.validation.warnings.forEach(warn => console.log(` ⚠️ ${warn}`));
|
||||
}
|
||||
|
||||
console.log('\nDisk Space:');
|
||||
console.log(`- Required: ${result.spaceCheck.requiredFormatted}`);
|
||||
console.log(`- Available: ${result.spaceCheck.availableFormatted}`);
|
||||
console.log(`- Sufficient: ${result.spaceCheck.hasEnoughSpace ? '✅ Yes' : '❌ No'}`);
|
||||
|
||||
} else {
|
||||
console.log('\n=== RESTORE RESULTS ===\n');
|
||||
|
||||
console.log(`Status: ${result.success ? '✅ SUCCESS' : '❌ FAILED'}`);
|
||||
console.log(`Duration: ${result.duration}s`);
|
||||
|
||||
if (result.result) {
|
||||
console.log('\nItems Restored:');
|
||||
if (result.result.databaseRestored !== undefined) {
|
||||
console.log(`- Database: ${result.result.databaseRestored ? '✅' : '❌'}`);
|
||||
}
|
||||
if (result.result.filesRestored !== undefined) {
|
||||
console.log(`- Files: ${result.result.filesRestored}`);
|
||||
}
|
||||
if (result.result.errors && result.result.errors.length > 0) {
|
||||
console.log('- Errors:');
|
||||
result.result.errors.forEach(err => console.log(` ❌ ${err}`));
|
||||
}
|
||||
}
|
||||
|
||||
if (result.verification) {
|
||||
console.log('\nVerification:');
|
||||
console.log(`- Valid: ${result.verification.isValid ? '✅ Yes' : '❌ No'}`);
|
||||
if (result.verification.errors.length > 0) {
|
||||
console.log('- Errors:');
|
||||
result.verification.errors.forEach(err => console.log(` ❌ ${err}`));
|
||||
}
|
||||
}
|
||||
|
||||
if (result.preRestoreBackup) {
|
||||
console.log('\nSafety Backup:');
|
||||
console.log(`- Location: ${result.preRestoreBackup}`);
|
||||
console.log('- This backup can be used to rollback if needed');
|
||||
}
|
||||
}
|
||||
|
||||
// Show recent log entries
|
||||
console.log('\nRecent Log Entries:');
|
||||
result.logs.slice(-10).forEach(log => {
|
||||
const icon = log.level === 'error' ? '❌' : log.level === 'warn' ? '⚠️ ' : 'ℹ️ ';
|
||||
console.log(`${icon} [${log.timestamp}] ${log.message}`);
|
||||
});
|
||||
|
||||
// Clean up test files
|
||||
if (options.source && options.source.includes('test-backup')) {
|
||||
await fs.rmdir(path.dirname(options.source), { recursive: true }).catch(() => {});
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
console.error('\n❌ Restore operation failed:', error.message);
|
||||
|
||||
// Show logs if available
|
||||
if (restoreService.restoreLog && restoreService.restoreLog.length > 0) {
|
||||
console.log('\nError Log:');
|
||||
restoreService.restoreLog.slice(-10).forEach(log => {
|
||||
if (log.level === 'error' || log.level === 'warn') {
|
||||
console.log(`[${log.timestamp}] ${log.level.toUpperCase()}: ${log.message}`);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Cleanup
|
||||
await db.destroy();
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
// Show help if requested
|
||||
if (args.includes('--help') || args.includes('-h')) {
|
||||
console.log(`
|
||||
PicPeak Restore Service Test
|
||||
|
||||
This script tests the restore service functionality with safety checks.
|
||||
|
||||
Usage:
|
||||
node scripts/test-restore-service.js [options]
|
||||
|
||||
Options:
|
||||
--dry-run Perform validation only without actual restore
|
||||
--force Force restore even with warnings
|
||||
--type Restore type: full, database, files, selective (default: full)
|
||||
--source Backup source path or S3 URL
|
||||
--manifest Path to backup manifest
|
||||
--help Show this help message
|
||||
|
||||
Examples:
|
||||
# Dry run with automatic backup selection
|
||||
node scripts/test-restore-service.js --dry-run
|
||||
|
||||
# Full restore from specific backup
|
||||
node scripts/test-restore-service.js --source /backup/2024-01-20 --manifest /backup/2024-01-20/manifest.json
|
||||
|
||||
# Database-only restore with force
|
||||
node scripts/test-restore-service.js --type database --force --source /backup/2024-01-20 --manifest /backup/2024-01-20/manifest.json
|
||||
|
||||
# Restore from S3
|
||||
node scripts/test-restore-service.js --source s3://my-bucket/backups/2024-01-20 --manifest s3://my-bucket/backups/2024-01-20/manifest.json
|
||||
|
||||
Safety Features:
|
||||
- Pre-restore validation checks compatibility and warns about potential issues
|
||||
- Automatic pre-restore backup is created (unless skipped)
|
||||
- Post-restore verification ensures data integrity
|
||||
- Rollback capability if restore fails
|
||||
- Detailed logging of all operations
|
||||
`);
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
// Run the test
|
||||
testRestore();
|
||||
+73
-35
@@ -4,19 +4,27 @@ require('dotenv').config();
|
||||
const { validateEnvironment } = require('./src/config/validateEnv');
|
||||
validateEnvironment();
|
||||
|
||||
// Initialize logger early to capture startup logs
|
||||
const logger = require('./src/utils/logger');
|
||||
logger.info('Server starting up', {
|
||||
nodeVersion: process.version,
|
||||
environment: process.env.NODE_ENV || 'development',
|
||||
timestamp: new Date().toISOString()
|
||||
});
|
||||
|
||||
const express = require('express');
|
||||
const helmet = require('helmet');
|
||||
const cors = require('cors');
|
||||
const rateLimit = require('express-rate-limit');
|
||||
const jwt = require('jsonwebtoken');
|
||||
const path = require('path');
|
||||
const { initializeDatabase, db } = require('./src/database/db');
|
||||
const { startFileWatcher } = require('./src/services/fileWatcher');
|
||||
const { startExpirationChecker } = require('./src/services/expirationChecker');
|
||||
const { initializeTransporter, startEmailQueueProcessor } = require('./src/services/emailProcessor');
|
||||
const { startBackupService } = require('./src/services/backupService');
|
||||
const { startScheduledBackups } = require('./src/services/databaseBackup');
|
||||
const { maintenanceMiddleware } = require('./src/middleware/maintenance');
|
||||
const { sessionTimeoutMiddleware } = require('./src/middleware/sessionTimeout');
|
||||
const logger = require('./src/utils/logger');
|
||||
const { createRateLimiter, createAuthRateLimiter } = require('./src/services/rateLimitService');
|
||||
|
||||
// Import routes
|
||||
const authRoutes = require('./src/routes/auth-enhanced');
|
||||
@@ -93,37 +101,23 @@ const corsOptions = {
|
||||
|
||||
app.use(cors(corsOptions));
|
||||
|
||||
// Rate limiting with admin bypass
|
||||
const limiter = rateLimit({
|
||||
windowMs: 15 * 60 * 1000, // 15 minutes
|
||||
max: process.env.NODE_ENV === 'development' ? 1000 : 100, // More lenient in development
|
||||
skip: (req) => {
|
||||
// Skip rate limiting for authenticated admin users
|
||||
if (req.path.startsWith('/api/admin/') && req.headers.authorization) {
|
||||
const token = req.headers.authorization.replace('Bearer ', '');
|
||||
try {
|
||||
const decoded = jwt.verify(token, process.env.JWT_SECRET);
|
||||
return decoded.type === 'admin';
|
||||
} catch (err) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
// Also skip rate limiting for public settings endpoint in development
|
||||
if (process.env.NODE_ENV === 'development' && req.path === '/api/public/settings') {
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
});
|
||||
// Initialize rate limiters (they will be created dynamically)
|
||||
let generalRateLimiter;
|
||||
let authRateLimiter;
|
||||
|
||||
const authLimiter = rateLimit({
|
||||
windowMs: 15 * 60 * 1000,
|
||||
max: 5 // limit auth attempts
|
||||
});
|
||||
// Function to initialize rate limiters
|
||||
async function initializeRateLimiters() {
|
||||
generalRateLimiter = await createRateLimiter();
|
||||
authRateLimiter = await createAuthRateLimiter();
|
||||
|
||||
// Apply rate limiting
|
||||
app.use('/api/', generalRateLimiter);
|
||||
app.use('/api/auth', authRateLimiter);
|
||||
app.use('/api/gallery/:slug/verify', authRateLimiter);
|
||||
app.use('/api/admin/auth/login', authRateLimiter);
|
||||
}
|
||||
|
||||
// Apply rate limiting - admin routes check will skip for valid admin tokens
|
||||
app.use('/api/', limiter);
|
||||
app.use('/api/auth', authLimiter);
|
||||
// Note: Rate limiters will be initialized after database connection
|
||||
|
||||
// Body parsing middleware with increased limits for large uploads
|
||||
app.use(express.json({ limit: '100mb' }));
|
||||
@@ -158,6 +152,28 @@ app.use('/thumbnails', require('./src/middleware/photoAuth'), setCorsHeaders, se
|
||||
// Static file serving for uploads (public - logos, favicons)
|
||||
app.use('/uploads', setCorsHeaders, secureStatic(path.join(storagePath, 'uploads')));
|
||||
|
||||
// Debug endpoint to check IP detection (only in development)
|
||||
if (process.env.NODE_ENV === 'development') {
|
||||
app.get('/api/debug/ip', (req, res) => {
|
||||
const clientIp = req.headers['x-forwarded-for']?.split(',')[0]?.trim() ||
|
||||
req.headers['x-real-ip'] ||
|
||||
req.connection.remoteAddress ||
|
||||
req.ip;
|
||||
|
||||
res.json({
|
||||
detectedIp: clientIp,
|
||||
reqIp: req.ip,
|
||||
headers: {
|
||||
'x-forwarded-for': req.headers['x-forwarded-for'],
|
||||
'x-real-ip': req.headers['x-real-ip'],
|
||||
'x-forwarded-proto': req.headers['x-forwarded-proto'],
|
||||
'x-forwarded-host': req.headers['x-forwarded-host']
|
||||
},
|
||||
trustProxy: app.get('trust proxy')
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// Health check endpoint
|
||||
app.get('/health', async (req, res) => {
|
||||
try {
|
||||
@@ -187,6 +203,10 @@ app.use('/api/gallery', galleryRoutes);
|
||||
app.use('/api/admin', adminRoutes);
|
||||
app.use('/api/admin/auth', adminAuthRoutes);
|
||||
app.use('/api/admin/system', require('./src/routes/adminSystem'));
|
||||
app.use('/api/admin/backup', require('./src/routes/adminBackup'));
|
||||
app.use('/api/admin/database-backup', require('./src/routes/adminDatabaseBackup'));
|
||||
app.use('/api/admin/feedback', require('./src/routes/adminFeedback'));
|
||||
app.use('/api/gallery', require('./src/routes/galleryFeedback'));
|
||||
app.use('/api/public/settings', require('./src/routes/publicSettings'));
|
||||
app.use('/api/public', require('./src/routes/publicCMS'));
|
||||
app.use('/api/images', require('./src/routes/protectedImages'));
|
||||
@@ -203,9 +223,21 @@ async function startServer() {
|
||||
// Initialize database
|
||||
await initializeDatabase();
|
||||
|
||||
// Initialize auth security cleanup job
|
||||
const { initializeCleanupJob } = require('./src/utils/authSecurity');
|
||||
initializeCleanupJob();
|
||||
// Initialize rate limiters after database is ready
|
||||
await initializeRateLimiters();
|
||||
logger.info('Rate limiters initialized with database configuration');
|
||||
|
||||
// Initialize auth security cleanup job
|
||||
const { initializeCleanupJob } = require('./src/utils/authSecurity');
|
||||
initializeCleanupJob();
|
||||
|
||||
// Initialize temp upload cleanup job
|
||||
const { cleanupTempUploads } = require('./src/utils/cleanupTempUploads');
|
||||
// Run cleanup on startup
|
||||
cleanupTempUploads();
|
||||
// Schedule periodic cleanup every hour
|
||||
setInterval(cleanupTempUploads, 60 * 60 * 1000);
|
||||
logger.info('Temp upload cleanup scheduled');
|
||||
|
||||
// Start file watcher
|
||||
startFileWatcher();
|
||||
@@ -217,6 +249,12 @@ async function startServer() {
|
||||
await initializeTransporter();
|
||||
startEmailQueueProcessor();
|
||||
|
||||
// Start backup service
|
||||
await startBackupService();
|
||||
|
||||
// Start database backup service
|
||||
await startScheduledBackups();
|
||||
|
||||
app.listen(PORT, () => {
|
||||
logger.info(`Server running on port ${PORT}`);
|
||||
logger.info(`Admin interface: ${process.env.ADMIN_URL || 'http://localhost:3000'}`);
|
||||
|
||||
@@ -34,7 +34,7 @@ function validateEnvironment() {
|
||||
if (name === 'JWT_SECRET' && value) {
|
||||
// Check for the insecure default value
|
||||
if (value === 'your-secret-key') {
|
||||
errors.push(`CRITICAL: JWT_SECRET is set to the insecure default value. Please set a secure secret key.`);
|
||||
errors.push('CRITICAL: JWT_SECRET is set to the insecure default value. Please set a secure secret key.');
|
||||
}
|
||||
|
||||
// Check minimum length (should be at least 32 characters for security)
|
||||
|
||||
@@ -4,6 +4,33 @@ const knexConfig = require('../../knexfile');
|
||||
// Create database connection with built-in retry logic
|
||||
const db = knex(knexConfig);
|
||||
|
||||
// Connection retry configuration
|
||||
const MAX_RETRIES = 3;
|
||||
const RETRY_DELAY = 1000;
|
||||
|
||||
// Wrapper function to handle connection retries
|
||||
async function withRetry(queryFn, retries = MAX_RETRIES) {
|
||||
for (let i = 0; i < retries; i++) {
|
||||
try {
|
||||
return await queryFn();
|
||||
} catch (error) {
|
||||
const isConnectionError = error.message && (
|
||||
error.message.includes('Connection terminated unexpectedly') ||
|
||||
error.message.includes('Connection ended unexpectedly') ||
|
||||
error.message.includes('ECONNREFUSED') ||
|
||||
error.message.includes('ETIMEDOUT')
|
||||
);
|
||||
|
||||
if (isConnectionError && i < retries - 1) {
|
||||
console.log(`Database connection error, retrying in ${RETRY_DELAY}ms... (attempt ${i + 1}/${retries})`);
|
||||
await new Promise(resolve => setTimeout(resolve, RETRY_DELAY * (i + 1)));
|
||||
continue;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function initializeDatabase() {
|
||||
// Events table
|
||||
const hasEventsTable = await db.schema.hasTable('events');
|
||||
@@ -59,9 +86,9 @@ async function initializeDatabase() {
|
||||
)
|
||||
`);
|
||||
|
||||
await db.raw(`INSERT INTO events_new SELECT * FROM events`);
|
||||
await db.raw(`DROP TABLE events`);
|
||||
await db.raw(`ALTER TABLE events_new RENAME TO events`);
|
||||
await db.raw('INSERT INTO events_new SELECT * FROM events');
|
||||
await db.raw('DROP TABLE events');
|
||||
await db.raw('ALTER TABLE events_new RENAME TO events');
|
||||
} catch (error) {
|
||||
// If the migration fails, it might already have been applied
|
||||
console.log('Color theme migration may have already been applied');
|
||||
@@ -225,4 +252,4 @@ async function logActivity(activityType, metadata = {}, eventId = null, actor =
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { db, initializeDatabase, logActivity };
|
||||
module.exports = { db, initializeDatabase, logActivity, withRetry };
|
||||
@@ -1,24 +1,83 @@
|
||||
const jwt = require('jsonwebtoken');
|
||||
const { db } = require('../database/db');
|
||||
const { formatBoolean } = require('../utils/dbCompat');
|
||||
const logger = require('../utils/logger');
|
||||
|
||||
async function adminAuth(req, res, next) {
|
||||
try {
|
||||
const token = req.headers.authorization?.split(' ')[1];
|
||||
if (!token) {
|
||||
const clientIp = req.headers['x-forwarded-for']?.split(',')[0]?.trim() ||
|
||||
req.headers['x-real-ip'] ||
|
||||
req.connection.remoteAddress ||
|
||||
req.ip;
|
||||
logger.warn('Admin auth attempt without token', {
|
||||
ip: clientIp,
|
||||
path: req.path,
|
||||
method: req.method,
|
||||
userAgent: req.headers['user-agent']
|
||||
});
|
||||
return res.status(401).json({ error: 'No token provided' });
|
||||
}
|
||||
|
||||
const decoded = jwt.verify(token, process.env.JWT_SECRET);
|
||||
let decoded;
|
||||
try {
|
||||
decoded = jwt.verify(token, process.env.JWT_SECRET);
|
||||
} catch (jwtError) {
|
||||
const clientIp = req.headers['x-forwarded-for']?.split(',')[0]?.trim() ||
|
||||
req.headers['x-real-ip'] ||
|
||||
req.connection.remoteAddress ||
|
||||
req.ip;
|
||||
|
||||
logger.warn('JWT validation failed', {
|
||||
ip: clientIp,
|
||||
path: req.path,
|
||||
method: req.method,
|
||||
userAgent: req.headers['user-agent'],
|
||||
error: jwtError.name,
|
||||
message: jwtError.message,
|
||||
timestamp: new Date().toISOString()
|
||||
});
|
||||
|
||||
if (jwtError.name === 'TokenExpiredError') {
|
||||
return res.status(401).json({ error: 'Token expired' });
|
||||
}
|
||||
return res.status(401).json({ error: 'Invalid token' });
|
||||
}
|
||||
|
||||
const admin = await db('admin_users').where({ id: decoded.id, is_active: formatBoolean(true) }).first();
|
||||
|
||||
if (!admin) {
|
||||
const clientIp = req.headers['x-forwarded-for']?.split(',')[0]?.trim() ||
|
||||
req.headers['x-real-ip'] ||
|
||||
req.connection.remoteAddress ||
|
||||
req.ip;
|
||||
|
||||
logger.warn('Admin auth failed - user not found or inactive', {
|
||||
ip: clientIp,
|
||||
userId: decoded.id,
|
||||
path: req.path,
|
||||
method: req.method,
|
||||
timestamp: new Date().toISOString()
|
||||
});
|
||||
return res.status(401).json({ error: 'Invalid token' });
|
||||
}
|
||||
|
||||
req.admin = admin;
|
||||
next();
|
||||
} catch (error) {
|
||||
const clientIp = req.headers['x-forwarded-for']?.split(',')[0]?.trim() ||
|
||||
req.headers['x-real-ip'] ||
|
||||
req.connection.remoteAddress ||
|
||||
req.ip;
|
||||
|
||||
logger.error('Admin auth middleware error', {
|
||||
ip: clientIp,
|
||||
path: req.path,
|
||||
error: error.message,
|
||||
stack: error.stack,
|
||||
timestamp: new Date().toISOString()
|
||||
});
|
||||
res.status(401).json({ error: 'Invalid token' });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,236 @@
|
||||
const crypto = require('crypto');
|
||||
const { db } = require('../database/db');
|
||||
const logger = require('../utils/logger');
|
||||
|
||||
/**
|
||||
* Generate a unique identifier for the guest
|
||||
*/
|
||||
function generateGuestIdentifier(req) {
|
||||
const ip = req.ip || req.connection.remoteAddress || 'unknown';
|
||||
const userAgent = req.headers['user-agent'] || 'unknown';
|
||||
return crypto
|
||||
.createHash('sha256')
|
||||
.update(`${ip}:${userAgent}`)
|
||||
.digest('hex');
|
||||
}
|
||||
|
||||
/**
|
||||
* Get rate limit settings from app_settings
|
||||
*/
|
||||
async function getRateLimitSettings() {
|
||||
try {
|
||||
const settings = await db('app_settings')
|
||||
.where('setting_key', 'feedback_rate_limits')
|
||||
.first();
|
||||
|
||||
if (settings && settings.setting_value) {
|
||||
return JSON.parse(settings.setting_value);
|
||||
}
|
||||
|
||||
// Default settings
|
||||
return {
|
||||
rating: { max: 100, window: 3600 }, // 100 ratings per hour
|
||||
comment: { max: 20, window: 3600 }, // 20 comments per hour
|
||||
like: { max: 200, window: 3600 }, // 200 likes per hour
|
||||
favorite: { max: 100, window: 3600 } // 100 favorites per hour
|
||||
};
|
||||
} catch (error) {
|
||||
logger.error('Error getting rate limit settings:', error);
|
||||
// Return defaults on error
|
||||
return {
|
||||
rating: { max: 100, window: 3600 },
|
||||
comment: { max: 20, window: 3600 },
|
||||
like: { max: 200, window: 3600 },
|
||||
favorite: { max: 100, window: 3600 }
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if action is rate limited
|
||||
*/
|
||||
async function checkRateLimit(identifier, eventId, actionType) {
|
||||
try {
|
||||
const settings = await getRateLimitSettings();
|
||||
const limit = settings[actionType] || { max: 100, window: 3600 };
|
||||
|
||||
// Clean old entries (older than window)
|
||||
const cutoff = new Date(Date.now() - limit.window * 1000);
|
||||
await db('feedback_rate_limits')
|
||||
.where('window_start', '<', cutoff)
|
||||
.delete();
|
||||
|
||||
// Count recent actions
|
||||
const recentActions = await db('feedback_rate_limits')
|
||||
.where({
|
||||
identifier,
|
||||
event_id: eventId,
|
||||
action_type: actionType
|
||||
})
|
||||
.where('window_start', '>', cutoff)
|
||||
.sum('action_count as total')
|
||||
.first();
|
||||
|
||||
const currentCount = recentActions?.total || 0;
|
||||
|
||||
if (currentCount >= limit.max) {
|
||||
return {
|
||||
limited: true,
|
||||
limit: limit.max,
|
||||
window: limit.window,
|
||||
current: currentCount,
|
||||
resetAt: new Date(Date.now() + limit.window * 1000)
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
limited: false,
|
||||
limit: limit.max,
|
||||
window: limit.window,
|
||||
current: currentCount,
|
||||
remaining: limit.max - currentCount
|
||||
};
|
||||
} catch (error) {
|
||||
logger.error('Error checking rate limit:', error);
|
||||
// Allow action on error to avoid blocking legitimate users
|
||||
return { limited: false };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Record an action for rate limiting
|
||||
*/
|
||||
async function recordAction(identifier, eventId, actionType) {
|
||||
try {
|
||||
await db('feedback_rate_limits').insert({
|
||||
identifier,
|
||||
event_id: eventId,
|
||||
action_type: actionType,
|
||||
action_count: 1,
|
||||
window_start: new Date()
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Error recording rate limit action:', error);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Middleware factory for feedback rate limiting
|
||||
*/
|
||||
function feedbackRateLimit(actionType) {
|
||||
return async (req, res, next) => {
|
||||
try {
|
||||
// Extract event ID from params or body
|
||||
const eventId = req.params.eventId || req.body?.event_id;
|
||||
if (!eventId) {
|
||||
return res.status(400).json({ error: 'Event ID required' });
|
||||
}
|
||||
|
||||
// Generate guest identifier
|
||||
const identifier = generateGuestIdentifier(req);
|
||||
req.guestIdentifier = identifier;
|
||||
|
||||
// Check rate limit
|
||||
const rateLimitStatus = await checkRateLimit(identifier, eventId, actionType);
|
||||
|
||||
// Set rate limit headers
|
||||
res.set({
|
||||
'X-RateLimit-Limit': rateLimitStatus.limit,
|
||||
'X-RateLimit-Remaining': rateLimitStatus.remaining || 0,
|
||||
'X-RateLimit-Reset': rateLimitStatus.resetAt ? rateLimitStatus.resetAt.toISOString() : new Date().toISOString()
|
||||
});
|
||||
|
||||
if (rateLimitStatus.limited) {
|
||||
logger.warn(`Rate limit exceeded for ${actionType}`, {
|
||||
identifier: identifier.substring(0, 16) + '...',
|
||||
eventId,
|
||||
actionType
|
||||
});
|
||||
|
||||
return res.status(429).json({
|
||||
error: 'Too many requests',
|
||||
message: `Rate limit exceeded. Please try again later.`,
|
||||
retryAfter: rateLimitStatus.window
|
||||
});
|
||||
}
|
||||
|
||||
// Record the action after successful processing
|
||||
res.on('finish', async () => {
|
||||
if (res.statusCode >= 200 && res.statusCode < 300) {
|
||||
await recordAction(identifier, eventId, actionType);
|
||||
}
|
||||
});
|
||||
|
||||
next();
|
||||
} catch (error) {
|
||||
logger.error('Error in rate limit middleware:', error);
|
||||
// Allow request to proceed on error
|
||||
next();
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* IP-based rate limiting for more strict control
|
||||
*/
|
||||
function strictRateLimit(options = {}) {
|
||||
const {
|
||||
windowMs = 15 * 60 * 1000, // 15 minutes
|
||||
max = 100, // limit each IP to 100 requests per windowMs
|
||||
message = 'Too many requests from this IP, please try again later.',
|
||||
skipSuccessfulRequests = false
|
||||
} = options;
|
||||
|
||||
const store = new Map();
|
||||
|
||||
// Clean up old entries periodically
|
||||
setInterval(() => {
|
||||
const now = Date.now();
|
||||
for (const [key, data] of store.entries()) {
|
||||
if (data.resetTime < now) {
|
||||
store.delete(key);
|
||||
}
|
||||
}
|
||||
}, windowMs);
|
||||
|
||||
return (req, res, next) => {
|
||||
const ip = req.ip || req.connection.remoteAddress;
|
||||
const now = Date.now();
|
||||
const resetTime = now + windowMs;
|
||||
|
||||
let data = store.get(ip);
|
||||
if (!data || data.resetTime < now) {
|
||||
data = {
|
||||
count: 0,
|
||||
resetTime
|
||||
};
|
||||
store.set(ip, data);
|
||||
}
|
||||
|
||||
if (data.count >= max) {
|
||||
return res.status(429).json({
|
||||
error: 'Too many requests',
|
||||
message,
|
||||
retryAfter: Math.ceil((data.resetTime - now) / 1000)
|
||||
});
|
||||
}
|
||||
|
||||
if (!skipSuccessfulRequests || res.statusCode >= 400) {
|
||||
data.count++;
|
||||
}
|
||||
|
||||
res.setHeader('X-RateLimit-Limit', max);
|
||||
res.setHeader('X-RateLimit-Remaining', Math.max(0, max - data.count));
|
||||
res.setHeader('X-RateLimit-Reset', new Date(data.resetTime).toISOString());
|
||||
|
||||
next();
|
||||
};
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
feedbackRateLimit,
|
||||
strictRateLimit,
|
||||
generateGuestIdentifier,
|
||||
checkRateLimit,
|
||||
recordAction
|
||||
};
|
||||
@@ -1,5 +1,5 @@
|
||||
const jwt = require('jsonwebtoken');
|
||||
const { db } = require('../database/db');
|
||||
const { db, withRetry } = require('../database/db');
|
||||
const { formatBoolean } = require('../utils/dbCompat');
|
||||
|
||||
// Middleware to verify gallery access
|
||||
@@ -11,13 +11,15 @@ async function verifyGalleryAccess(req, res, next) {
|
||||
}
|
||||
|
||||
const decoded = jwt.verify(token, process.env.JWT_SECRET);
|
||||
const event = await db('events')
|
||||
.where({
|
||||
id: decoded.eventId,
|
||||
is_active: formatBoolean(true),
|
||||
is_archived: formatBoolean(false)
|
||||
})
|
||||
.first();
|
||||
const event = await withRetry(async () => {
|
||||
return await db('events')
|
||||
.where({
|
||||
id: decoded.eventId,
|
||||
is_active: formatBoolean(true),
|
||||
is_archived: formatBoolean(false)
|
||||
})
|
||||
.first();
|
||||
});
|
||||
|
||||
if (!event) {
|
||||
return res.status(404).json({ error: 'Gallery not found or expired' });
|
||||
|
||||
@@ -10,7 +10,7 @@ const DEFAULT_SESSION_TIMEOUT = 60 * 60 * 1000;
|
||||
// Cache for session timeout setting
|
||||
let cachedTimeout = null;
|
||||
let cacheExpiry = 0;
|
||||
const CACHE_DURATION = 5 * 60 * 1000; // 5 minutes
|
||||
const CACHE_DURATION = 30 * 60 * 1000; // 30 minutes - reduced DB queries
|
||||
|
||||
// Clean up expired sessions every 5 minutes
|
||||
setInterval(() => {
|
||||
|
||||
@@ -0,0 +1,111 @@
|
||||
const fs = require('fs').promises;
|
||||
const path = require('path');
|
||||
const sharp = require('sharp');
|
||||
const logger = require('../utils/logger');
|
||||
|
||||
/**
|
||||
* Validate uploaded file is complete and not corrupted
|
||||
*/
|
||||
async function validateUploadedFile(filePath) {
|
||||
try {
|
||||
// Check file exists and has size
|
||||
const stats = await fs.stat(filePath);
|
||||
if (stats.size === 0) {
|
||||
throw new Error('File is empty');
|
||||
}
|
||||
|
||||
// For image files, verify they can be read by Sharp
|
||||
const ext = path.extname(filePath).toLowerCase();
|
||||
const imageExtensions = ['.jpg', '.jpeg', '.png', '.gif', '.webp'];
|
||||
|
||||
if (imageExtensions.includes(ext)) {
|
||||
// Try to read metadata - this will fail if image is corrupted
|
||||
let metadata;
|
||||
try {
|
||||
metadata = await sharp(filePath, {
|
||||
failOnError: false, // Don't fail on recoverable errors
|
||||
limitInputPixels: 268402689 // ~16k x 16k max
|
||||
}).metadata();
|
||||
} catch (metadataError) {
|
||||
// If metadata reading fails, the file is likely incomplete
|
||||
throw new Error(`Invalid image file: ${metadataError.message}`);
|
||||
}
|
||||
|
||||
if (!metadata || !metadata.width || !metadata.height) {
|
||||
throw new Error('Invalid image dimensions - file may be incomplete');
|
||||
}
|
||||
|
||||
// Check for reasonable dimensions
|
||||
if (metadata.width < 10 || metadata.height < 10) {
|
||||
throw new Error('Image dimensions too small');
|
||||
}
|
||||
|
||||
// Additional check: verify we can actually decode a small portion of the image
|
||||
try {
|
||||
await sharp(filePath, {
|
||||
failOnError: false,
|
||||
limitInputPixels: 268402689
|
||||
})
|
||||
.resize(10, 10) // Try to resize to very small size
|
||||
.toBuffer();
|
||||
} catch (decodeError) {
|
||||
throw new Error(`Image decode failed - file may be corrupted: ${decodeError.message}`);
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
return true;
|
||||
} catch (error) {
|
||||
logger.error(`File validation failed for ${filePath}:`, error.message);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Middleware to validate uploaded files after multer processing
|
||||
*/
|
||||
async function validateUploadedFiles(req, res, next) {
|
||||
if (!req.files || req.files.length === 0) {
|
||||
return next();
|
||||
}
|
||||
|
||||
const validFiles = [];
|
||||
const invalidFiles = [];
|
||||
|
||||
// Validate each file
|
||||
for (const file of req.files) {
|
||||
try {
|
||||
await validateUploadedFile(file.path);
|
||||
validFiles.push(file);
|
||||
} catch (error) {
|
||||
logger.warn(`Removing invalid upload ${file.originalname}: ${error.message}`);
|
||||
invalidFiles.push({
|
||||
filename: file.originalname,
|
||||
error: error.message
|
||||
});
|
||||
|
||||
// Delete the invalid file
|
||||
try {
|
||||
await fs.unlink(file.path);
|
||||
} catch (unlinkErr) {
|
||||
logger.error(`Failed to delete invalid file ${file.path}:`, unlinkErr.message);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Update req.files to only include valid files
|
||||
req.files = validFiles;
|
||||
|
||||
// Store invalid files info for response
|
||||
if (invalidFiles.length > 0) {
|
||||
req.invalidFiles = invalidFiles;
|
||||
}
|
||||
|
||||
next();
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
validateUploadedFile,
|
||||
validateUploadedFiles
|
||||
};
|
||||
@@ -11,6 +11,8 @@ const photosRoutes = require('./adminPhotos');
|
||||
const categoriesRoutes = require('./adminCategories');
|
||||
const cmsRoutes = require('./adminCMS');
|
||||
const notificationsRoutes = require('./adminNotifications');
|
||||
const backupRoutes = require('./adminBackup');
|
||||
const restoreRoutes = require('./adminRestore');
|
||||
|
||||
// Mount sub-routers
|
||||
router.use('/dashboard', dashboardRoutes);
|
||||
@@ -22,5 +24,7 @@ router.use('/events', photosRoutes);
|
||||
router.use('/categories', categoriesRoutes);
|
||||
router.use('/cms', cmsRoutes);
|
||||
router.use('/notifications', notificationsRoutes);
|
||||
router.use('/backup', backupRoutes);
|
||||
router.use('/restore', restoreRoutes);
|
||||
|
||||
module.exports = router;
|
||||
@@ -251,7 +251,7 @@ router.post('/:id/restore', adminAuth, async (req, res) => {
|
||||
} catch (statError) {
|
||||
console.error(`Failed to stat file: ${actualFilePath}`);
|
||||
console.error(`Entry name was: ${entry.entryName}`);
|
||||
console.error(`Error:`, statError.message);
|
||||
console.error('Error:', statError.message);
|
||||
// Skip this file if we can't stat it
|
||||
continue;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,956 @@
|
||||
const express = require('express');
|
||||
const { db } = require('../database/db');
|
||||
const { adminAuth } = require('../middleware/auth');
|
||||
const { triggerManualBackup, getBackupStatus, cleanupOldBackupRuns, getBackupManifest, validateBackupManifest } = require('../services/backupService');
|
||||
const logger = require('../utils/logger');
|
||||
const fs = require('fs').promises;
|
||||
const path = require('path');
|
||||
const crypto = require('crypto');
|
||||
const archiver = require('archiver');
|
||||
const S3StorageAdapter = require('../services/storage/s3Storage');
|
||||
|
||||
const router = express.Router();
|
||||
|
||||
// Get backup configuration
|
||||
router.get('/config', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const settings = await db('app_settings')
|
||||
.where('setting_type', 'backup')
|
||||
.select('setting_key', 'setting_value');
|
||||
|
||||
const config = {};
|
||||
settings.forEach(setting => {
|
||||
try {
|
||||
config[setting.setting_key] = JSON.parse(setting.setting_value);
|
||||
} catch (e) {
|
||||
config[setting.setting_key] = setting.setting_value;
|
||||
}
|
||||
});
|
||||
|
||||
res.json(config);
|
||||
} catch (error) {
|
||||
logger.error('Failed to get backup configuration:', error);
|
||||
res.status(500).json({ error: 'Failed to get backup configuration' });
|
||||
}
|
||||
});
|
||||
|
||||
// Update backup configuration
|
||||
router.put('/config', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const updates = req.body;
|
||||
|
||||
// Validate required fields based on destination type
|
||||
if (updates.backup_destination_type) {
|
||||
switch (updates.backup_destination_type) {
|
||||
case 'local':
|
||||
if (!updates.backup_destination_path) {
|
||||
return res.status(400).json({ error: 'Local backup requires destination path' });
|
||||
}
|
||||
break;
|
||||
case 'rsync':
|
||||
if (!updates.backup_rsync_host || !updates.backup_rsync_path) {
|
||||
return res.status(400).json({ error: 'Rsync backup requires host and path' });
|
||||
}
|
||||
break;
|
||||
case 's3':
|
||||
if (!updates.backup_s3_endpoint || !updates.backup_s3_bucket ||
|
||||
!updates.backup_s3_access_key || !updates.backup_s3_secret_key) {
|
||||
return res.status(400).json({ error: 'S3 backup requires endpoint, bucket, and credentials' });
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
// Update settings
|
||||
for (const [key, value] of Object.entries(updates)) {
|
||||
if (key.startsWith('backup_')) {
|
||||
await db('app_settings')
|
||||
.insert({
|
||||
setting_key: key,
|
||||
setting_value: JSON.stringify(value),
|
||||
setting_type: 'backup',
|
||||
updated_at: new Date()
|
||||
})
|
||||
.onConflict('setting_key')
|
||||
.merge({
|
||||
setting_value: JSON.stringify(value),
|
||||
updated_at: new Date()
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Restart backup service if enabled status changed
|
||||
if ('backup_enabled' in updates) {
|
||||
const { startBackupService, stopBackupService } = require('../services/backupService');
|
||||
if (updates.backup_enabled) {
|
||||
await startBackupService();
|
||||
} else {
|
||||
stopBackupService();
|
||||
}
|
||||
}
|
||||
|
||||
res.json({ success: true, message: 'Backup configuration updated' });
|
||||
} catch (error) {
|
||||
logger.error('Failed to update backup configuration:', error);
|
||||
res.status(500).json({ error: 'Failed to update backup configuration' });
|
||||
}
|
||||
});
|
||||
|
||||
// Get backup status and history
|
||||
router.get('/status', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const limit = parseInt(req.query.limit) || 10;
|
||||
const status = await getBackupStatus(limit);
|
||||
|
||||
res.json(status);
|
||||
} catch (error) {
|
||||
logger.error('Failed to get backup status:', error);
|
||||
res.status(500).json({ error: 'Failed to get backup status' });
|
||||
}
|
||||
});
|
||||
|
||||
// Trigger manual backup
|
||||
router.post('/run', adminAuth, async (req, res) => {
|
||||
try {
|
||||
// Check if backup is already running
|
||||
const status = await getBackupStatus();
|
||||
if (status.isRunning) {
|
||||
return res.status(409).json({ error: 'Backup is already running' });
|
||||
}
|
||||
|
||||
// Start backup in background
|
||||
triggerManualBackup().catch(error => {
|
||||
logger.error('Manual backup failed:', error);
|
||||
});
|
||||
|
||||
res.json({ success: true, message: 'Backup started' });
|
||||
} catch (error) {
|
||||
logger.error('Failed to trigger manual backup:', error);
|
||||
res.status(500).json({ error: 'Failed to trigger backup' });
|
||||
}
|
||||
});
|
||||
|
||||
// Get backup run details
|
||||
router.get('/runs/:id', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const { id } = req.params;
|
||||
|
||||
const run = await db('backup_runs')
|
||||
.where('id', id)
|
||||
.first();
|
||||
|
||||
if (!run) {
|
||||
return res.status(404).json({ error: 'Backup run not found' });
|
||||
}
|
||||
|
||||
// Parse JSON fields
|
||||
if (run.statistics) {
|
||||
try {
|
||||
run.statistics = JSON.parse(run.statistics);
|
||||
} catch (e) {
|
||||
// Keep as string if parsing fails
|
||||
}
|
||||
}
|
||||
|
||||
res.json(run);
|
||||
} catch (error) {
|
||||
logger.error('Failed to get backup run details:', error);
|
||||
res.status(500).json({ error: 'Failed to get backup run details' });
|
||||
}
|
||||
});
|
||||
|
||||
// Get file states (for debugging/monitoring)
|
||||
router.get('/files', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const { page = 1, limit = 50, search = '' } = req.query;
|
||||
const offset = (page - 1) * limit;
|
||||
|
||||
let query = db('backup_file_states');
|
||||
|
||||
if (search) {
|
||||
query = query.where('file_path', 'like', `%${search}%`);
|
||||
}
|
||||
|
||||
const [files, totalCount] = await Promise.all([
|
||||
query
|
||||
.orderBy('last_backed_up', 'desc')
|
||||
.limit(limit)
|
||||
.offset(offset),
|
||||
db('backup_file_states').count('* as count').first()
|
||||
]);
|
||||
|
||||
res.json({
|
||||
files,
|
||||
pagination: {
|
||||
page: parseInt(page),
|
||||
limit: parseInt(limit),
|
||||
total: totalCount.count,
|
||||
pages: Math.ceil(totalCount.count / limit)
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to get backup file states:', error);
|
||||
res.status(500).json({ error: 'Failed to get file states' });
|
||||
}
|
||||
});
|
||||
|
||||
// Clean up old backup runs
|
||||
router.delete('/cleanup', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const { days = 30 } = req.body;
|
||||
|
||||
await cleanupOldBackupRuns(days);
|
||||
|
||||
res.json({ success: true, message: `Cleaned up backup runs older than ${days} days` });
|
||||
} catch (error) {
|
||||
logger.error('Failed to cleanup old backup runs:', error);
|
||||
res.status(500).json({ error: 'Failed to cleanup backup runs' });
|
||||
}
|
||||
});
|
||||
|
||||
// Test backup destination connectivity
|
||||
router.post('/test-connection', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const { destination_type, ...config } = req.body;
|
||||
|
||||
switch (destination_type) {
|
||||
case 'local':
|
||||
// Test local path access
|
||||
const fs = require('fs').promises;
|
||||
try {
|
||||
await fs.access(config.path, fs.constants.W_OK);
|
||||
res.json({ success: true, message: 'Local path is writable' });
|
||||
} catch (error) {
|
||||
res.json({ success: false, message: 'Cannot write to local path: ' + error.message });
|
||||
}
|
||||
break;
|
||||
|
||||
case 'rsync':
|
||||
// Test rsync connection
|
||||
const { exec } = require('child_process');
|
||||
const { promisify } = require('util');
|
||||
const execAsync = promisify(exec);
|
||||
|
||||
const sshCommand = config.ssh_key
|
||||
? `ssh -i ${config.ssh_key} -o StrictHostKeyChecking=no -o ConnectTimeout=10`
|
||||
: 'ssh -o StrictHostKeyChecking=no -o ConnectTimeout=10';
|
||||
|
||||
const testCommand = config.user
|
||||
? `${sshCommand} ${config.user}@${config.host} "echo 'Connection successful'"`
|
||||
: `${sshCommand} ${config.host} "echo 'Connection successful'"`;
|
||||
|
||||
try {
|
||||
const { stdout } = await execAsync(testCommand);
|
||||
res.json({ success: true, message: 'Rsync connection successful' });
|
||||
} catch (error) {
|
||||
res.json({ success: false, message: 'Rsync connection failed: ' + error.message });
|
||||
}
|
||||
break;
|
||||
|
||||
case 's3':
|
||||
// Test S3 connection (would need AWS SDK)
|
||||
res.json({ success: false, message: 'S3 testing not implemented yet' });
|
||||
break;
|
||||
|
||||
default:
|
||||
res.status(400).json({ error: 'Invalid destination type' });
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error('Failed to test backup connection:', error);
|
||||
res.status(500).json({ error: 'Failed to test connection' });
|
||||
}
|
||||
});
|
||||
|
||||
// Get backup manifest for a specific backup run
|
||||
router.get('/manifest/:backupRunId', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const { backupRunId } = req.params;
|
||||
const result = await getBackupManifest(backupRunId);
|
||||
|
||||
res.json({
|
||||
backupRunId,
|
||||
manifest: result.manifest,
|
||||
summary: result.summary
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to get backup manifest:', error);
|
||||
res.status(404).json({ error: error.message || 'Backup manifest not found' });
|
||||
}
|
||||
});
|
||||
|
||||
// Validate a backup manifest
|
||||
router.post('/manifest/validate', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const { manifestPath } = req.body;
|
||||
|
||||
if (!manifestPath) {
|
||||
return res.status(400).json({ error: 'manifestPath is required' });
|
||||
}
|
||||
|
||||
const result = await validateBackupManifest(manifestPath);
|
||||
|
||||
res.json({
|
||||
valid: result.valid,
|
||||
error: result.error,
|
||||
manifestPath
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to validate manifest:', error);
|
||||
res.status(500).json({ error: 'Failed to validate manifest' });
|
||||
}
|
||||
});
|
||||
|
||||
// Download backup manifest
|
||||
router.get('/manifest/:backupRunId/download', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const { backupRunId } = req.params;
|
||||
const { format = 'json' } = req.query;
|
||||
|
||||
const result = await getBackupManifest(backupRunId);
|
||||
|
||||
// Set appropriate headers
|
||||
const filename = `backup-manifest-${backupRunId}.${format}`;
|
||||
res.setHeader('Content-Type', format === 'yaml' ? 'text/yaml' : 'application/json');
|
||||
res.setHeader('Content-Disposition', `attachment; filename="${filename}"`);
|
||||
|
||||
// Send the manifest in requested format
|
||||
if (format === 'yaml') {
|
||||
const yaml = require('js-yaml');
|
||||
res.send(yaml.dump(result.manifest, {
|
||||
indent: 2,
|
||||
lineWidth: -1,
|
||||
noRefs: true,
|
||||
sortKeys: true
|
||||
}));
|
||||
} else {
|
||||
res.json(result.manifest);
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error('Failed to download backup manifest:', error);
|
||||
res.status(404).json({ error: error.message || 'Backup manifest not found' });
|
||||
}
|
||||
});
|
||||
|
||||
// Get manifest for specific backup
|
||||
router.get('/manifests/:backupId', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const { backupId } = req.params;
|
||||
const result = await getBackupManifest(backupId);
|
||||
|
||||
res.json({
|
||||
backupId,
|
||||
manifest: result.manifest,
|
||||
summary: result.summary
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to get backup manifest:', error);
|
||||
res.status(404).json({ error: error.message || 'Backup manifest not found' });
|
||||
}
|
||||
});
|
||||
|
||||
// Download manifest file
|
||||
router.get('/manifests/:backupId/download', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const { backupId } = req.params;
|
||||
const { format = 'json' } = req.query;
|
||||
|
||||
const result = await getBackupManifest(backupId);
|
||||
|
||||
// Set appropriate headers
|
||||
const filename = `backup-manifest-${backupId}.${format}`;
|
||||
res.setHeader('Content-Type', format === 'yaml' ? 'text/yaml' : 'application/json');
|
||||
res.setHeader('Content-Disposition', `attachment; filename="${filename}"`);
|
||||
|
||||
// Send the manifest in requested format
|
||||
if (format === 'yaml') {
|
||||
const yaml = require('js-yaml');
|
||||
res.send(yaml.dump(result.manifest, {
|
||||
indent: 2,
|
||||
lineWidth: -1,
|
||||
noRefs: true,
|
||||
sortKeys: true
|
||||
}));
|
||||
} else {
|
||||
res.json(result.manifest);
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error('Failed to download backup manifest:', error);
|
||||
res.status(404).json({ error: error.message || 'Backup manifest not found' });
|
||||
}
|
||||
});
|
||||
|
||||
// Validate a manifest
|
||||
router.post('/manifests/validate', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const { manifestPath, manifestData } = req.body;
|
||||
|
||||
if (!manifestPath && !manifestData) {
|
||||
return res.status(400).json({ error: 'Either manifestPath or manifestData is required' });
|
||||
}
|
||||
|
||||
if (manifestData) {
|
||||
// Validate provided manifest data directly
|
||||
const validationResult = await validateManifestData(manifestData);
|
||||
return res.json(validationResult);
|
||||
}
|
||||
|
||||
// Use existing validation function for path
|
||||
const result = await validateBackupManifest(manifestPath);
|
||||
|
||||
res.json({
|
||||
valid: result.valid,
|
||||
error: result.error,
|
||||
manifestPath
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to validate manifest:', error);
|
||||
res.status(500).json({ error: 'Failed to validate manifest' });
|
||||
}
|
||||
});
|
||||
|
||||
// List S3 buckets
|
||||
router.get('/s3/buckets', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const config = await getBackupConfig();
|
||||
|
||||
if (config.backup_destination_type !== 's3') {
|
||||
return res.status(400).json({ error: 'S3 backup not configured' });
|
||||
}
|
||||
|
||||
const s3Adapter = new S3StorageAdapter({
|
||||
endpoint: config.backup_s3_endpoint,
|
||||
bucket: config.backup_s3_bucket,
|
||||
accessKeyId: config.backup_s3_access_key,
|
||||
secretAccessKey: config.backup_s3_secret_key,
|
||||
region: config.backup_s3_region || 'us-east-1',
|
||||
forcePathStyle: config.backup_s3_force_path_style || false
|
||||
});
|
||||
|
||||
// List buckets using the S3 client
|
||||
const { ListBucketsCommand } = require('@aws-sdk/client-s3');
|
||||
const result = await s3Adapter.s3Client.send(new ListBucketsCommand({}));
|
||||
|
||||
res.json({
|
||||
buckets: result.Buckets || [],
|
||||
owner: result.Owner || null
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to list S3 buckets:', error);
|
||||
res.status(500).json({ error: 'Failed to list S3 buckets: ' + error.message });
|
||||
}
|
||||
});
|
||||
|
||||
// List files in S3 backup location
|
||||
router.get('/s3/files', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const { prefix = '', maxKeys = 100, continuationToken } = req.query;
|
||||
const config = await getBackupConfig();
|
||||
|
||||
if (config.backup_destination_type !== 's3') {
|
||||
return res.status(400).json({ error: 'S3 backup not configured' });
|
||||
}
|
||||
|
||||
const s3Adapter = new S3StorageAdapter({
|
||||
endpoint: config.backup_s3_endpoint,
|
||||
bucket: config.backup_s3_bucket,
|
||||
accessKeyId: config.backup_s3_access_key,
|
||||
secretAccessKey: config.backup_s3_secret_key,
|
||||
region: config.backup_s3_region || 'us-east-1',
|
||||
forcePathStyle: config.backup_s3_force_path_style || false
|
||||
});
|
||||
|
||||
const result = await s3Adapter.list(prefix, {
|
||||
maxKeys: parseInt(maxKeys),
|
||||
continuationToken
|
||||
});
|
||||
|
||||
res.json({
|
||||
files: result.objects || [],
|
||||
directories: result.directories || [],
|
||||
isTruncated: result.isTruncated,
|
||||
nextContinuationToken: result.nextContinuationToken,
|
||||
prefix: prefix
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to list S3 files:', error);
|
||||
res.status(500).json({ error: 'Failed to list S3 files: ' + error.message });
|
||||
}
|
||||
});
|
||||
|
||||
// Clean up old S3 backups
|
||||
router.delete('/s3/cleanup', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const { retentionDays = 30, dryRun = false } = req.body;
|
||||
const config = await getBackupConfig();
|
||||
|
||||
if (config.backup_destination_type !== 's3') {
|
||||
return res.status(400).json({ error: 'S3 backup not configured' });
|
||||
}
|
||||
|
||||
const s3Adapter = new S3StorageAdapter({
|
||||
endpoint: config.backup_s3_endpoint,
|
||||
bucket: config.backup_s3_bucket,
|
||||
accessKeyId: config.backup_s3_access_key,
|
||||
secretAccessKey: config.backup_s3_secret_key,
|
||||
region: config.backup_s3_region || 'us-east-1',
|
||||
forcePathStyle: config.backup_s3_force_path_style || false
|
||||
});
|
||||
|
||||
const cutoffDate = new Date();
|
||||
cutoffDate.setDate(cutoffDate.getDate() - retentionDays);
|
||||
|
||||
// List all backup files
|
||||
const backupFiles = await s3Adapter.list('backups/', { maxKeys: 1000 });
|
||||
const filesToDelete = [];
|
||||
let totalSize = 0;
|
||||
|
||||
for (const file of backupFiles.objects || []) {
|
||||
if (file.lastModified && new Date(file.lastModified) < cutoffDate) {
|
||||
filesToDelete.push(file.key);
|
||||
totalSize += file.size || 0;
|
||||
}
|
||||
}
|
||||
|
||||
if (dryRun) {
|
||||
return res.json({
|
||||
wouldDelete: filesToDelete.length,
|
||||
totalSize: totalSize,
|
||||
files: filesToDelete.slice(0, 100), // Limit preview
|
||||
message: 'Dry run completed - no files were deleted'
|
||||
});
|
||||
}
|
||||
|
||||
// Delete files in batches
|
||||
const deleteResult = await s3Adapter.deleteMany(filesToDelete);
|
||||
const deletedCount = deleteResult.Deleted ? deleteResult.Deleted.length : 0;
|
||||
|
||||
// Also clean up database records
|
||||
await cleanupOldBackupRuns(retentionDays);
|
||||
|
||||
res.json({
|
||||
success: true,
|
||||
deletedCount: deletedCount,
|
||||
totalSize: totalSize,
|
||||
message: `Cleaned up ${deletedCount} S3 backup files older than ${retentionDays} days`
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to cleanup S3 backups:', error);
|
||||
res.status(500).json({ error: 'Failed to cleanup S3 backups: ' + error.message });
|
||||
}
|
||||
});
|
||||
|
||||
// Test S3 upload functionality
|
||||
router.post('/s3/test-upload', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const config = await getBackupConfig();
|
||||
|
||||
if (config.backup_destination_type !== 's3') {
|
||||
return res.status(400).json({ error: 'S3 backup not configured' });
|
||||
}
|
||||
|
||||
const s3Adapter = new S3StorageAdapter({
|
||||
endpoint: config.backup_s3_endpoint,
|
||||
bucket: config.backup_s3_bucket,
|
||||
accessKeyId: config.backup_s3_access_key,
|
||||
secretAccessKey: config.backup_s3_secret_key,
|
||||
region: config.backup_s3_region || 'us-east-1',
|
||||
forcePathStyle: config.backup_s3_force_path_style || false
|
||||
});
|
||||
|
||||
// Create test content
|
||||
const testKey = `test/backup-test-${Date.now()}.txt`;
|
||||
const testContent = `PicPeak S3 backup test\nTimestamp: ${new Date().toISOString()}\nEndpoint: ${config.backup_s3_endpoint || 'AWS'}\nBucket: ${config.backup_s3_bucket}`;
|
||||
|
||||
// Test upload
|
||||
const uploadStart = Date.now();
|
||||
await s3Adapter.upload(testKey, Buffer.from(testContent));
|
||||
const uploadTime = Date.now() - uploadStart;
|
||||
|
||||
// Test download
|
||||
const downloadStart = Date.now();
|
||||
const downloadedContent = await s3Adapter.download(testKey);
|
||||
const downloadTime = Date.now() - downloadStart;
|
||||
|
||||
// Verify content
|
||||
const contentMatch = downloadedContent.toString() === testContent;
|
||||
|
||||
// Test deletion
|
||||
await s3Adapter.delete(testKey);
|
||||
|
||||
res.json({
|
||||
success: true,
|
||||
testKey: testKey,
|
||||
uploadTime: uploadTime,
|
||||
downloadTime: downloadTime,
|
||||
contentMatch: contentMatch,
|
||||
message: 'S3 upload test completed successfully'
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('S3 upload test failed:', error);
|
||||
res.status(500).json({ error: 'S3 upload test failed: ' + error.message });
|
||||
}
|
||||
});
|
||||
|
||||
// Download entire backup
|
||||
router.get('/download/:backupId', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const { backupId } = req.params;
|
||||
|
||||
// Get backup run details
|
||||
const backupRun = await db('backup_runs')
|
||||
.where('id', backupId)
|
||||
.first();
|
||||
|
||||
if (!backupRun) {
|
||||
return res.status(404).json({ error: 'Backup not found' });
|
||||
}
|
||||
|
||||
if (backupRun.status !== 'completed') {
|
||||
return res.status(400).json({ error: 'Backup is not completed' });
|
||||
}
|
||||
|
||||
const config = await getBackupConfig();
|
||||
|
||||
// Handle different backup types
|
||||
switch (config.backup_destination_type) {
|
||||
case 'local':
|
||||
// Stream local backup as zip
|
||||
const backupPath = path.join(config.backup_destination_path, `backup-${backupRun.id}`);
|
||||
const archive = archiver('zip', { zlib: { level: 9 } });
|
||||
|
||||
res.attachment(`picpeak-backup-${backupRun.id}.zip`);
|
||||
archive.pipe(res);
|
||||
|
||||
// Add backup directory contents
|
||||
archive.directory(backupPath, false);
|
||||
|
||||
// Add manifest if exists
|
||||
if (backupRun.manifest_path && await fs.access(backupRun.manifest_path).then(() => true).catch(() => false)) {
|
||||
archive.file(backupRun.manifest_path, { name: 'manifest.json' });
|
||||
}
|
||||
|
||||
await archive.finalize();
|
||||
break;
|
||||
|
||||
case 's3':
|
||||
// For S3, provide pre-signed URLs or stream files
|
||||
const s3Adapter = new S3StorageAdapter({
|
||||
endpoint: config.backup_s3_endpoint,
|
||||
bucket: config.backup_s3_bucket,
|
||||
accessKeyId: config.backup_s3_access_key,
|
||||
secretAccessKey: config.backup_s3_secret_key,
|
||||
region: config.backup_s3_region || 'us-east-1',
|
||||
forcePathStyle: config.backup_s3_force_path_style || false
|
||||
});
|
||||
|
||||
// List all files for this backup
|
||||
const prefix = `backups/${backupRun.id}/`;
|
||||
const files = await s3Adapter.list(prefix, { maxKeys: 1000 });
|
||||
|
||||
// Generate pre-signed URLs
|
||||
const urls = [];
|
||||
for (const file of files.objects || []) {
|
||||
const url = await s3Adapter.getSignedUrl('getObject', file.key, { expiresIn: 3600 }); // 1 hour
|
||||
urls.push({
|
||||
key: file.key,
|
||||
size: file.size,
|
||||
url: url
|
||||
});
|
||||
}
|
||||
|
||||
res.json({
|
||||
backupId: backupRun.id,
|
||||
type: 's3',
|
||||
files: urls,
|
||||
expiresIn: 3600,
|
||||
message: 'Use the provided URLs to download individual files'
|
||||
});
|
||||
break;
|
||||
|
||||
case 'rsync':
|
||||
return res.status(400).json({ error: 'Direct download not available for rsync backups' });
|
||||
|
||||
default:
|
||||
return res.status(400).json({ error: 'Unknown backup type' });
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error('Failed to download backup:', error);
|
||||
res.status(500).json({ error: 'Failed to download backup: ' + error.message });
|
||||
}
|
||||
});
|
||||
|
||||
// Get current file checksums
|
||||
router.get('/checksums', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const { path: targetPath = '', recursive = true } = req.query;
|
||||
const checksums = {};
|
||||
|
||||
// Get storage path
|
||||
const storagePath = process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
|
||||
const basePath = targetPath ? path.join(storagePath, targetPath) : storagePath;
|
||||
|
||||
// Calculate checksums for files
|
||||
async function calculateDirChecksums(dirPath, relative = '') {
|
||||
try {
|
||||
const entries = await fs.readdir(dirPath, { withFileTypes: true });
|
||||
|
||||
for (const entry of entries) {
|
||||
const fullPath = path.join(dirPath, entry.name);
|
||||
const relativePath = path.join(relative, entry.name);
|
||||
|
||||
if (entry.isDirectory() && recursive) {
|
||||
await calculateDirChecksums(fullPath, relativePath);
|
||||
} else if (entry.isFile()) {
|
||||
const hash = crypto.createHash('sha256');
|
||||
const stream = require('fs').createReadStream(fullPath);
|
||||
|
||||
await new Promise((resolve, reject) => {
|
||||
stream.on('data', data => hash.update(data));
|
||||
stream.on('end', () => {
|
||||
checksums[relativePath] = {
|
||||
checksum: hash.digest('hex'),
|
||||
size: entry.size,
|
||||
modified: entry.mtime
|
||||
};
|
||||
resolve();
|
||||
});
|
||||
stream.on('error', reject);
|
||||
});
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error(`Failed to calculate checksums for ${dirPath}:`, error);
|
||||
}
|
||||
}
|
||||
|
||||
await calculateDirChecksums(basePath);
|
||||
|
||||
// Also get database checksums from backup_file_states
|
||||
const dbChecksums = await db('backup_file_states')
|
||||
.select('file_path', 'checksum', 'size_bytes', 'last_modified');
|
||||
|
||||
res.json({
|
||||
currentChecksums: checksums,
|
||||
totalFiles: Object.keys(checksums).length,
|
||||
databaseChecksums: dbChecksums.reduce((acc, row) => {
|
||||
acc[row.file_path] = {
|
||||
checksum: row.checksum,
|
||||
size: row.size_bytes,
|
||||
modified: row.last_modified
|
||||
};
|
||||
return acc;
|
||||
}, {}),
|
||||
path: targetPath || '/'
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to get file checksums:', error);
|
||||
res.status(500).json({ error: 'Failed to get file checksums: ' + error.message });
|
||||
}
|
||||
});
|
||||
|
||||
// Estimate backup size before running
|
||||
router.post('/estimate', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const { includeArchived = true } = req.body;
|
||||
|
||||
// Get storage path
|
||||
const storagePath = process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
|
||||
let totalSize = 0;
|
||||
let fileCount = 0;
|
||||
const breakdown = {};
|
||||
|
||||
// Estimate size for each directory
|
||||
async function estimateDir(dirPath, category) {
|
||||
let dirSize = 0;
|
||||
let dirCount = 0;
|
||||
|
||||
try {
|
||||
const entries = await fs.readdir(dirPath, { withFileTypes: true });
|
||||
|
||||
for (const entry of entries) {
|
||||
const fullPath = path.join(dirPath, entry.name);
|
||||
|
||||
if (entry.isDirectory()) {
|
||||
const subResult = await estimateDir(fullPath, category);
|
||||
dirSize += subResult.size;
|
||||
dirCount += subResult.count;
|
||||
} else if (entry.isFile()) {
|
||||
const stats = await fs.stat(fullPath);
|
||||
dirSize += stats.size;
|
||||
dirCount++;
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
if (error.code !== 'ENOENT') {
|
||||
logger.error(`Failed to estimate ${dirPath}:`, error);
|
||||
}
|
||||
}
|
||||
|
||||
return { size: dirSize, count: dirCount };
|
||||
}
|
||||
|
||||
// Estimate each category
|
||||
const categories = [
|
||||
{ path: 'events/active', name: 'Active Events' },
|
||||
{ path: 'thumbnails', name: 'Thumbnails' },
|
||||
{ path: 'uploads', name: 'Uploads' }
|
||||
];
|
||||
|
||||
if (includeArchived) {
|
||||
categories.push({ path: 'events/archived', name: 'Archived Events' });
|
||||
}
|
||||
|
||||
for (const category of categories) {
|
||||
const result = await estimateDir(path.join(storagePath, category.path), category.name);
|
||||
breakdown[category.name] = {
|
||||
size: result.size,
|
||||
sizeFormatted: formatBytes(result.size),
|
||||
fileCount: result.count
|
||||
};
|
||||
totalSize += result.size;
|
||||
fileCount += result.count;
|
||||
}
|
||||
|
||||
// Estimate database size
|
||||
const dbPath = process.env.DB_TYPE === 'postgresql'
|
||||
? null
|
||||
: path.join(__dirname, '../../database.sqlite');
|
||||
|
||||
if (dbPath) {
|
||||
try {
|
||||
const dbStats = await fs.stat(dbPath);
|
||||
breakdown['Database'] = {
|
||||
size: dbStats.size,
|
||||
sizeFormatted: formatBytes(dbStats.size),
|
||||
fileCount: 1
|
||||
};
|
||||
totalSize += dbStats.size;
|
||||
fileCount += 1;
|
||||
} catch (error) {
|
||||
logger.error('Failed to get database size:', error);
|
||||
}
|
||||
}
|
||||
|
||||
// Estimate compression ratio (typically 20-40% for mixed media)
|
||||
const estimatedCompressedSize = Math.round(totalSize * 0.7);
|
||||
|
||||
res.json({
|
||||
totalSize: totalSize,
|
||||
totalSizeFormatted: formatBytes(totalSize),
|
||||
estimatedCompressedSize: estimatedCompressedSize,
|
||||
estimatedCompressedSizeFormatted: formatBytes(estimatedCompressedSize),
|
||||
fileCount: fileCount,
|
||||
breakdown: breakdown,
|
||||
includeArchived: includeArchived,
|
||||
estimatedDuration: Math.max(60, Math.round(totalSize / (50 * 1024 * 1024))), // Estimate 50MB/s
|
||||
warnings: totalSize > 10 * 1024 * 1024 * 1024 ? ['Backup size exceeds 10GB, may take significant time'] : []
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to estimate backup size:', error);
|
||||
res.status(500).json({ error: 'Failed to estimate backup size: ' + error.message });
|
||||
}
|
||||
});
|
||||
|
||||
// Helper function to format bytes
|
||||
function formatBytes(bytes, decimals = 2) {
|
||||
if (bytes === 0) return '0 Bytes';
|
||||
const k = 1024;
|
||||
const dm = decimals < 0 ? 0 : decimals;
|
||||
const sizes = ['Bytes', 'KB', 'MB', 'GB', 'TB'];
|
||||
const i = Math.floor(Math.log(bytes) / Math.log(k));
|
||||
return parseFloat((bytes / Math.pow(k, i)).toFixed(dm)) + ' ' + sizes[i];
|
||||
}
|
||||
|
||||
// Helper function to get backup configuration
|
||||
async function getBackupConfig() {
|
||||
try {
|
||||
const settings = await db('app_settings')
|
||||
.where('setting_type', 'backup')
|
||||
.select('setting_key', 'setting_value');
|
||||
|
||||
const config = {};
|
||||
settings.forEach(setting => {
|
||||
try {
|
||||
config[setting.setting_key] = JSON.parse(setting.setting_value);
|
||||
} catch (e) {
|
||||
config[setting.setting_key] = setting.setting_value;
|
||||
}
|
||||
});
|
||||
|
||||
return config;
|
||||
} catch (error) {
|
||||
logger.error('Failed to get backup configuration:', error);
|
||||
return {};
|
||||
}
|
||||
}
|
||||
|
||||
// Helper function to validate manifest data
|
||||
async function validateManifestData(manifestData) {
|
||||
try {
|
||||
// Check required fields
|
||||
const requiredFields = ['version', 'backupId', 'timestamp', 'files'];
|
||||
const missingFields = requiredFields.filter(field => !manifestData[field]);
|
||||
|
||||
if (missingFields.length > 0) {
|
||||
return {
|
||||
valid: false,
|
||||
error: `Missing required fields: ${missingFields.join(', ')}`,
|
||||
details: { missingFields }
|
||||
};
|
||||
}
|
||||
|
||||
// Validate version
|
||||
if (manifestData.version !== '1.0') {
|
||||
return {
|
||||
valid: false,
|
||||
error: `Unsupported manifest version: ${manifestData.version}`,
|
||||
details: { version: manifestData.version }
|
||||
};
|
||||
}
|
||||
|
||||
// Validate files array
|
||||
if (!Array.isArray(manifestData.files)) {
|
||||
return {
|
||||
valid: false,
|
||||
error: 'Files must be an array',
|
||||
details: { filesType: typeof manifestData.files }
|
||||
};
|
||||
}
|
||||
|
||||
// Validate each file entry
|
||||
const invalidFiles = [];
|
||||
for (let i = 0; i < manifestData.files.length; i++) {
|
||||
const file = manifestData.files[i];
|
||||
if (!file.path || !file.checksum || typeof file.size !== 'number') {
|
||||
invalidFiles.push({ index: i, file });
|
||||
}
|
||||
}
|
||||
|
||||
if (invalidFiles.length > 0) {
|
||||
return {
|
||||
valid: false,
|
||||
error: `Invalid file entries: ${invalidFiles.length}`,
|
||||
details: { invalidFiles: invalidFiles.slice(0, 10) } // Limit to first 10
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
valid: true,
|
||||
details: {
|
||||
version: manifestData.version,
|
||||
backupId: manifestData.backupId,
|
||||
timestamp: manifestData.timestamp,
|
||||
fileCount: manifestData.files.length,
|
||||
totalSize: manifestData.files.reduce((sum, f) => sum + (f.size || 0), 0)
|
||||
}
|
||||
};
|
||||
} catch (error) {
|
||||
return {
|
||||
valid: false,
|
||||
error: `Validation error: ${error.message}`,
|
||||
details: { error: error.message }
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = router;
|
||||
@@ -48,9 +48,9 @@ router.get('/stats', adminAuth, async (req, res) => {
|
||||
.count('id as count')
|
||||
.first();
|
||||
|
||||
// Get total downloads (last 30 days)
|
||||
// Get total downloads (last 30 days) - include both single and bulk downloads
|
||||
const totalDownloads = await db('access_logs')
|
||||
.where('action', 'download')
|
||||
.whereIn('action', ['download', 'download_all'])
|
||||
.where('timestamp', '>=', thirtyDaysAgo.toISOString())
|
||||
.count('id as count')
|
||||
.first();
|
||||
@@ -73,7 +73,7 @@ router.get('/stats', adminAuth, async (req, res) => {
|
||||
.first();
|
||||
|
||||
const previousDownloads = await db('access_logs')
|
||||
.where('action', 'download')
|
||||
.whereIn('action', ['download', 'download_all'])
|
||||
.where('timestamp', '>=', sixtyDaysAgo.toISOString())
|
||||
.where('timestamp', '<', thirtyDaysAgo.toISOString())
|
||||
.count('id as count')
|
||||
@@ -243,10 +243,10 @@ router.get('/analytics', adminAuth, async (req, res) => {
|
||||
.where('timestamp', '>=', startDateStr)
|
||||
.groupByRaw('DATE(timestamp)');
|
||||
|
||||
// Get downloads per day
|
||||
// Get downloads per day - include both single and bulk downloads
|
||||
const downloadsData = await db('access_logs')
|
||||
.select(db.raw('DATE(timestamp) as date'), db.raw('COUNT(*) as count'))
|
||||
.where('action', 'download')
|
||||
.whereIn('action', ['download', 'download_all'])
|
||||
.where('timestamp', '>=', startDateStr)
|
||||
.groupByRaw('DATE(timestamp)');
|
||||
|
||||
@@ -272,14 +272,15 @@ router.get('/analytics', adminAuth, async (req, res) => {
|
||||
if (dateObj) dateObj.uniqueVisitors = row.count;
|
||||
});
|
||||
|
||||
// Get top galleries by views
|
||||
// Get top galleries by views with additional metrics
|
||||
const topGalleries = await db('access_logs')
|
||||
.select('events.event_name', 'events.slug')
|
||||
.select(db.raw('COUNT(*) as views'))
|
||||
.select('events.id', 'events.event_name', 'events.slug')
|
||||
.select(db.raw('COUNT(CASE WHEN action = \'view\' THEN 1 END) as views'))
|
||||
.select(db.raw('COUNT(DISTINCT CASE WHEN action = \'view\' THEN ip_address END) as uniqueVisitors'))
|
||||
.select(db.raw('COUNT(CASE WHEN action IN (\'download\', \'download_all\') THEN 1 END) as downloads'))
|
||||
.join('events', 'access_logs.event_id', 'events.id')
|
||||
.where('access_logs.action', 'view')
|
||||
.where('access_logs.timestamp', '>=', startDateStr)
|
||||
.groupBy('events.id')
|
||||
.groupBy('events.id', 'events.event_name', 'events.slug')
|
||||
.orderBy('views', 'desc')
|
||||
.limit(5);
|
||||
|
||||
@@ -309,10 +310,33 @@ router.get('/analytics', adminAuth, async (req, res) => {
|
||||
devices[d.device_type] = Math.round((d.count / totalDevices) * 100);
|
||||
});
|
||||
|
||||
// Calculate totals for the period (matching /stats logic)
|
||||
const totalViews = await db('access_logs')
|
||||
.where('action', 'view')
|
||||
.where('timestamp', '>=', startDateStr)
|
||||
.count('id as count')
|
||||
.first();
|
||||
|
||||
const totalDownloadsCount = await db('access_logs')
|
||||
.whereIn('action', ['download', 'download_all'])
|
||||
.where('timestamp', '>=', startDateStr)
|
||||
.count('id as count')
|
||||
.first();
|
||||
|
||||
const totalUniqueVisitors = await db('access_logs')
|
||||
.where('timestamp', '>=', startDateStr)
|
||||
.countDistinct('ip_address as count')
|
||||
.first();
|
||||
|
||||
res.json({
|
||||
chartData: dates,
|
||||
topGalleries,
|
||||
devices
|
||||
devices,
|
||||
totals: {
|
||||
views: totalViews?.count || 0,
|
||||
downloads: totalDownloadsCount?.count || 0,
|
||||
uniqueVisitors: totalUniqueVisitors?.count || 0
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Analytics error:', error);
|
||||
|
||||
@@ -0,0 +1,273 @@
|
||||
const express = require('express');
|
||||
const router = express.Router();
|
||||
const { adminAuth } = require('../middleware/auth');
|
||||
const { databaseBackupService } = require('../services/databaseBackup');
|
||||
const { db } = require('../database/db');
|
||||
const logger = require('../utils/logger');
|
||||
|
||||
// All routes require admin authentication
|
||||
router.use(adminAuth);
|
||||
|
||||
/**
|
||||
* Get database backup status and configuration
|
||||
*/
|
||||
router.get('/status', async (req, res) => {
|
||||
try {
|
||||
// Get configuration
|
||||
const config = await databaseBackupService.getBackupConfig();
|
||||
|
||||
// Get recent backup history
|
||||
const history = await databaseBackupService.getBackupHistory(10);
|
||||
|
||||
// Get current progress if running
|
||||
const progress = databaseBackupService.getProgress();
|
||||
|
||||
// Calculate health status
|
||||
const lastBackup = history[0];
|
||||
const isHealthy = lastBackup && lastBackup.status === 'completed' &&
|
||||
new Date(lastBackup.completed_at) > new Date(Date.now() - 48 * 60 * 60 * 1000); // Within 48 hours
|
||||
|
||||
res.json({
|
||||
config,
|
||||
isRunning: databaseBackupService.isRunning,
|
||||
isHealthy,
|
||||
currentProgress: progress,
|
||||
lastBackup,
|
||||
recentBackups: history,
|
||||
dbType: databaseBackupService.dbType
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to get database backup status:', error);
|
||||
res.status(500).json({ error: 'Failed to get backup status' });
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* Update database backup configuration
|
||||
*/
|
||||
router.put('/config', async (req, res) => {
|
||||
try {
|
||||
const allowedSettings = [
|
||||
'database_backup_enabled',
|
||||
'database_backup_schedule',
|
||||
'database_backup_destination_path',
|
||||
'database_backup_compress',
|
||||
'database_backup_validate_integrity',
|
||||
'database_backup_include_checksums',
|
||||
'database_backup_retention_days',
|
||||
'database_backup_email_on_failure',
|
||||
'database_backup_email_on_success'
|
||||
];
|
||||
|
||||
const updates = [];
|
||||
|
||||
for (const [key, value] of Object.entries(req.body)) {
|
||||
if (allowedSettings.includes(key)) {
|
||||
// Check if setting exists
|
||||
const existing = await db('app_settings')
|
||||
.where('setting_key', key)
|
||||
.first();
|
||||
|
||||
if (existing) {
|
||||
await db('app_settings')
|
||||
.where('setting_key', key)
|
||||
.update({
|
||||
setting_value: JSON.stringify(value),
|
||||
updated_at: new Date()
|
||||
});
|
||||
} else {
|
||||
await db('app_settings').insert({
|
||||
setting_key: key,
|
||||
setting_value: JSON.stringify(value),
|
||||
setting_type: 'database_backup'
|
||||
});
|
||||
}
|
||||
|
||||
updates.push(key);
|
||||
}
|
||||
}
|
||||
|
||||
// Restart scheduled backups if enabled state changed
|
||||
if (updates.includes('database_backup_enabled') || updates.includes('database_backup_schedule')) {
|
||||
const { startScheduledBackups, stopScheduledBackups } = require('../services/databaseBackup');
|
||||
stopScheduledBackups();
|
||||
await startScheduledBackups();
|
||||
}
|
||||
|
||||
res.json({
|
||||
success: true,
|
||||
updatedSettings: updates,
|
||||
message: 'Database backup configuration updated successfully'
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to update database backup config:', error);
|
||||
res.status(500).json({ error: 'Failed to update configuration' });
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* Trigger manual database backup
|
||||
*/
|
||||
router.post('/backup', async (req, res) => {
|
||||
try {
|
||||
if (databaseBackupService.isRunning) {
|
||||
return res.status(409).json({ error: 'Backup already in progress' });
|
||||
}
|
||||
|
||||
// Start backup asynchronously
|
||||
res.json({
|
||||
success: true,
|
||||
message: 'Database backup started',
|
||||
trackingUrl: '/api/admin/database-backup/progress'
|
||||
});
|
||||
|
||||
// Run backup in background
|
||||
databaseBackupService.backup(req.body).catch(error => {
|
||||
logger.error('Manual database backup failed:', error);
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to start database backup:', error);
|
||||
res.status(500).json({ error: 'Failed to start backup' });
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* Get current backup progress
|
||||
*/
|
||||
router.get('/progress', async (req, res) => {
|
||||
try {
|
||||
const progress = databaseBackupService.getProgress();
|
||||
|
||||
res.json({
|
||||
isRunning: databaseBackupService.isRunning,
|
||||
progress
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to get backup progress:', error);
|
||||
res.status(500).json({ error: 'Failed to get progress' });
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* Get backup history with pagination
|
||||
*/
|
||||
router.get('/history', async (req, res) => {
|
||||
try {
|
||||
const page = parseInt(req.query.page) || 1;
|
||||
const limit = parseInt(req.query.limit) || 20;
|
||||
const offset = (page - 1) * limit;
|
||||
|
||||
const [backups, totalCount] = await Promise.all([
|
||||
db('database_backup_runs')
|
||||
.orderBy('started_at', 'desc')
|
||||
.limit(limit)
|
||||
.offset(offset),
|
||||
db('database_backup_runs').count('* as count').first()
|
||||
]);
|
||||
|
||||
res.json({
|
||||
backups,
|
||||
pagination: {
|
||||
page,
|
||||
limit,
|
||||
total: totalCount.count,
|
||||
pages: Math.ceil(totalCount.count / limit)
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to get backup history:', error);
|
||||
res.status(500).json({ error: 'Failed to get history' });
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* Delete old backup files
|
||||
*/
|
||||
router.delete('/cleanup', async (req, res) => {
|
||||
try {
|
||||
const { retentionDays = 30 } = req.body;
|
||||
|
||||
await databaseBackupService.cleanupOldBackups(retentionDays);
|
||||
|
||||
res.json({
|
||||
success: true,
|
||||
message: `Cleaned up backups older than ${retentionDays} days`
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to cleanup old backups:', error);
|
||||
res.status(500).json({ error: 'Failed to cleanup backups' });
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* Test database backup configuration
|
||||
*/
|
||||
router.post('/test', async (req, res) => {
|
||||
try {
|
||||
const config = await databaseBackupService.getBackupConfig();
|
||||
|
||||
// Test database connection
|
||||
const testResults = {
|
||||
databaseConnection: false,
|
||||
destinationWritable: false,
|
||||
compressionAvailable: true,
|
||||
estimatedSize: null
|
||||
};
|
||||
|
||||
// Test database connection
|
||||
try {
|
||||
await db.raw('SELECT 1');
|
||||
testResults.databaseConnection = true;
|
||||
} catch (error) {
|
||||
testResults.databaseConnectionError = error.message;
|
||||
}
|
||||
|
||||
// Test destination path
|
||||
if (config.destinationPath) {
|
||||
try {
|
||||
const fs = require('fs').promises;
|
||||
const testFile = `${config.destinationPath}/.test-${Date.now()}`;
|
||||
await fs.writeFile(testFile, 'test');
|
||||
await fs.unlink(testFile);
|
||||
testResults.destinationWritable = true;
|
||||
} catch (error) {
|
||||
testResults.destinationError = error.message;
|
||||
}
|
||||
}
|
||||
|
||||
// Estimate database size
|
||||
try {
|
||||
testResults.estimatedSize = await databaseBackupService.getDatabaseSize();
|
||||
} catch (error) {
|
||||
testResults.sizeError = error.message;
|
||||
}
|
||||
|
||||
res.json({
|
||||
success: testResults.databaseConnection && testResults.destinationWritable,
|
||||
results: testResults
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to test backup configuration:', error);
|
||||
res.status(500).json({ error: 'Failed to test configuration' });
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* Get table checksums
|
||||
*/
|
||||
router.get('/checksums', async (req, res) => {
|
||||
try {
|
||||
const checksums = await databaseBackupService.getTableChecksums();
|
||||
|
||||
res.json({
|
||||
checksums,
|
||||
tableCount: Object.keys(checksums).length,
|
||||
totalRows: Object.values(checksums).reduce((sum, table) => sum + table.rowCount, 0)
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to get table checksums:', error);
|
||||
res.status(500).json({ error: 'Failed to get checksums' });
|
||||
}
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
@@ -210,15 +210,15 @@ router.get('/templates', adminAuth, async (req, res) => {
|
||||
id: template.id,
|
||||
template_key: template.template_key,
|
||||
variables: (() => {
|
||||
try {
|
||||
if (!template.variables) return [];
|
||||
if (typeof template.variables === 'object') return template.variables;
|
||||
return JSON.parse(template.variables);
|
||||
} catch (e) {
|
||||
console.warn('Failed to parse variables for template:', template.template_key, e.message);
|
||||
return [];
|
||||
}
|
||||
})(),
|
||||
try {
|
||||
if (!template.variables) return [];
|
||||
if (typeof template.variables === 'object') return template.variables;
|
||||
return JSON.parse(template.variables);
|
||||
} catch (e) {
|
||||
console.warn('Failed to parse variables for template:', template.template_key, e.message);
|
||||
return [];
|
||||
}
|
||||
})(),
|
||||
updated_at: template.updated_at
|
||||
};
|
||||
|
||||
|
||||
@@ -42,7 +42,7 @@ router.post('/', adminAuth, [
|
||||
} = req.body;
|
||||
|
||||
// Validate password strength for gallery
|
||||
const passwordValidation = validatePasswordInContext(password, 'gallery', {
|
||||
const passwordValidation = await validatePasswordInContext(password, 'gallery', {
|
||||
eventName: event_name
|
||||
});
|
||||
|
||||
|
||||
@@ -8,8 +8,9 @@ const crypto = require('crypto');
|
||||
const fs = require('fs').promises;
|
||||
const path = require('path');
|
||||
const { archiveEvent } = require('../services/archiveService');
|
||||
const { queueEmail } = require('../services/emailProcessor');
|
||||
const { escapeLikePattern } = require('../utils/sqlSecurity');
|
||||
const { formatDate } = require('../utils/dateFormatter');
|
||||
// formatDate import removed - dates are formatted by email processor
|
||||
const { validatePasswordInContext, getBcryptRounds } = require('../utils/passwordValidation');
|
||||
const { formatBoolean } = require('../utils/dbCompat');
|
||||
|
||||
@@ -52,7 +53,7 @@ router.post('/', adminAuth, [
|
||||
} = req.body;
|
||||
|
||||
// Validate password strength
|
||||
const passwordValidation = validatePasswordInContext(password, 'gallery', {
|
||||
const passwordValidation = await validatePasswordInContext(password, 'gallery', {
|
||||
eventName: event_name
|
||||
});
|
||||
|
||||
@@ -66,7 +67,12 @@ router.post('/', adminAuth, [
|
||||
}
|
||||
|
||||
// Generate unique slug
|
||||
const baseSlug = `${event_type}-${event_name.toLowerCase().replace(/[^a-z0-9]/g, '-')}-${event_date}`;
|
||||
const processedEventName = event_name
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9]/g, '-') // Replace non-alphanumeric with dash
|
||||
.replace(/-+/g, '-') // Replace multiple dashes with single dash
|
||||
.replace(/^-|-$/g, ''); // Remove leading/trailing dashes
|
||||
const baseSlug = `${event_type}-${processedEventName}-${event_date}`;
|
||||
let slug = baseSlug;
|
||||
let counter = 1;
|
||||
|
||||
@@ -83,7 +89,14 @@ router.post('/', adminAuth, [
|
||||
const password_hash = await bcrypt.hash(password, getBcryptRounds());
|
||||
|
||||
// Calculate expiration date (days after event date)
|
||||
const expires_at = new Date(event_date);
|
||||
// Parse YYYY-MM-DD format as local date to avoid timezone issues
|
||||
let expires_at;
|
||||
if (event_date.match(/^\d{4}-\d{2}-\d{2}$/)) {
|
||||
const [year, month, day] = event_date.split('-').map(num => parseInt(num, 10));
|
||||
expires_at = new Date(year, month - 1, day);
|
||||
} else {
|
||||
expires_at = new Date(event_date);
|
||||
}
|
||||
expires_at.setDate(expires_at.getDate() + parseInt(expiration_days, 10));
|
||||
|
||||
// Create folder structure
|
||||
@@ -122,8 +135,7 @@ router.post('/', adminAuth, [
|
||||
);
|
||||
|
||||
// Queue creation email
|
||||
// Determine language based on email domain
|
||||
const emailLang = host_email.endsWith('.de') ? 'de' : 'en';
|
||||
// Language detection is handled by email processor
|
||||
|
||||
await db('email_queue').insert({
|
||||
event_id: eventId,
|
||||
@@ -132,10 +144,10 @@ router.post('/', adminAuth, [
|
||||
email_data: JSON.stringify({
|
||||
host_name: host_name,
|
||||
event_name,
|
||||
event_date: await formatDate(event_date, emailLang),
|
||||
event_date: event_date, // Pass raw date - will be formatted by email processor
|
||||
gallery_link: shareLink,
|
||||
gallery_password: password,
|
||||
expiry_date: await formatDate(expires_at, emailLang),
|
||||
expiry_date: expires_at.toISOString(), // Pass ISO string - will be formatted by email processor
|
||||
welcome_message: welcome_message || ''
|
||||
}),
|
||||
status: 'pending',
|
||||
@@ -388,13 +400,56 @@ router.delete('/:id', adminAuth, async (req, res) => {
|
||||
return res.status(404).json({ error: 'Event not found' });
|
||||
}
|
||||
|
||||
// Delete associated photos
|
||||
await db('photos').where('event_id', id).del();
|
||||
// Start a transaction to ensure all deletions succeed or fail together
|
||||
await db.transaction(async (trx) => {
|
||||
// 1. Delete activity logs (audit trail)
|
||||
await trx('activity_logs').where('event_id', id).del();
|
||||
|
||||
// Delete event
|
||||
await db('events').where('id', id).del();
|
||||
// 2. Delete access logs
|
||||
await trx('access_logs').where('event_id', id).del();
|
||||
|
||||
// Log activity
|
||||
// 3. Delete email queue entries
|
||||
await trx('email_queue').where('event_id', id).del();
|
||||
|
||||
// 4. Delete photos (this will also handle hero_photo_id foreign key)
|
||||
await trx('photos').where('event_id', id).del();
|
||||
|
||||
// 5. Delete categories (photo_categories has CASCADE delete for event_id)
|
||||
await trx('photo_categories').where('event_id', id).del();
|
||||
|
||||
// 6. Finally delete the event
|
||||
await trx('events').where('id', id).del();
|
||||
|
||||
// Delete event folder from storage if it exists
|
||||
if (event.folder_path) {
|
||||
const storagePath = process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
|
||||
const eventFolderPath = path.join(storagePath, 'events', 'active', event.folder_path);
|
||||
|
||||
try {
|
||||
const fsPromises = require('fs').promises;
|
||||
await fsPromises.rm(eventFolderPath, { recursive: true, force: true });
|
||||
} catch (err) {
|
||||
console.error('Failed to delete event folder:', err);
|
||||
// Don't fail the transaction if folder deletion fails
|
||||
}
|
||||
}
|
||||
|
||||
// Delete archive if exists
|
||||
if (event.archive_path) {
|
||||
const storagePath = process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
|
||||
const archivePath = path.join(storagePath, event.archive_path);
|
||||
|
||||
try {
|
||||
const fsPromises = require('fs').promises;
|
||||
await fsPromises.unlink(archivePath);
|
||||
} catch (err) {
|
||||
console.error('Failed to delete archive file:', err);
|
||||
// Don't fail the transaction if file deletion fails
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// Log activity (outside transaction)
|
||||
await logActivity('event_deleted',
|
||||
{ event_name: event.event_name },
|
||||
null,
|
||||
@@ -404,7 +459,19 @@ router.delete('/:id', adminAuth, async (req, res) => {
|
||||
res.json({ message: 'Event deleted successfully' });
|
||||
} catch (error) {
|
||||
console.error('Error deleting event:', error);
|
||||
res.status(500).json({ error: 'Failed to delete event' });
|
||||
|
||||
// Provide more specific error messages
|
||||
if (error.message && error.message.includes('foreign key constraint')) {
|
||||
res.status(500).json({
|
||||
error: 'Cannot delete event due to existing references. Please contact support.',
|
||||
details: error.message
|
||||
});
|
||||
} else {
|
||||
res.status(500).json({
|
||||
error: 'Failed to delete event',
|
||||
details: process.env.NODE_ENV === 'development' ? error.message : undefined
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
@@ -480,16 +547,15 @@ router.post('/:id/reset-password', adminAuth, async (req, res) => {
|
||||
|
||||
// Queue email notification if requested
|
||||
if (sendEmail) {
|
||||
const { queueEmail } = require('../services/emailProcessor');
|
||||
// For password reset, we'll need to create a template or use a different approach
|
||||
// For now, let's use the gallery_created template with updated password
|
||||
await queueEmail(id, event.host_email, 'gallery_created', {
|
||||
host_name: event.host_email.split('@')[0],
|
||||
event_name: event.event_name,
|
||||
event_date: new Date(event.event_date).toLocaleDateString(),
|
||||
event_date: event.event_date, // Pass raw date - will be formatted by email processor
|
||||
gallery_link: event.share_link,
|
||||
gallery_password: newPassword,
|
||||
expiry_date: new Date(event.expires_at).toLocaleDateString()
|
||||
expiry_date: event.expires_at // Pass raw date - will be formatted by email processor
|
||||
});
|
||||
}
|
||||
|
||||
@@ -504,6 +570,81 @@ router.post('/:id/reset-password', adminAuth, async (req, res) => {
|
||||
}
|
||||
});
|
||||
|
||||
// Resend creation email
|
||||
router.post('/:id/resend-email', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const { id } = req.params;
|
||||
|
||||
// Get event details
|
||||
const event = await db('events')
|
||||
.where('id', id)
|
||||
.first();
|
||||
|
||||
if (!event) {
|
||||
return res.status(404).json({ error: 'Event not found' });
|
||||
}
|
||||
|
||||
// The email processor will determine the language based on:
|
||||
// 1. Event language setting
|
||||
// 2. App settings general_default_language
|
||||
// 3. Email config default language
|
||||
// 4. Domain-based detection
|
||||
// So we don't need to determine it here
|
||||
|
||||
// For resending creation email, we need the actual password
|
||||
// First, try to get it from the request body if provided
|
||||
let galleryPassword = req.body.password;
|
||||
|
||||
// If no password provided, we can't decrypt the existing one
|
||||
// So we'll show a security message
|
||||
if (!galleryPassword) {
|
||||
// We'll let the email processor determine the language for the security message
|
||||
galleryPassword = '{{password_security_message}}';
|
||||
}
|
||||
|
||||
// Dates will be formatted by the email processor based on recipient language
|
||||
|
||||
// Queue the email
|
||||
await queueEmail(id, event.host_email, 'gallery_created', {
|
||||
host_name: event.host_name || event.host_email.split('@')[0],
|
||||
event_name: event.event_name,
|
||||
event_date: event.event_date, // Pass raw date - will be formatted by email processor
|
||||
gallery_link: event.share_link,
|
||||
gallery_password: galleryPassword,
|
||||
expiry_date: event.expires_at, // Pass raw date - will be formatted by email processor
|
||||
welcome_message: event.welcome_message || '',
|
||||
eventId: id,
|
||||
isResend: true // Flag to indicate this is a resend
|
||||
});
|
||||
|
||||
// Log the activity using the proper schema
|
||||
try {
|
||||
await logActivity('email_resent', {
|
||||
email_type: 'gallery_created',
|
||||
recipient: event.host_email,
|
||||
ip_address: req.ip || '0.0.0.0',
|
||||
user_agent: req.get('user-agent') || 'Unknown'
|
||||
}, id, {
|
||||
type: 'admin',
|
||||
id: req.admin.id,
|
||||
name: req.admin.username
|
||||
});
|
||||
} catch (logError) {
|
||||
console.error('Warning: Failed to log activity:', logError);
|
||||
// Don't fail the request if activity logging fails
|
||||
}
|
||||
|
||||
res.json({
|
||||
success: true,
|
||||
message: 'Creation email has been queued for sending'
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Error resending creation email:', error);
|
||||
console.error('Stack trace:', error.stack);
|
||||
res.status(500).json({ error: 'Failed to resend creation email' });
|
||||
}
|
||||
});
|
||||
|
||||
// Archive event
|
||||
router.post('/:id/archive', adminAuth, async (req, res) => {
|
||||
try {
|
||||
|
||||
@@ -0,0 +1,411 @@
|
||||
const express = require('express');
|
||||
const router = express.Router();
|
||||
const { adminAuth } = require('../middleware/auth-enhanced-v2');
|
||||
const feedbackService = require('../services/feedbackService');
|
||||
const feedbackModeration = require('../services/feedbackModeration');
|
||||
const { db, logActivity } = require('../database/db');
|
||||
const logger = require('../utils/logger');
|
||||
const {
|
||||
validateEventId,
|
||||
validateFeedbackSettings,
|
||||
validateWordFilter,
|
||||
checkValidation
|
||||
} = require('../utils/feedbackValidation');
|
||||
|
||||
// Get event feedback settings
|
||||
router.get('/events/:eventId/feedback-settings',
|
||||
adminAuth,
|
||||
validateEventId,
|
||||
checkValidation,
|
||||
async (req, res) => {
|
||||
try {
|
||||
const { eventId } = req.params;
|
||||
|
||||
// Verify event exists and belongs to admin
|
||||
const event = await db('events').where('id', eventId).first();
|
||||
if (!event) {
|
||||
return res.status(404).json({ error: 'Event not found' });
|
||||
}
|
||||
|
||||
const settings = await feedbackService.getEventFeedbackSettings(eventId);
|
||||
res.json(settings);
|
||||
} catch (error) {
|
||||
logger.error('Error getting feedback settings:', error);
|
||||
res.status(500).json({ error: 'Failed to get feedback settings' });
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
// Update event feedback settings
|
||||
router.put('/events/:eventId/feedback-settings',
|
||||
adminAuth,
|
||||
validateEventId,
|
||||
validateFeedbackSettings,
|
||||
checkValidation,
|
||||
async (req, res) => {
|
||||
try {
|
||||
const { eventId } = req.params;
|
||||
const settings = req.body;
|
||||
|
||||
// Verify event exists
|
||||
const event = await db('events').where('id', eventId).first();
|
||||
if (!event) {
|
||||
return res.status(404).json({ error: 'Event not found' });
|
||||
}
|
||||
|
||||
const updatedSettings = await feedbackService.updateEventFeedbackSettings(eventId, settings);
|
||||
|
||||
await logActivity('feedback_settings_updated', {
|
||||
event_id: eventId,
|
||||
settings: updatedSettings
|
||||
}, eventId, {
|
||||
type: 'admin',
|
||||
id: req.user.id,
|
||||
name: req.user.username
|
||||
});
|
||||
|
||||
res.json(updatedSettings);
|
||||
} catch (error) {
|
||||
logger.error('Error updating feedback settings:', error);
|
||||
res.status(500).json({ error: 'Failed to update feedback settings' });
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
// Get feedback for an event (with filters)
|
||||
router.get('/events/:eventId/feedback',
|
||||
adminAuth,
|
||||
validateEventId,
|
||||
checkValidation,
|
||||
async (req, res) => {
|
||||
try {
|
||||
const { eventId } = req.params;
|
||||
const { type, status, photoId, page = 1, limit = 50 } = req.query;
|
||||
|
||||
// Build query
|
||||
let query = db('photo_feedback')
|
||||
.join('photos', 'photo_feedback.photo_id', 'photos.id')
|
||||
.where('photo_feedback.event_id', eventId)
|
||||
.select(
|
||||
'photo_feedback.*',
|
||||
'photos.filename',
|
||||
'photos.path'
|
||||
);
|
||||
|
||||
if (type) {
|
||||
query = query.where('photo_feedback.feedback_type', type);
|
||||
}
|
||||
|
||||
if (status === 'pending') {
|
||||
query = query.where('photo_feedback.is_approved', false)
|
||||
.where('photo_feedback.is_hidden', false);
|
||||
} else if (status === 'approved') {
|
||||
query = query.where('photo_feedback.is_approved', true);
|
||||
} else if (status === 'hidden') {
|
||||
query = query.where('photo_feedback.is_hidden', true);
|
||||
}
|
||||
|
||||
if (photoId) {
|
||||
query = query.where('photo_feedback.photo_id', photoId);
|
||||
}
|
||||
|
||||
// Pagination
|
||||
const offset = (page - 1) * limit;
|
||||
const totalCount = await query.clone().count('photo_feedback.id as count').first();
|
||||
|
||||
const feedback = await query
|
||||
.orderBy('photo_feedback.created_at', 'desc')
|
||||
.limit(limit)
|
||||
.offset(offset);
|
||||
|
||||
res.json({
|
||||
feedback,
|
||||
pagination: {
|
||||
page: parseInt(page),
|
||||
limit: parseInt(limit),
|
||||
total: totalCount?.count || 0,
|
||||
pages: Math.ceil((totalCount?.count || 0) / limit)
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Error getting feedback:', error);
|
||||
res.status(500).json({ error: 'Failed to get feedback' });
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
// Moderate feedback (approve/hide/reject)
|
||||
router.put('/feedback/:feedbackId/:action',
|
||||
adminAuth,
|
||||
async (req, res) => {
|
||||
try {
|
||||
const { feedbackId, action } = req.params;
|
||||
|
||||
if (!['approve', 'hide', 'reject'].includes(action)) {
|
||||
return res.status(400).json({ error: 'Invalid action' });
|
||||
}
|
||||
|
||||
await feedbackService.moderateFeedback(feedbackId, action, req.user.id);
|
||||
|
||||
res.json({ success: true });
|
||||
} catch (error) {
|
||||
logger.error('Error moderating feedback:', error);
|
||||
res.status(500).json({ error: 'Failed to moderate feedback' });
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
// Delete feedback
|
||||
router.delete('/feedback/:feedbackId',
|
||||
adminAuth,
|
||||
async (req, res) => {
|
||||
try {
|
||||
const { feedbackId } = req.params;
|
||||
|
||||
await feedbackService.deleteFeedback(feedbackId, req.user.id);
|
||||
|
||||
res.json({ success: true });
|
||||
} catch (error) {
|
||||
logger.error('Error deleting feedback:', error);
|
||||
res.status(500).json({ error: 'Failed to delete feedback' });
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
// Get feedback analytics for an event
|
||||
router.get('/events/:eventId/feedback-analytics',
|
||||
adminAuth,
|
||||
validateEventId,
|
||||
checkValidation,
|
||||
async (req, res) => {
|
||||
try {
|
||||
const { eventId } = req.params;
|
||||
|
||||
// Get summary statistics
|
||||
const summaryData = await feedbackService.getEventFeedbackSummary(eventId);
|
||||
|
||||
// Calculate average rating and other summary stats
|
||||
const avgRatingResult = await db('photo_feedback')
|
||||
.where('event_id', eventId)
|
||||
.where('feedback_type', 'rating')
|
||||
.avg('rating as average_rating')
|
||||
.first();
|
||||
|
||||
const pendingModeration = await db('photo_feedback')
|
||||
.where('event_id', eventId)
|
||||
.where('feedback_type', 'comment')
|
||||
.where('is_approved', false)
|
||||
.where('is_hidden', false)
|
||||
.count('* as count')
|
||||
.first();
|
||||
|
||||
const summary = {
|
||||
average_rating: parseFloat(avgRatingResult?.average_rating || 0),
|
||||
total_ratings: summaryData.stats?.total_ratings || 0,
|
||||
total_likes: summaryData.stats?.total_likes || 0,
|
||||
total_comments: summaryData.stats?.total_comments || 0,
|
||||
total_favorites: summaryData.stats?.total_favorites || 0,
|
||||
pending_moderation: pendingModeration?.count || 0,
|
||||
total_feedback: (summaryData.stats?.total_ratings || 0) +
|
||||
(summaryData.stats?.total_likes || 0) +
|
||||
(summaryData.stats?.total_comments || 0) +
|
||||
(summaryData.stats?.total_favorites || 0)
|
||||
};
|
||||
|
||||
// Get top-rated photos
|
||||
const topRated = await db('photos')
|
||||
.where('event_id', eventId)
|
||||
.where('average_rating', '>', 0)
|
||||
.orderBy('average_rating', 'desc')
|
||||
.orderBy('feedback_count', 'desc')
|
||||
.limit(10)
|
||||
.select('id', 'filename', 'average_rating', 'feedback_count', 'like_count');
|
||||
|
||||
// Get most liked photos
|
||||
const mostLiked = await db('photos')
|
||||
.where('event_id', eventId)
|
||||
.where('like_count', '>', 0)
|
||||
.orderBy('like_count', 'desc')
|
||||
.limit(10)
|
||||
.select('id', 'filename', 'like_count', 'average_rating');
|
||||
|
||||
// Get recent comments
|
||||
const recentComments = await db('photo_feedback')
|
||||
.join('photos', 'photo_feedback.photo_id', 'photos.id')
|
||||
.where('photo_feedback.event_id', eventId)
|
||||
.where('photo_feedback.feedback_type', 'comment')
|
||||
.where('photo_feedback.is_approved', true)
|
||||
.where('photo_feedback.is_hidden', false)
|
||||
.orderBy('photo_feedback.created_at', 'desc')
|
||||
.limit(10)
|
||||
.select(
|
||||
'photo_feedback.comment_text',
|
||||
'photo_feedback.guest_name',
|
||||
'photo_feedback.created_at',
|
||||
'photos.filename'
|
||||
);
|
||||
|
||||
// Get feedback timeline (last 7 days)
|
||||
const timeline = await db('photo_feedback')
|
||||
.where('event_id', eventId)
|
||||
.where('created_at', '>', new Date(Date.now() - 7 * 24 * 60 * 60 * 1000))
|
||||
.select(
|
||||
db.raw('DATE(created_at) as date'),
|
||||
db.raw('COUNT(*) as count'),
|
||||
'feedback_type'
|
||||
)
|
||||
.groupBy('date', 'feedback_type')
|
||||
.orderBy('date', 'asc');
|
||||
|
||||
res.json({
|
||||
summary,
|
||||
topRated,
|
||||
mostLiked,
|
||||
recentComments,
|
||||
timeline
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Error getting feedback analytics:', error);
|
||||
res.status(500).json({ error: 'Failed to get feedback analytics' });
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
// Export feedback data
|
||||
router.get('/events/:eventId/feedback/export',
|
||||
adminAuth,
|
||||
validateEventId,
|
||||
checkValidation,
|
||||
async (req, res) => {
|
||||
try {
|
||||
const { eventId } = req.params;
|
||||
const { format = 'json' } = req.query;
|
||||
|
||||
const feedback = await feedbackService.exportEventFeedback(eventId);
|
||||
|
||||
if (format === 'csv') {
|
||||
// Convert to CSV
|
||||
const csv = convertToCSV(feedback);
|
||||
res.setHeader('Content-Type', 'text/csv');
|
||||
res.setHeader('Content-Disposition', `attachment; filename="feedback-${eventId}.csv"`);
|
||||
res.send(csv);
|
||||
} else {
|
||||
res.json(feedback);
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error('Error exporting feedback:', error);
|
||||
res.status(500).json({ error: 'Failed to export feedback' });
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
// Get pending moderation items (across all events)
|
||||
router.get('/feedback/pending-moderation',
|
||||
adminAuth,
|
||||
async (req, res) => {
|
||||
try {
|
||||
const pending = await feedbackService.getPendingModeration();
|
||||
res.json(pending);
|
||||
} catch (error) {
|
||||
logger.error('Error getting pending moderation:', error);
|
||||
res.status(500).json({ error: 'Failed to get pending moderation' });
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
// Word filter management
|
||||
router.get('/feedback/word-filters',
|
||||
adminAuth,
|
||||
async (req, res) => {
|
||||
try {
|
||||
const filters = await feedbackModeration.getAllWordFilters();
|
||||
res.json(filters);
|
||||
} catch (error) {
|
||||
logger.error('Error getting word filters:', error);
|
||||
res.status(500).json({ error: 'Failed to get word filters' });
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
router.post('/feedback/word-filters',
|
||||
adminAuth,
|
||||
validateWordFilter,
|
||||
checkValidation,
|
||||
async (req, res) => {
|
||||
try {
|
||||
const { word, severity = 'moderate' } = req.body;
|
||||
|
||||
await feedbackModeration.addWordFilter(word, severity);
|
||||
|
||||
await logActivity('word_filter_added', { word, severity }, null, {
|
||||
type: 'admin',
|
||||
id: req.user.id,
|
||||
name: req.user.username
|
||||
});
|
||||
|
||||
res.json({ success: true });
|
||||
} catch (error) {
|
||||
if (error.message === 'Word filter already exists') {
|
||||
return res.status(409).json({ error: error.message });
|
||||
}
|
||||
logger.error('Error adding word filter:', error);
|
||||
res.status(500).json({ error: 'Failed to add word filter' });
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
router.put('/feedback/word-filters/:id',
|
||||
adminAuth,
|
||||
async (req, res) => {
|
||||
try {
|
||||
const { id } = req.params;
|
||||
const updates = req.body;
|
||||
|
||||
await feedbackModeration.updateWordFilter(id, updates);
|
||||
|
||||
res.json({ success: true });
|
||||
} catch (error) {
|
||||
logger.error('Error updating word filter:', error);
|
||||
res.status(500).json({ error: 'Failed to update word filter' });
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
router.delete('/feedback/word-filters/:id',
|
||||
adminAuth,
|
||||
async (req, res) => {
|
||||
try {
|
||||
const { id } = req.params;
|
||||
|
||||
await feedbackModeration.deleteWordFilter(id);
|
||||
|
||||
res.json({ success: true });
|
||||
} catch (error) {
|
||||
logger.error('Error deleting word filter:', error);
|
||||
res.status(500).json({ error: 'Failed to delete word filter' });
|
||||
}
|
||||
}
|
||||
);
|
||||
|
||||
// Helper function to convert JSON to CSV
|
||||
function convertToCSV(data) {
|
||||
if (!data || data.length === 0) return '';
|
||||
|
||||
const headers = Object.keys(data[0]);
|
||||
const csvHeaders = headers.join(',');
|
||||
|
||||
const csvRows = data.map(row => {
|
||||
return headers.map(header => {
|
||||
const value = row[header];
|
||||
// Escape quotes and wrap in quotes if contains comma
|
||||
if (typeof value === 'string' && (value.includes(',') || value.includes('"'))) {
|
||||
return `"${value.replace(/"/g, '""')}"`;
|
||||
}
|
||||
return value || '';
|
||||
}).join(',');
|
||||
});
|
||||
|
||||
return [csvHeaders, ...csvRows].join('\n');
|
||||
}
|
||||
|
||||
module.exports = router;
|
||||
+245
-109
@@ -4,55 +4,41 @@ const path = require('path');
|
||||
const fs = require('fs').promises;
|
||||
const { db, logActivity } = require('../database/db');
|
||||
const { adminAuth } = require('../middleware/auth');
|
||||
const { generateThumbnail } = require('../services/imageProcessor');
|
||||
const { generateThumbnail, ensureThumbnail } = require('../services/imageProcessor');
|
||||
const { generatePhotoFilename } = require('../utils/filenameSanitizer');
|
||||
const { escapeLikePattern } = require('../utils/sqlSecurity');
|
||||
const { validateUploadedFiles } = require('../middleware/uploadValidation');
|
||||
const router = express.Router();
|
||||
|
||||
// Get storage path from environment or default
|
||||
const getStoragePath = () => process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
|
||||
|
||||
// Configure multer for file uploads
|
||||
// IMPORTANT: Using synchronous functions to prevent file corruption
|
||||
const storage = multer.diskStorage({
|
||||
destination: async (req, file, cb) => {
|
||||
destination: (req, file, cb) => {
|
||||
console.log('Multer destination called for file:', file.originalname);
|
||||
const { eventId } = req.params;
|
||||
|
||||
try {
|
||||
// Get event details
|
||||
const event = await db('events').where({ id: eventId }).first();
|
||||
if (!event) {
|
||||
console.error('Event not found in multer destination:', eventId);
|
||||
return cb(new Error('Event not found'));
|
||||
}
|
||||
|
||||
// Store event in request for use in filename generation
|
||||
req.eventData = event;
|
||||
|
||||
// Create destination path - now just event folder, no type subfolder
|
||||
const destPath = path.join(getStoragePath(), 'events/active', event.slug);
|
||||
console.log('Destination path:', destPath);
|
||||
|
||||
// Ensure directory exists
|
||||
await fs.mkdir(destPath, { recursive: true });
|
||||
|
||||
cb(null, destPath);
|
||||
} catch (error) {
|
||||
console.error('Error in multer destination:', error);
|
||||
cb(error);
|
||||
}
|
||||
// We'll validate the event exists in the route handler
|
||||
// For now, just create a temp destination
|
||||
const tempPath = path.join(getStoragePath(), 'temp', `upload_${Date.now()}_${Math.random().toString(36).substring(7)}`);
|
||||
|
||||
// Create directory synchronously
|
||||
require('fs').mkdirSync(tempPath, { recursive: true });
|
||||
console.log('Temp destination path:', tempPath);
|
||||
|
||||
// Store temp path for cleanup
|
||||
req.tempUploadPath = tempPath;
|
||||
|
||||
cb(null, tempPath);
|
||||
},
|
||||
filename: async (req, file, cb) => {
|
||||
filename: (req, file, cb) => {
|
||||
console.log('Multer filename called for file:', file.originalname);
|
||||
try {
|
||||
// Use temporary filename for now, will rename after getting category info
|
||||
const tempName = `temp_${Date.now()}_${Math.round(Math.random() * 1E9)}${path.extname(file.originalname)}`;
|
||||
console.log('Temp filename:', tempName);
|
||||
cb(null, tempName);
|
||||
} catch (error) {
|
||||
console.error('Error in multer filename:', error);
|
||||
cb(error);
|
||||
}
|
||||
// Use a simple temporary filename
|
||||
const tempName = `temp_${Date.now()}_${Math.round(Math.random() * 1E9)}${path.extname(file.originalname)}`;
|
||||
console.log('Temp filename:', tempName);
|
||||
cb(null, tempName);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -65,6 +51,9 @@ const upload = multer({
|
||||
files: 500, // Maximum 500 files
|
||||
// Set a reasonable field size limit to prevent memory issues
|
||||
fieldSize: 10 * 1024 * 1024, // 10MB for non-file fields
|
||||
// Add part size limits to prevent incomplete uploads
|
||||
parts: 10000, // Maximum number of parts (fields + files)
|
||||
headerPairs: 2000 // Maximum number of header key-value pairs
|
||||
},
|
||||
fileFilter: (req, file, cb) => {
|
||||
// Accept images only with proper validation
|
||||
@@ -75,7 +64,9 @@ const upload = multer({
|
||||
} else {
|
||||
cb(new Error('Only JPEG, PNG and WebP images are allowed'));
|
||||
}
|
||||
}
|
||||
},
|
||||
// Add abort on limit to stop processing when limits are exceeded
|
||||
abortOnLimit: true
|
||||
});
|
||||
|
||||
const { createFileUploadValidator } = require('../utils/fileSecurityUtils');
|
||||
@@ -87,9 +78,29 @@ const validateUploadContent = createFileUploadValidator({
|
||||
validateContent: true
|
||||
});
|
||||
|
||||
// Request timeout middleware for uploads
|
||||
const uploadTimeout = (timeout = 300000) => { // 5 minutes default
|
||||
return (req, res, next) => {
|
||||
// Set timeout for the request
|
||||
req.setTimeout(timeout, () => {
|
||||
console.error('Upload request timed out');
|
||||
if (!res.headersSent) {
|
||||
res.status(408).json({ error: 'Upload request timed out' });
|
||||
}
|
||||
});
|
||||
|
||||
// Set response timeout as well
|
||||
res.setTimeout(timeout, () => {
|
||||
console.error('Upload response timed out');
|
||||
});
|
||||
|
||||
next();
|
||||
};
|
||||
};
|
||||
|
||||
// Upload photos for an event
|
||||
// Increased limit to 500 files, but recommend chunked uploads for better performance
|
||||
router.post('/:eventId/upload', adminAuth, (req, res, next) => {
|
||||
router.post('/:eventId/upload', adminAuth, uploadTimeout(600000), (req, res, next) => { // 10 minute timeout
|
||||
upload.array('photos', 500)(req, res, (err) => {
|
||||
if (err) {
|
||||
console.error('Multer error:', err);
|
||||
@@ -106,7 +117,7 @@ router.post('/:eventId/upload', adminAuth, (req, res, next) => {
|
||||
}
|
||||
next();
|
||||
});
|
||||
}, validateUploadContent, async (req, res) => {
|
||||
}, validateUploadContent, validateUploadedFiles, async (req, res) => {
|
||||
try {
|
||||
const { eventId } = req.params;
|
||||
const { category_id } = req.body;
|
||||
@@ -121,12 +132,28 @@ router.post('/:eventId/upload', adminAuth, (req, res, next) => {
|
||||
const event = await db('events').where({ id: eventId }).first();
|
||||
if (!event) {
|
||||
console.error('Event not found:', eventId);
|
||||
// Clean up temp files
|
||||
if (req.tempUploadPath) {
|
||||
try {
|
||||
await fs.rm(req.tempUploadPath, { recursive: true, force: true });
|
||||
} catch (e) {
|
||||
console.error('Failed to clean up temp path:', e);
|
||||
}
|
||||
}
|
||||
return res.status(404).json({ error: 'Event not found' });
|
||||
}
|
||||
|
||||
if (!req.files || req.files.length === 0) {
|
||||
console.error('No files in request. req.files:', req.files);
|
||||
console.error('Request body keys:', Object.keys(req.body));
|
||||
// Clean up temp files
|
||||
if (req.tempUploadPath) {
|
||||
try {
|
||||
await fs.rm(req.tempUploadPath, { recursive: true, force: true });
|
||||
} catch (e) {
|
||||
console.error('Failed to clean up temp path:', e);
|
||||
}
|
||||
}
|
||||
return res.status(400).json({ error: 'No files uploaded' });
|
||||
}
|
||||
|
||||
@@ -138,15 +165,27 @@ router.post('/:eventId/upload', adminAuth, (req, res, next) => {
|
||||
if (parsedCategoryId) {
|
||||
category = await db('photo_categories').where({ id: parsedCategoryId }).first();
|
||||
if (!category) {
|
||||
// Clean up temp files
|
||||
if (req.tempUploadPath) {
|
||||
try {
|
||||
await fs.rm(req.tempUploadPath, { recursive: true, force: true });
|
||||
} catch (e) {
|
||||
console.error('Failed to clean up temp path:', e);
|
||||
}
|
||||
}
|
||||
return res.status(400).json({ error: 'Invalid category' });
|
||||
}
|
||||
}
|
||||
|
||||
// Create final destination directory
|
||||
const finalDestPath = path.join(getStoragePath(), 'events/active', event.slug);
|
||||
await fs.mkdir(finalDestPath, { recursive: true });
|
||||
|
||||
const uploadedPhotos = [];
|
||||
const errors = [];
|
||||
|
||||
// Process files in batches to optimize database operations
|
||||
const BATCH_SIZE = 10; // Process 10 files at a time for database operations
|
||||
const BATCH_SIZE = 25; // Increased batch size for better performance with large uploads
|
||||
|
||||
for (let i = 0; i < req.files.length; i += BATCH_SIZE) {
|
||||
const batch = req.files.slice(i, i + BATCH_SIZE);
|
||||
@@ -169,16 +208,25 @@ router.post('/:eventId/upload', adminAuth, (req, res, next) => {
|
||||
.whereNull('category_id')
|
||||
.count('id as count')
|
||||
.first();
|
||||
batchCounter = (uncategorizedCount.count || 0) + 1;
|
||||
batchCounter = (parseInt(uncategorizedCount.count) || 0) + 1;
|
||||
}
|
||||
|
||||
const batchPhotos = [];
|
||||
const fileRenameOperations = []; // Store rename operations to do after commit
|
||||
|
||||
// First pass: prepare data and move files from temp to final location
|
||||
for (let fileIndex = 0; fileIndex < batch.length; fileIndex++) {
|
||||
const file = batch[fileIndex];
|
||||
const counter = batchCounter + fileIndex;
|
||||
const tempPath = file.path; // Original temp path
|
||||
|
||||
try {
|
||||
// Verify file is complete before processing
|
||||
const tempStats = await fs.stat(tempPath);
|
||||
if (tempStats.size === 0) {
|
||||
throw new Error('File is empty - upload may have been interrupted');
|
||||
}
|
||||
|
||||
// Generate new filename
|
||||
const extension = path.extname(file.originalname);
|
||||
const newFilename = generatePhotoFilename(
|
||||
@@ -188,77 +236,142 @@ router.post('/:eventId/upload', adminAuth, (req, res, next) => {
|
||||
extension
|
||||
);
|
||||
|
||||
// Rename the file
|
||||
const oldPath = file.path;
|
||||
const newPath = path.join(path.dirname(oldPath), newFilename);
|
||||
await fs.rename(oldPath, newPath);
|
||||
|
||||
// Update file object
|
||||
file.filename = newFilename;
|
||||
file.path = newPath;
|
||||
|
||||
// Generate thumbnail with new filename
|
||||
const thumbnailPath = await generateThumbnail(file.path);
|
||||
|
||||
// Calculate relative paths
|
||||
// Calculate final path
|
||||
const finalPath = path.join(finalDestPath, newFilename);
|
||||
const storagePath = getStoragePath();
|
||||
const relativePath = path.relative(path.join(storagePath, 'events/active'), file.path);
|
||||
const relativeThumbPath = thumbnailPath;
|
||||
const relativePath = path.relative(path.join(storagePath, 'events/active'), finalPath);
|
||||
|
||||
// Prepare photo data for batch insert
|
||||
batchPhotos.push({
|
||||
event_id: eventId,
|
||||
filename: file.filename,
|
||||
const photoData = {
|
||||
event_id: parseInt(eventId),
|
||||
filename: newFilename,
|
||||
path: relativePath,
|
||||
thumbnail_path: relativeThumbPath,
|
||||
category_id: parsedCategoryId || null,
|
||||
thumbnail_path: null, // Will generate after successful commit
|
||||
category_id: parsedCategoryId ? parseInt(parsedCategoryId) : null,
|
||||
type: 'individual',
|
||||
size_bytes: file.size
|
||||
size_bytes: tempStats.size // Use actual file size from stat
|
||||
};
|
||||
|
||||
batchPhotos.push(photoData);
|
||||
|
||||
// Store move operation for later
|
||||
fileRenameOperations.push({
|
||||
tempPath: tempPath,
|
||||
finalPath: finalPath,
|
||||
filename: newFilename,
|
||||
photoData: photoData
|
||||
});
|
||||
} catch (error) {
|
||||
console.error(`Error processing file ${file.originalname}:`, error);
|
||||
console.error(`Error preparing file ${file.originalname}:`, error);
|
||||
errors.push({ filename: file.originalname, error: error.message });
|
||||
// Delete the file if it was partially processed
|
||||
if (file.path) {
|
||||
try { await fs.unlink(file.path); } catch (e) {}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Batch insert all photos from this batch
|
||||
// Insert all photos in this batch
|
||||
if (batchPhotos.length > 0) {
|
||||
console.log(`Inserting batch of ${batchPhotos.length} photos with category_id: ${parsedCategoryId}`);
|
||||
|
||||
const insertedIds = await trx('photos').insert(batchPhotos).returning('id');
|
||||
|
||||
// Update category counter if needed
|
||||
if (category) {
|
||||
if (category && parsedCategoryId) {
|
||||
const newCounter = batchCounter + batchPhotos.length - 1;
|
||||
await trx('photo_categories')
|
||||
.where({ id: parsedCategoryId })
|
||||
.update({ photo_counter: batchCounter + batchPhotos.length - 1 });
|
||||
.update({ photo_counter: newCounter });
|
||||
console.log(`Updated category ${parsedCategoryId} counter to ${newCounter}`);
|
||||
}
|
||||
|
||||
// Add to uploaded photos array
|
||||
batchPhotos.forEach((photo, index) => {
|
||||
uploadedPhotos.push({
|
||||
id: insertedIds[index]?.id || insertedIds[index],
|
||||
filename: photo.filename,
|
||||
size: photo.size_bytes,
|
||||
category_id: photo.category_id
|
||||
});
|
||||
});
|
||||
// Commit the transaction first
|
||||
await trx.commit();
|
||||
console.log(`Successfully committed batch of ${batchPhotos.length} photos`);
|
||||
|
||||
// Now move files from temp to final location after successful commit
|
||||
for (let idx = 0; idx < fileRenameOperations.length; idx++) {
|
||||
const operation = fileRenameOperations[idx];
|
||||
try {
|
||||
// Move the file from temp to final location
|
||||
await fs.rename(operation.tempPath, operation.finalPath);
|
||||
console.log(`Moved file from ${operation.tempPath} to ${operation.finalPath}`);
|
||||
|
||||
// Verify the file was moved successfully
|
||||
const finalStats = await fs.stat(operation.finalPath);
|
||||
if (finalStats.size !== operation.photoData.size_bytes) {
|
||||
throw new Error(`File size mismatch after move: expected ${operation.photoData.size_bytes}, got ${finalStats.size}`);
|
||||
}
|
||||
|
||||
// Generate thumbnail with final path
|
||||
let thumbnailPath = null;
|
||||
try {
|
||||
thumbnailPath = await generateThumbnail(operation.finalPath);
|
||||
|
||||
// Update the database with thumbnail path
|
||||
if (thumbnailPath && insertedIds[idx]) {
|
||||
const photoId = insertedIds[idx]?.id || insertedIds[idx];
|
||||
await db('photos')
|
||||
.where({ id: photoId })
|
||||
.update({ thumbnail_path: thumbnailPath });
|
||||
}
|
||||
} catch (thumbError) {
|
||||
console.error(`Thumbnail generation failed for ${operation.filename}:`, thumbError.message);
|
||||
}
|
||||
|
||||
// Add to successful uploads
|
||||
uploadedPhotos.push({
|
||||
id: insertedIds[idx]?.id || insertedIds[idx],
|
||||
filename: operation.filename,
|
||||
size: operation.photoData.size_bytes,
|
||||
category_id: operation.photoData.category_id
|
||||
});
|
||||
} catch (moveError) {
|
||||
console.error(`Failed to move file ${operation.tempPath} to ${operation.finalPath}:`, moveError);
|
||||
errors.push({
|
||||
filename: operation.filename,
|
||||
error: `File move failed: ${moveError.message}`
|
||||
});
|
||||
|
||||
// Try to clean up the database entry if file move failed
|
||||
if (insertedIds[idx]) {
|
||||
const photoId = insertedIds[idx]?.id || insertedIds[idx];
|
||||
try {
|
||||
await db('photos').where({ id: photoId }).delete();
|
||||
console.log(`Cleaned up database entry for failed photo ${photoId}`);
|
||||
} catch (cleanupError) {
|
||||
console.error(`Failed to clean up database entry:`, cleanupError);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// No photos to insert, just rollback
|
||||
await trx.rollback();
|
||||
}
|
||||
|
||||
// Commit the batch transaction
|
||||
await trx.commit();
|
||||
} catch (error) {
|
||||
console.error(`Error processing batch starting at index ${i}:`, error);
|
||||
await trx.rollback();
|
||||
console.error('Stack trace:', error.stack);
|
||||
|
||||
// Try to clean up files from failed batch
|
||||
for (const file of batch) {
|
||||
if (file.path) {
|
||||
try { await fs.unlink(file.path); } catch (e) {}
|
||||
}
|
||||
// Rollback if not already committed
|
||||
if (!trx.isCompleted()) {
|
||||
await trx.rollback();
|
||||
}
|
||||
|
||||
// Add all files in this batch to errors
|
||||
for (const file of batch) {
|
||||
errors.push({
|
||||
filename: file.originalname,
|
||||
error: `Batch processing failed: ${error.message}`
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Clean up temp upload directory
|
||||
if (req.tempUploadPath) {
|
||||
try {
|
||||
await fs.rm(req.tempUploadPath, { recursive: true, force: true });
|
||||
console.log(`Cleaned up temp upload directory: ${req.tempUploadPath}`);
|
||||
} catch (e) {
|
||||
console.error('Failed to clean up temp upload directory:', e);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -269,24 +382,39 @@ router.post('/:eventId/upload', adminAuth, (req, res, next) => {
|
||||
{ type: 'admin', id: req.admin.id, name: req.admin.username }
|
||||
);
|
||||
|
||||
// Include any files that were invalid from the validation middleware
|
||||
const totalInvalidFiles = (req.invalidFiles || []).concat(errors);
|
||||
|
||||
// Prepare response
|
||||
const totalAttempted = req.files.length + (req.invalidFiles ? req.invalidFiles.length : 0);
|
||||
const response = {
|
||||
message: `Successfully uploaded ${uploadedPhotos.length} photos`,
|
||||
photos: uploadedPhotos,
|
||||
totalFiles: req.files.length,
|
||||
totalFiles: totalAttempted,
|
||||
successCount: uploadedPhotos.length,
|
||||
failureCount: errors.length
|
||||
failureCount: totalInvalidFiles.length
|
||||
};
|
||||
|
||||
// Include error details if any files failed
|
||||
if (errors.length > 0) {
|
||||
response.errors = errors;
|
||||
response.message = `Uploaded ${uploadedPhotos.length} of ${req.files.length} photos. ${errors.length} failed.`;
|
||||
if (totalInvalidFiles.length > 0) {
|
||||
response.errors = totalInvalidFiles;
|
||||
response.message = `Uploaded ${uploadedPhotos.length} of ${totalAttempted} photos. ${totalInvalidFiles.length} failed.`;
|
||||
}
|
||||
|
||||
res.json(response);
|
||||
} catch (error) {
|
||||
console.error('Error uploading photos:', error);
|
||||
|
||||
// Clean up temp upload directory on error
|
||||
if (req.tempUploadPath) {
|
||||
try {
|
||||
await fs.rm(req.tempUploadPath, { recursive: true, force: true });
|
||||
console.log(`Cleaned up temp upload directory after error: ${req.tempUploadPath}`);
|
||||
} catch (e) {
|
||||
console.error('Failed to clean up temp upload directory:', e);
|
||||
}
|
||||
}
|
||||
|
||||
res.status(500).json({ error: 'Failed to upload photos' });
|
||||
}
|
||||
});
|
||||
@@ -319,9 +447,14 @@ router.delete('/:eventId/photos/:photoId', adminAuth, async (req, res) => {
|
||||
if (photo.thumbnail_path) {
|
||||
const thumbPath = path.join(storagePath, 'events/active', photo.thumbnail_path);
|
||||
try {
|
||||
// Check if file exists before attempting to delete
|
||||
await fs.access(thumbPath);
|
||||
await fs.unlink(thumbPath);
|
||||
} catch (error) {
|
||||
console.error('Error deleting thumbnail:', error);
|
||||
// Only log if it's not a "file not found" error
|
||||
if (error.code !== 'ENOENT') {
|
||||
console.error('Error deleting thumbnail:', error);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -406,9 +539,14 @@ router.post('/:eventId/photos/bulk-delete', adminAuth, async (req, res) => {
|
||||
if (photo.thumbnail_path) {
|
||||
const thumbPath = path.join(storagePath, photo.thumbnail_path);
|
||||
try {
|
||||
// Check if file exists before attempting to delete
|
||||
await fs.access(thumbPath);
|
||||
await fs.unlink(thumbPath);
|
||||
} catch (error) {
|
||||
console.error('Error deleting thumbnail:', error);
|
||||
// Only log if it's not a "file not found" error
|
||||
if (error.code !== 'ENOENT') {
|
||||
console.error('Error deleting thumbnail:', error);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -613,26 +751,24 @@ router.get('/:eventId/thumbnail/:photoId', adminAuth, async (req, res) => {
|
||||
.where({ id: photoId, event_id: eventId })
|
||||
.first();
|
||||
|
||||
if (!photo || !photo.thumbnail_path) {
|
||||
console.error(`Thumbnail not found for photo ${photoId}, event ${eventId}`);
|
||||
return res.status(404).json({ error: 'Thumbnail not found' });
|
||||
if (!photo) {
|
||||
console.error(`Photo not found: ${photoId}, event ${eventId}`);
|
||||
return res.status(404).json({ error: 'Photo not found' });
|
||||
}
|
||||
|
||||
// Ensure thumbnail exists and is valid, regenerate if needed
|
||||
const thumbnailPath = await ensureThumbnail(photo);
|
||||
|
||||
if (!thumbnailPath) {
|
||||
console.error(`Failed to generate thumbnail for photo ${photoId}`);
|
||||
return res.status(404).json({ error: 'Thumbnail generation failed' });
|
||||
}
|
||||
|
||||
const storagePath = getStoragePath();
|
||||
const filePath = path.join(storagePath, photo.thumbnail_path);
|
||||
|
||||
console.log(`Attempting to serve thumbnail: ${filePath}`);
|
||||
|
||||
// Check if file exists
|
||||
try {
|
||||
await fs.access(filePath);
|
||||
} catch (error) {
|
||||
console.error(`Thumbnail file not found: ${filePath}`, error);
|
||||
return res.status(404).json({ error: 'Thumbnail file not found' });
|
||||
}
|
||||
const filePath = path.join(storagePath, thumbnailPath);
|
||||
|
||||
// Set appropriate headers
|
||||
res.setHeader('Content-Type', `image/${path.extname(photo.thumbnail_path).slice(1)}`);
|
||||
res.setHeader('Content-Type', 'image/jpeg'); // Thumbnails are always JPEG
|
||||
res.setHeader('Cache-Control', 'private, max-age=3600');
|
||||
res.setHeader('Cross-Origin-Resource-Policy', 'cross-origin');
|
||||
|
||||
|
||||
@@ -0,0 +1,461 @@
|
||||
const express = require('express');
|
||||
const router = express.Router();
|
||||
const { restoreService } = require('../services/restoreService');
|
||||
const { adminAuth } = require('../middleware/auth');
|
||||
const { body, query, validationResult } = require('express-validator');
|
||||
const logger = require('../utils/logger');
|
||||
const { db } = require('../database/db');
|
||||
const path = require('path');
|
||||
const fs = require('fs').promises;
|
||||
|
||||
/**
|
||||
* Admin routes for restore operations
|
||||
* All routes require admin authentication
|
||||
*/
|
||||
|
||||
// Apply admin authentication to all routes
|
||||
router.use(adminAuth);
|
||||
|
||||
/**
|
||||
* Get restore service status and history
|
||||
*/
|
||||
router.get('/status', async (req, res) => {
|
||||
try {
|
||||
const limit = parseInt(req.query.limit) || 10;
|
||||
const history = await restoreService.getRestoreHistory(limit);
|
||||
|
||||
const status = {
|
||||
isRunning: restoreService.isRunning,
|
||||
currentProgress: restoreService.getProgress(),
|
||||
history: history,
|
||||
settings: await getRestoreSettings()
|
||||
};
|
||||
|
||||
res.json({
|
||||
success: true,
|
||||
data: status
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to get restore status:', error);
|
||||
res.status(500).json({
|
||||
success: false,
|
||||
error: 'Failed to get restore status'
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* Validate restore request
|
||||
*/
|
||||
router.post('/validate', [
|
||||
body('source').notEmpty().withMessage('Backup source is required'),
|
||||
body('manifestPath').notEmpty().withMessage('Manifest path is required'),
|
||||
body('restoreType').isIn(['full', 'database', 'files', 'selective']).withMessage('Invalid restore type'),
|
||||
body('selectedItems').optional().isArray(),
|
||||
body('s3Config').optional().isObject()
|
||||
], async (req, res) => {
|
||||
const errors = validationResult(req);
|
||||
if (!errors.isEmpty()) {
|
||||
return res.status(400).json({
|
||||
success: false,
|
||||
errors: errors.array()
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
// Perform dry run validation
|
||||
const result = await restoreService.restore({
|
||||
...req.body,
|
||||
dryRun: true,
|
||||
force: false
|
||||
});
|
||||
|
||||
res.json({
|
||||
success: true,
|
||||
data: {
|
||||
validation: result.validation,
|
||||
spaceCheck: result.spaceCheck,
|
||||
logs: result.logs
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Restore validation failed:', error);
|
||||
res.status(400).json({
|
||||
success: false,
|
||||
error: error.message,
|
||||
logs: restoreService.restoreLog
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* Start restore operation
|
||||
*/
|
||||
router.post('/start', [
|
||||
body('source').notEmpty().withMessage('Backup source is required'),
|
||||
body('manifestPath').notEmpty().withMessage('Manifest path is required'),
|
||||
body('restoreType').isIn(['full', 'database', 'files', 'selective']).withMessage('Invalid restore type'),
|
||||
body('selectedItems').optional().isArray(),
|
||||
body('skipPreBackup').optional().isBoolean(),
|
||||
body('force').optional().isBoolean(),
|
||||
body('s3Config').optional().isObject()
|
||||
], async (req, res) => {
|
||||
const errors = validationResult(req);
|
||||
if (!errors.isEmpty()) {
|
||||
return res.status(400).json({
|
||||
success: false,
|
||||
errors: errors.array()
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
// Check if restore is already running
|
||||
if (restoreService.isRunning) {
|
||||
return res.status(409).json({
|
||||
success: false,
|
||||
error: 'Restore operation already in progress'
|
||||
});
|
||||
}
|
||||
|
||||
// Check permissions for dangerous options
|
||||
const settings = await getRestoreSettings();
|
||||
if (req.body.force && !settings.restore_allow_force) {
|
||||
return res.status(403).json({
|
||||
success: false,
|
||||
error: 'Force restore is not allowed by system settings'
|
||||
});
|
||||
}
|
||||
|
||||
if (req.body.skipPreBackup && settings.restore_require_pre_backup) {
|
||||
return res.status(403).json({
|
||||
success: false,
|
||||
error: 'Skipping pre-restore backup is not allowed by system settings'
|
||||
});
|
||||
}
|
||||
|
||||
// Log restore attempt
|
||||
logger.warn('Restore operation started', {
|
||||
user: req.user.email,
|
||||
ip: req.ip,
|
||||
restoreType: req.body.restoreType,
|
||||
source: req.body.source
|
||||
});
|
||||
|
||||
// Start restore in background
|
||||
restoreService.restore({
|
||||
...req.body,
|
||||
dryRun: false,
|
||||
operator: {
|
||||
type: 'manual',
|
||||
userId: req.user.id,
|
||||
ip: req.ip
|
||||
}
|
||||
}).catch(error => {
|
||||
logger.error('Background restore failed:', error);
|
||||
});
|
||||
|
||||
res.json({
|
||||
success: true,
|
||||
message: 'Restore operation started'
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to start restore:', error);
|
||||
res.status(500).json({
|
||||
success: false,
|
||||
error: error.message
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* Get current restore progress
|
||||
*/
|
||||
router.get('/progress', async (req, res) => {
|
||||
try {
|
||||
const progress = restoreService.getProgress();
|
||||
const logs = restoreService.restoreLog.slice(-50); // Last 50 log entries
|
||||
|
||||
res.json({
|
||||
success: true,
|
||||
data: {
|
||||
isRunning: restoreService.isRunning,
|
||||
progress: progress,
|
||||
logs: logs
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to get restore progress:', error);
|
||||
res.status(500).json({
|
||||
success: false,
|
||||
error: 'Failed to get restore progress'
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* Get restore run details
|
||||
*/
|
||||
router.get('/run/:id', async (req, res) => {
|
||||
try {
|
||||
const run = await db('restore_runs')
|
||||
.where('id', req.params.id)
|
||||
.first();
|
||||
|
||||
if (!run) {
|
||||
return res.status(404).json({
|
||||
success: false,
|
||||
error: 'Restore run not found'
|
||||
});
|
||||
}
|
||||
|
||||
// Parse JSON fields
|
||||
if (run.statistics) run.statistics = JSON.parse(run.statistics);
|
||||
if (run.restore_log) run.restore_log = JSON.parse(run.restore_log);
|
||||
if (run.metadata) run.metadata = JSON.parse(run.metadata);
|
||||
|
||||
// Get validation results
|
||||
const validations = await db('restore_validation_results')
|
||||
.where('restore_run_id', run.id)
|
||||
.select('*');
|
||||
|
||||
validations.forEach(v => {
|
||||
if (v.errors) v.errors = JSON.parse(v.errors);
|
||||
if (v.warnings) v.warnings = JSON.parse(v.warnings);
|
||||
if (v.checksums) v.checksums = JSON.parse(v.checksums);
|
||||
});
|
||||
|
||||
// Get file operations summary
|
||||
const fileOps = await db('restore_file_operations')
|
||||
.where('restore_run_id', run.id)
|
||||
.select('status', db.raw('COUNT(*) as count'))
|
||||
.groupBy('status');
|
||||
|
||||
res.json({
|
||||
success: true,
|
||||
data: {
|
||||
run: run,
|
||||
validations: validations,
|
||||
fileOperations: fileOps
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to get restore run details:', error);
|
||||
res.status(500).json({
|
||||
success: false,
|
||||
error: 'Failed to get restore run details'
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* Get restore run report
|
||||
*/
|
||||
router.get('/run/:id/report', async (req, res) => {
|
||||
try {
|
||||
const run = await db('restore_runs')
|
||||
.where('id', req.params.id)
|
||||
.first();
|
||||
|
||||
if (!run) {
|
||||
return res.status(404).json({
|
||||
success: false,
|
||||
error: 'Restore run not found'
|
||||
});
|
||||
}
|
||||
|
||||
// Parse JSON fields
|
||||
if (run.statistics) run.statistics = JSON.parse(run.statistics);
|
||||
if (run.restore_log) run.restore_log = JSON.parse(run.restore_log);
|
||||
|
||||
// Generate report
|
||||
const report = restoreService.generateRestoreReport({
|
||||
success: run.status === 'completed',
|
||||
duration: run.duration_seconds,
|
||||
dryRun: run.is_dry_run,
|
||||
result: run.statistics,
|
||||
logs: run.restore_log || []
|
||||
});
|
||||
|
||||
res.type('text/plain').send(report);
|
||||
} catch (error) {
|
||||
logger.error('Failed to generate restore report:', error);
|
||||
res.status(500).json({
|
||||
success: false,
|
||||
error: 'Failed to generate restore report'
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* List available backups for restore
|
||||
*/
|
||||
router.get('/available-backups', async (req, res) => {
|
||||
try {
|
||||
const backups = [];
|
||||
|
||||
// Get local file backups
|
||||
const backupConfig = await getBackupConfig();
|
||||
if (backupConfig.backup_destination_type === 'local' && backupConfig.backup_destination_path) {
|
||||
try {
|
||||
const files = await fs.readdir(backupConfig.backup_destination_path);
|
||||
for (const file of files) {
|
||||
if (file.endsWith('.json') || file.endsWith('.yaml')) {
|
||||
const filePath = path.join(backupConfig.backup_destination_path, file);
|
||||
const stats = await fs.stat(filePath);
|
||||
backups.push({
|
||||
type: 'local',
|
||||
name: file,
|
||||
path: filePath,
|
||||
size: stats.size,
|
||||
modified: stats.mtime
|
||||
});
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
logger.warn('Failed to list local backups:', error);
|
||||
}
|
||||
}
|
||||
|
||||
// Get database backups from backup_runs table
|
||||
const backupRuns = await db('backup_runs')
|
||||
.where('status', 'completed')
|
||||
.whereNotNull('manifest_path')
|
||||
.orderBy('completed_at', 'desc')
|
||||
.limit(20);
|
||||
|
||||
for (const run of backupRuns) {
|
||||
backups.push({
|
||||
type: run.manifest_path.startsWith('s3://') ? 's3' : 'local',
|
||||
name: `Backup ${run.completed_at}`,
|
||||
path: run.manifest_path,
|
||||
manifestId: run.manifest_id,
|
||||
size: run.total_size_bytes,
|
||||
filesCount: run.files_backed_up,
|
||||
duration: run.duration_seconds,
|
||||
completed: run.completed_at
|
||||
});
|
||||
}
|
||||
|
||||
res.json({
|
||||
success: true,
|
||||
data: backups
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to list available backups:', error);
|
||||
res.status(500).json({
|
||||
success: false,
|
||||
error: 'Failed to list available backups'
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* Get restore settings
|
||||
*/
|
||||
router.get('/settings', async (req, res) => {
|
||||
try {
|
||||
const settings = await getRestoreSettings();
|
||||
res.json({
|
||||
success: true,
|
||||
data: settings
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to get restore settings:', error);
|
||||
res.status(500).json({
|
||||
success: false,
|
||||
error: 'Failed to get restore settings'
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* Update restore settings
|
||||
*/
|
||||
router.put('/settings', [
|
||||
body('restore_allow_force').optional().isBoolean(),
|
||||
body('restore_require_pre_backup').optional().isBoolean(),
|
||||
body('restore_max_file_size_mb').optional().isInt({ min: 1 }),
|
||||
body('restore_verify_checksums').optional().isBoolean(),
|
||||
body('restore_email_on_completion').optional().isBoolean(),
|
||||
body('restore_retention_days').optional().isInt({ min: 1 })
|
||||
], async (req, res) => {
|
||||
const errors = validationResult(req);
|
||||
if (!errors.isEmpty()) {
|
||||
return res.status(400).json({
|
||||
success: false,
|
||||
errors: errors.array()
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
// Update settings
|
||||
for (const [key, value] of Object.entries(req.body)) {
|
||||
await db('app_settings')
|
||||
.where('setting_key', key)
|
||||
.where('setting_type', 'restore')
|
||||
.update({
|
||||
setting_value: typeof value === 'boolean' ? (value ? '1' : '0') : value.toString(),
|
||||
updated_at: db.fn.now()
|
||||
});
|
||||
}
|
||||
|
||||
logger.info('Restore settings updated', {
|
||||
user: req.user.email,
|
||||
settings: req.body
|
||||
});
|
||||
|
||||
res.json({
|
||||
success: true,
|
||||
message: 'Settings updated successfully'
|
||||
});
|
||||
} catch (error) {
|
||||
logger.error('Failed to update restore settings:', error);
|
||||
res.status(500).json({
|
||||
success: false,
|
||||
error: 'Failed to update settings'
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
/**
|
||||
* Helper function to get restore settings
|
||||
*/
|
||||
async function getRestoreSettings() {
|
||||
const settings = await db('app_settings')
|
||||
.where('setting_type', 'restore')
|
||||
.select('setting_key', 'setting_value');
|
||||
|
||||
const result = {};
|
||||
settings.forEach(setting => {
|
||||
// Convert boolean strings to actual booleans
|
||||
if (setting.setting_value === '1' || setting.setting_value === '0') {
|
||||
result[setting.setting_key] = setting.setting_value === '1';
|
||||
} else {
|
||||
result[setting.setting_key] = setting.setting_value;
|
||||
}
|
||||
});
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Helper function to get backup configuration
|
||||
*/
|
||||
async function getBackupConfig() {
|
||||
const settings = await db('app_settings')
|
||||
.where('setting_type', 'backup')
|
||||
.select('setting_key', 'setting_value');
|
||||
|
||||
const config = {};
|
||||
settings.forEach(setting => {
|
||||
try {
|
||||
config[setting.setting_key] = JSON.parse(setting.setting_value);
|
||||
} catch (e) {
|
||||
config[setting.setting_key] = setting.setting_value;
|
||||
}
|
||||
});
|
||||
|
||||
return config;
|
||||
}
|
||||
|
||||
module.exports = router;
|
||||
@@ -7,6 +7,7 @@ const { db, logActivity } = require('../database/db');
|
||||
const { formatBoolean } = require('../utils/dbCompat');
|
||||
const { adminAuth } = require('../middleware/auth');
|
||||
const { clearMaintenanceCache } = require('../middleware/maintenance');
|
||||
const { clearSettingsCache } = require('../services/rateLimitService');
|
||||
const router = express.Router();
|
||||
|
||||
// Configure multer for logo uploads
|
||||
@@ -495,6 +496,43 @@ router.put('/security', adminAuth, async (req, res) => {
|
||||
}
|
||||
});
|
||||
|
||||
// Update analytics settings
|
||||
router.put('/analytics', adminAuth, async (req, res) => {
|
||||
try {
|
||||
const settings = req.body;
|
||||
|
||||
// Update or insert each setting
|
||||
for (const [key, value] of Object.entries(settings)) {
|
||||
await db('app_settings')
|
||||
.insert({
|
||||
setting_key: key,
|
||||
setting_value: JSON.stringify(value),
|
||||
setting_type: 'analytics',
|
||||
updated_at: new Date()
|
||||
})
|
||||
.onConflict('setting_key')
|
||||
.merge({
|
||||
setting_value: JSON.stringify(value),
|
||||
updated_at: new Date()
|
||||
});
|
||||
}
|
||||
|
||||
// Log activity
|
||||
await db('activity_logs').insert({
|
||||
activity_type: 'analytics_settings_updated',
|
||||
actor_type: 'admin',
|
||||
actor_id: req.admin.id,
|
||||
actor_name: req.admin.username,
|
||||
metadata: JSON.stringify({ settings_count: Object.keys(settings).length })
|
||||
});
|
||||
|
||||
res.json({ message: 'Analytics settings updated successfully' });
|
||||
} catch (error) {
|
||||
console.error('Analytics settings update error:', error);
|
||||
res.status(500).json({ error: 'Failed to update analytics settings' });
|
||||
}
|
||||
});
|
||||
|
||||
// Get storage info
|
||||
router.get('/storage/info', adminAuth, async (req, res) => {
|
||||
try {
|
||||
@@ -520,11 +558,15 @@ router.get('/storage/info', adminAuth, async (req, res) => {
|
||||
|
||||
let archiveStorage = 0;
|
||||
for (const archive of archives) {
|
||||
try {
|
||||
const stats = await fs.stat(archive.archive_path);
|
||||
archiveStorage += stats.size;
|
||||
} catch (error) {
|
||||
console.error('Archive file not found:', archive.archive_path);
|
||||
if (archive.archive_path) {
|
||||
try {
|
||||
const storagePath = process.env.STORAGE_PATH || path.join(__dirname, '../../../storage');
|
||||
const fullArchivePath = path.join(storagePath, archive.archive_path);
|
||||
const stats = await fs.stat(fullArchivePath);
|
||||
archiveStorage += stats.size;
|
||||
} catch (error) {
|
||||
console.error('Archive file not found:', archive.archive_path, error.message);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -578,4 +620,68 @@ router.post('/favicon', adminAuth, faviconUpload.single('favicon'), async (req,
|
||||
}
|
||||
});
|
||||
|
||||
// Update rate limit settings
|
||||
router.put('/security/rate-limit', adminAuth, [
|
||||
body('rate_limit_enabled').isBoolean().withMessage('Enabled must be a boolean'),
|
||||
body('rate_limit_window_minutes').isInt({ min: 1, max: 60 }).withMessage('Window must be between 1 and 60 minutes'),
|
||||
body('rate_limit_max_requests').isInt({ min: 10, max: 10000 }).withMessage('Max requests must be between 10 and 10000'),
|
||||
body('rate_limit_auth_max_requests').isInt({ min: 1, max: 100 }).withMessage('Auth max requests must be between 1 and 100'),
|
||||
body('rate_limit_skip_authenticated').isBoolean().withMessage('Skip authenticated must be a boolean'),
|
||||
body('rate_limit_public_endpoints_only').isBoolean().withMessage('Public endpoints only must be a boolean')
|
||||
], async (req, res) => {
|
||||
try {
|
||||
const errors = validationResult(req);
|
||||
if (!errors.isEmpty()) {
|
||||
return res.status(400).json({ errors: errors.array() });
|
||||
}
|
||||
|
||||
const {
|
||||
rate_limit_enabled,
|
||||
rate_limit_window_minutes,
|
||||
rate_limit_max_requests,
|
||||
rate_limit_auth_max_requests,
|
||||
rate_limit_skip_authenticated,
|
||||
rate_limit_public_endpoints_only
|
||||
} = req.body;
|
||||
|
||||
// Update each setting
|
||||
const settings = [
|
||||
{ key: 'rate_limit_enabled', value: rate_limit_enabled },
|
||||
{ key: 'rate_limit_window_minutes', value: rate_limit_window_minutes },
|
||||
{ key: 'rate_limit_max_requests', value: rate_limit_max_requests },
|
||||
{ key: 'rate_limit_auth_max_requests', value: rate_limit_auth_max_requests },
|
||||
{ key: 'rate_limit_skip_authenticated', value: rate_limit_skip_authenticated },
|
||||
{ key: 'rate_limit_public_endpoints_only', value: rate_limit_public_endpoints_only }
|
||||
];
|
||||
|
||||
for (const { key, value } of settings) {
|
||||
await db('app_settings')
|
||||
.where('setting_key', key)
|
||||
.update({
|
||||
setting_value: JSON.stringify(value),
|
||||
updated_at: new Date()
|
||||
});
|
||||
}
|
||||
|
||||
// Clear the rate limit settings cache to apply changes immediately
|
||||
clearSettingsCache();
|
||||
|
||||
// Log activity
|
||||
await logActivity('settings_updated',
|
||||
{
|
||||
category: 'security',
|
||||
subcategory: 'rate_limit',
|
||||
changes: settings.length
|
||||
},
|
||||
null,
|
||||
{ type: 'admin', id: req.admin.id, name: req.admin.username }
|
||||
);
|
||||
|
||||
res.json({ message: 'Rate limit settings updated successfully' });
|
||||
} catch (error) {
|
||||
console.error('Rate limit settings update error:', error);
|
||||
res.status(500).json({ error: 'Failed to update rate limit settings' });
|
||||
}
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user