diff --git a/MIGRATION_ERROR_FIX.md b/MIGRATION_ERROR_FIX.md new file mode 100644 index 0000000..c02d6f6 --- /dev/null +++ b/MIGRATION_ERROR_FIX.md @@ -0,0 +1,111 @@ +# Quick Fix for Migration Error + +## Immediate Fix + +The error "relation photo_categories already exists" occurs because the database already has tables but the migration tracking doesn't know they were applied. + +### Option 1: Use Safe Migration Runner (Recommended) + +Update your `docker-compose.prod.yml` to use the safe migration command: + +```yaml +backend: + environment: + - NODE_ENV=production + # ... other config ... +``` + +Then update the `wait-for-db.sh` (already done) to use `npm run migrate:safe` in production. + +### Option 2: Quick Manual Fix + +If you need to fix the running system immediately: + +```bash +# 1. Enter the backend container +docker-compose -f docker-compose.prod.yml exec backend sh + +# 2. Run the safe migration script +npm run migrate:safe + +# 3. If that fails, manually mark migrations as applied: +docker-compose -f docker-compose.prod.yml exec db psql -U picpeak -d picpeak + +# In PostgreSQL: +CREATE TABLE IF NOT EXISTS migrations ( + id SERIAL PRIMARY KEY, + filename VARCHAR(255) UNIQUE NOT NULL, + applied_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP +); + +-- Mark existing migrations as applied +INSERT INTO migrations (filename) VALUES + ('init.js'), + ('004_add_categories_and_cms.js'), + ('006_add_photo_counter_to_categories.js'), + ('007_add_read_at_to_activity_logs.js'), + ('008_add_language_support_to_email_templates.js'), + ('009_update_german_email_templates.js'), + ('010_add_missing_email_templates.js'), + ('011_add_user_upload_settings.js'), + ('012_add_hero_photo_id.js'), + ('013_fix_email_links_and_date_format.js'), + ('014_add_default_welcome_message.js'), + ('014_add_host_name_to_events.js'), + ('015_add_login_attempts_table.js'), + ('016_add_auth_security_columns.js'), + ('017_add_token_revocation_tables.js') +ON CONFLICT (filename) DO NOTHING; + +\q +``` + +### Option 3: Fresh Start (Nuclear Option) + +If you don't have important data yet: + +```bash +# Stop everything +docker-compose -f docker-compose.prod.yml down + +# Remove database volume +docker volume rm wedding-photo-sharing_postgres_data + +# Start fresh +docker-compose -f docker-compose.prod.yml up -d +``` + +## Root Cause + +The issue happens when: +1. Database volume persists between deployments +2. Migration tracking table gets out of sync +3. The original migration runner doesn't check for existing tables + +## Permanent Solution + +The new safe migration runner (`migrate:safe`) handles this by: +1. Checking if tables exist before creating them +2. Catching "already exists" errors gracefully +3. Auto-detecting existing schema and marking migrations as applied + +## Next Steps + +After fixing the migration issue: + +1. Create admin user: +```bash +docker-compose -f docker-compose.prod.yml exec backend node scripts/create-admin.js \ + --username admin \ + --email admin@yourdomain.com +``` + +2. Check health: +```bash +curl http://yourdomain.com/api/health +``` + +3. Monitor logs: +```bash +docker-compose -f docker-compose.prod.yml logs -f backend +``` \ No newline at end of file diff --git a/PRODUCTION_DEPLOYMENT_GUIDE.md b/PRODUCTION_DEPLOYMENT_GUIDE.md new file mode 100644 index 0000000..e60ffd7 --- /dev/null +++ b/PRODUCTION_DEPLOYMENT_GUIDE.md @@ -0,0 +1,312 @@ +# Production Deployment Guide + +This guide addresses all known production deployment issues and provides solutions. + +## Pre-Deployment Checklist + +### 1. Environment Variables +Create a `.env` file with ALL required variables: + +```bash +# Required +JWT_SECRET= +DB_PASSWORD= +ADMIN_URL=https://yourdomain.com +FRONTEND_URL=https://yourdomain.com + +# Database +DB_USER=picpeak +DB_NAME=picpeak + +# Email (Optional but recommended) +SMTP_HOST=smtp.gmail.com +SMTP_PORT=587 +SMTP_SECURE=false +SMTP_USER=your-email@gmail.com +SMTP_PASS=your-app-password +EMAIL_FROM=noreply@yourdomain.com + +# Umami Analytics (Optional) +UMAMI_URL=https://analytics.yourdomain.com +UMAMI_WEBSITE_ID=your-website-id +UMAMI_HASH_SALT= +``` + +### 2. Generate Secrets + +```bash +# Generate JWT Secret +openssl rand -base64 32 + +# Generate Database Password +openssl rand -base64 24 + +# Generate Umami Hash Salt +openssl rand -hex 32 +``` + +## Deployment Steps + +### 1. Initial Setup + +```bash +# Clone repository +git clone https://github.com/yourusername/wedding-photo-sharing.git +cd wedding-photo-sharing + +# Create required directories +mkdir -p storage/events/active storage/events/archived storage/thumbnails storage/uploads +mkdir -p data logs +mkdir -p certbot/conf certbot/www + +# Set permissions (important!) +chmod -R 755 storage data logs +``` + +### 2. Fix Docker Volume Permissions + +Create `docker-compose.override.yml` for local volume configuration: + +```yaml +version: '3.8' + +services: + backend: + volumes: + - ./storage:/app/storage:delegated + - ./data:/app/data:delegated + - ./logs:/app/logs:delegated + user: "1001:1001" # nodejs user + + db: + volumes: + - ./postgres-data:/var/lib/postgresql/data +``` + +### 3. Build and Deploy + +```bash +# Build images +docker-compose -f docker-compose.prod.yml build + +# Start services +docker-compose -f docker-compose.prod.yml up -d + +# Check logs +docker-compose -f docker-compose.prod.yml logs -f backend +``` + +### 4. Create Admin User + +After deployment, create the first admin user: + +```bash +# Enter backend container +docker-compose -f docker-compose.prod.yml exec backend sh + +# Create admin +node scripts/create-admin.js \ + --username admin \ + --email admin@yourdomain.com \ + --password + +# Exit container +exit +``` + +### 5. Configure Email (if using database config) + +1. Login to admin panel: https://yourdomain.com/admin +2. Go to Settings > Email Configuration +3. Enter SMTP details +4. Test email sending + +## Common Issues and Solutions + +### Issue 1: Migration Failures + +**Error**: "relation already exists" + +**Solution**: The safe migration runner handles this automatically. If issues persist: + +```bash +# Reset migrations tracking +docker-compose -f docker-compose.prod.yml exec db psql -U picpeak -d picpeak + +# In PostgreSQL: +DROP TABLE IF EXISTS migrations; +\q + +# Re-run migrations +docker-compose -f docker-compose.prod.yml exec backend npm run migrate:safe +``` + +### Issue 2: Permission Denied Errors + +**Error**: "EACCES: permission denied" + +**Solution**: Fix container permissions: + +```bash +# Stop containers +docker-compose -f docker-compose.prod.yml down + +# Fix permissions on host +sudo chown -R 1001:1001 storage data logs + +# Restart +docker-compose -f docker-compose.prod.yml up -d +``` + +### Issue 3: Database Connection Failed + +**Error**: "no pg_hba.conf entry" + +**Solution**: Already fixed in docker-compose.prod.yml with: +- SSL disabled for internal Docker network +- Proper authentication method (scram-sha-256) + +### Issue 4: Frontend Can't Connect to Backend + +**Error**: CORS errors or connection refused + +**Solution**: Ensure environment variables match: +- Backend: `FRONTEND_URL` must match your frontend URL +- Frontend: `VITE_API_URL` must be set during build + +### Issue 5: Email Not Sending + +**Solution**: Check email configuration: + +```bash +# Check backend logs +docker-compose -f docker-compose.prod.yml logs backend | grep email + +# Verify SMTP settings +# Gmail users: Use app password, not regular password +# Enable "Less secure app access" or use OAuth2 +``` + +## SSL/HTTPS Setup + +1. Update `nginx/sites-enabled/default` with your domain +2. Run certbot: + +```bash +# Initial certificate +docker-compose -f docker-compose.prod.yml run --rm certbot certonly \ + --webroot --webroot-path=/var/www/certbot \ + -d yourdomain.com -d www.yourdomain.com + +# Auto-renewal is handled by the certbot container +``` + +## Monitoring + +### Health Checks + +```bash +# Backend health +curl http://localhost/api/health + +# Database connection +docker-compose -f docker-compose.prod.yml exec backend \ + psql -U picpeak -d picpeak -c "SELECT 1" +``` + +### Logs + +```bash +# All services +docker-compose -f docker-compose.prod.yml logs -f + +# Specific service +docker-compose -f docker-compose.prod.yml logs -f backend +``` + +## Backup and Restore + +### Backup + +```bash +#!/bin/bash +# backup.sh +DATE=$(date +%Y%m%d_%H%M%S) +BACKUP_DIR="./backups/$DATE" + +mkdir -p $BACKUP_DIR + +# Database +docker-compose -f docker-compose.prod.yml exec -T db \ + pg_dump -U picpeak picpeak > $BACKUP_DIR/database.sql + +# Files +tar -czf $BACKUP_DIR/storage.tar.gz storage/ + +echo "Backup completed: $BACKUP_DIR" +``` + +### Restore + +```bash +# Database +docker-compose -f docker-compose.prod.yml exec -T db \ + psql -U picpeak picpeak < ./backups/20240713_120000/database.sql + +# Files +tar -xzf ./backups/20240713_120000/storage.tar.gz +``` + +## Production Best Practices + +1. **Always use named volumes** in production for better data persistence +2. **Set up monitoring** with Prometheus/Grafana +3. **Enable backups** with automated scripts +4. **Use a reverse proxy** (Nginx) for SSL termination +5. **Implement rate limiting** at the Nginx level +6. **Regular updates** - Keep Docker images updated +7. **Log rotation** - Configure log rotation for application logs + +## Troubleshooting Commands + +```bash +# Check running containers +docker-compose -f docker-compose.prod.yml ps + +# Restart a service +docker-compose -f docker-compose.prod.yml restart backend + +# View real-time logs +docker-compose -f docker-compose.prod.yml logs -f --tail=100 + +# Execute commands in container +docker-compose -f docker-compose.prod.yml exec backend sh + +# Database shell +docker-compose -f docker-compose.prod.yml exec db psql -U picpeak + +# Clean restart +docker-compose -f docker-compose.prod.yml down +docker-compose -f docker-compose.prod.yml up -d +``` + +## Security Checklist + +- [ ] Strong JWT_SECRET (min 32 chars) +- [ ] Strong database password +- [ ] SSL/HTTPS enabled +- [ ] Firewall configured (only 80/443 open) +- [ ] Regular security updates +- [ ] Backup encryption +- [ ] Access logs monitored +- [ ] Rate limiting enabled +- [ ] File upload restrictions configured + +## Support + +For issues not covered here: +1. Check application logs +2. Review error messages carefully +3. Ensure all environment variables are set +4. Verify file permissions +5. Check Docker daemon logs \ No newline at end of file diff --git a/backend/init-production.sh b/backend/init-production.sh new file mode 100755 index 0000000..d3fb599 --- /dev/null +++ b/backend/init-production.sh @@ -0,0 +1,50 @@ +#!/bin/sh +# init-production.sh - Production initialization script + +set -e + +echo "šŸš€ Initializing PicPeak Production Environment..." + +# Wait for services to be ready +echo "ā³ Waiting for database to be fully ready..." +sleep 3 + +# Fix permissions if running as root (shouldn't happen with proper Dockerfile) +if [ "$(id -u)" = "0" ]; then + echo "šŸ”§ Fixing file permissions..." + chown -R nodejs:nodejs /app/storage /app/data /app/logs 2>/dev/null || true +fi + +# Create required directories +echo "šŸ“ Creating required directories..." +mkdir -p /app/storage/events/active \ + /app/storage/events/archived \ + /app/storage/thumbnails \ + /app/storage/uploads/logos \ + /app/storage/uploads/favicons \ + /app/data \ + /app/logs + +# Run migrations with safe runner +echo "šŸ—„ļø Running database migrations (safe mode)..." +NODE_ENV=production npm run migrate:safe + +# Create admin user if environment variables are set +if [ -n "$ADMIN_EMAIL" ] && [ -n "$ADMIN_PASSWORD" ]; then + echo "šŸ‘¤ Creating admin user..." + node scripts/create-admin.js \ + --email "$ADMIN_EMAIL" \ + --username "${ADMIN_USERNAME:-admin}" \ + --password "$ADMIN_PASSWORD" || echo "Admin user might already exist" +fi + +# Initialize email configuration if variables are set +if [ -n "$SMTP_HOST" ]; then + echo "šŸ“§ Email configuration detected via environment variables" +fi + +echo "āœ… Production initialization complete!" +echo "🌐 Starting application server..." + +# Start the application +exec node server.js \ No newline at end of file diff --git a/backend/migrations/helpers.js b/backend/migrations/helpers.js new file mode 100644 index 0000000..a2227cd --- /dev/null +++ b/backend/migrations/helpers.js @@ -0,0 +1,83 @@ +/** + * Migration helper functions for production-safe migrations + */ + +/** + * Create a table only if it doesn't already exist + */ +async function createTableIfNotExists(knex, tableName, callback) { + const exists = await knex.schema.hasTable(tableName); + if (!exists) { + console.log(`Creating table: ${tableName}`); + return knex.schema.createTable(tableName, callback); + } else { + console.log(`Table ${tableName} already exists, skipping...`); + } +} + +/** + * Add column to table only if it doesn't exist + */ +async function addColumnIfNotExists(knex, tableName, columnName, callback) { + const hasColumn = await knex.schema.hasColumn(tableName, columnName); + if (!hasColumn) { + console.log(`Adding column ${columnName} to table ${tableName}`); + return knex.schema.alterTable(tableName, (table) => { + callback(table); + }); + } else { + console.log(`Column ${columnName} already exists in table ${tableName}, skipping...`); + } +} + +/** + * Insert data only if it doesn't already exist + */ +async function insertIfNotExists(knex, tableName, data, uniqueField) { + const exists = await knex(tableName) + .where(uniqueField, data[uniqueField]) + .first(); + + if (!exists) { + console.log(`Inserting ${uniqueField}: ${data[uniqueField]} into ${tableName}`); + return knex(tableName).insert(data); + } else { + console.log(`${uniqueField}: ${data[uniqueField]} already exists in ${tableName}, skipping...`); + } +} + +/** + * Create index only if it doesn't exist + */ +async function createIndexIfNotExists(knex, tableName, columns, indexName) { + // This is database-specific, works for PostgreSQL + if (knex.client.config.client === 'pg') { + const result = await knex.raw(` + SELECT 1 FROM pg_indexes + WHERE tablename = ? AND indexname = ? + `, [tableName, indexName]); + + if (result.rows.length === 0) { + console.log(`Creating index ${indexName} on ${tableName}`); + return knex.schema.alterTable(tableName, (table) => { + table.index(columns, indexName); + }); + } + } else { + // For SQLite, just try to create and ignore errors + try { + await knex.schema.alterTable(tableName, (table) => { + table.index(columns, indexName); + }); + } catch (error) { + // Index probably already exists + } + } +} + +module.exports = { + createTableIfNotExists, + addColumnIfNotExists, + insertIfNotExists, + createIndexIfNotExists +}; \ No newline at end of file diff --git a/backend/migrations/run-migrations-safe.js b/backend/migrations/run-migrations-safe.js new file mode 100644 index 0000000..cda8bba --- /dev/null +++ b/backend/migrations/run-migrations-safe.js @@ -0,0 +1,170 @@ +const fs = require('fs').promises; +const path = require('path'); +const { db } = require('../src/database/db'); + +/** + * Production-safe migration runner that handles existing schema + */ + +// Create or verify migrations tracking table +async function ensureMigrationsTable() { + const tableExists = await db.schema.hasTable('migrations'); + if (!tableExists) { + await db.schema.createTable('migrations', (table) => { + table.increments('id').primary(); + table.string('filename').unique().notNullable(); + table.timestamp('applied_at').defaultTo(db.fn.now()); + }); + console.log('Created migrations tracking table'); + } +} + +// Check if a migration has been applied +async function isMigrationApplied(filename) { + const result = await db('migrations').where('filename', filename).first(); + return !!result; +} + +// Mark migration as applied without running it (for existing schema) +async function markMigrationAsApplied(filename) { + await db('migrations').insert({ filename }); + console.log(`Marked migration ${filename} as applied`); +} + +// Detect existing schema and mark migrations as applied +async function detectExistingSchema() { + console.log('Detecting existing schema...'); + + const tableChecks = [ + { table: 'events', migration: 'init.js' }, + { table: 'photos', migration: 'init.js' }, + { table: 'photo_categories', migration: '004_add_categories_and_cms.js' }, + { table: 'cms_pages', migration: '004_add_categories_and_cms.js' }, + { table: 'login_attempts', migration: '015_add_login_attempts_table.js' }, + { table: 'token_blacklist', migration: '017_add_token_revocation_tables.js' }, + ]; + + for (const check of tableChecks) { + const exists = await db.schema.hasTable(check.table); + if (exists) { + const isApplied = await isMigrationApplied(check.migration); + if (!isApplied) { + await markMigrationAsApplied(check.migration); + } + } + } +} + +// Run a single migration safely +async function runMigrationSafely(filename) { + try { + const migrationPath = path.join(__dirname, filename); + const migration = require(migrationPath); + + if (migration.up) { + console.log(`Running migration: ${filename}`); + + // Run migration in a transaction if possible + if (db.client.config.client === 'pg') { + await db.transaction(async (trx) => { + await migration.up(trx); + }); + } else { + await migration.up(db); + } + + await db('migrations').insert({ filename }); + console.log(`Migration ${filename} completed successfully`); + } + } catch (error) { + // Check if error is because schema already exists + if (error.code === '42P07' || // PostgreSQL: relation already exists + error.code === 'SQLITE_ERROR' && error.message.includes('already exists')) { + console.log(`Migration ${filename} - schema already exists, marking as applied`); + await markMigrationAsApplied(filename); + } else { + throw error; + } + } +} + +// Main migration runner +async function runMigrations() { + let connection; + try { + console.log('Starting production-safe database migrations...'); + + // Ensure database connection is ready + await db.raw('SELECT 1'); + console.log('Database connection verified'); + + // Create migrations tracking table + await ensureMigrationsTable(); + + // Detect and mark existing schema + await detectExistingSchema(); + + // Get all migration files + const files = await fs.readdir(__dirname); + const migrationFiles = files + .filter(f => f.match(/^\d{3}_.*\.js$/) || f === 'init.js') + .sort((a, b) => { + // Ensure init.js runs first + if (a === 'init.js') return -1; + if (b === 'init.js') return 1; + return a.localeCompare(b); + }); + + // Run pending migrations + let pendingCount = 0; + let skippedCount = 0; + + for (const file of migrationFiles) { + const isApplied = await isMigrationApplied(file); + if (!isApplied) { + await runMigrationSafely(file); + pendingCount++; + } else { + skippedCount++; + } + } + + console.log(`\nMigration Summary:`); + console.log(`- Applied: ${pendingCount} migration(s)`); + console.log(`- Skipped: ${skippedCount} migration(s) (already applied)`); + console.log(`- Total: ${migrationFiles.length} migration(s)`); + console.log('\nAll migrations completed successfully'); + + // Close database connection + await db.destroy(); + process.exit(0); + } catch (error) { + console.error('\nāŒ Migration failed:', error.message); + console.error('Error details:', error); + + // Close database connection on error + try { + await db.destroy(); + } catch (e) { + // Ignore + } + + process.exit(1); + } +} + +// Add delay for database readiness in production +async function waitAndRun() { + if (process.env.NODE_ENV === 'production') { + console.log('Waiting 2 seconds for database readiness...'); + await new Promise(resolve => setTimeout(resolve, 2000)); + } + await runMigrations(); +} + +// Only run if called directly +if (require.main === module) { + waitAndRun(); +} + +module.exports = { runMigrations }; \ No newline at end of file diff --git a/backend/package.json b/backend/package.json index 3b5a38a..64ebb36 100644 --- a/backend/package.json +++ b/backend/package.json @@ -7,6 +7,7 @@ "start": "node server.js", "dev": "nodemon server.js", "migrate": "node migrations/run-migrations.js", + "migrate:safe": "node migrations/run-migrations-safe.js", "test": "jest", "lint": "eslint src/" }, diff --git a/backend/wait-for-db.sh b/backend/wait-for-db.sh index 75094df..764ca01 100755 --- a/backend/wait-for-db.sh +++ b/backend/wait-for-db.sh @@ -17,9 +17,13 @@ done >&2 echo "PostgreSQL is up - executing command" -# Run migrations +# Run migrations (use safe runner in production) echo "Running database migrations..." -npm run migrate +if [ "$NODE_ENV" = "production" ]; then + npm run migrate:safe +else + npm run migrate +fi # Execute the main command exec "$@" \ No newline at end of file