Compare commits

...

10 Commits

Author SHA1 Message Date
Gitea Actions Bot 6ebc4f3fc4 chore: bump version to 1.0.10
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-13 21:01:24 +00:00
paul de973f5613 fix: comprehensive production deployment fixes and migration safety
Test and Lint / backend-test (push) Successful in 1m9s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m15s
Version and Release / version-bump (push) Successful in 36s
Version and Release / trigger-drone (push) Successful in 4s
Major fixes for production deployment issues:

1. Migration System:
   - Add safe migration runner that handles existing schema
   - Create migration helper functions for idempotent operations
   - Auto-detect existing tables and mark migrations as applied
   - Handle "relation already exists" errors gracefully

2. Production Initialization:
   - Create init-production.sh script for proper startup sequence
   - Fix directory creation and permissions
   - Add admin user creation from environment variables
   - Ensure proper service initialization order

3. Documentation:
   - Add comprehensive PRODUCTION_DEPLOYMENT_GUIDE.md
   - Add MIGRATION_ERROR_FIX.md for immediate issue resolution
   - Document all known production issues and solutions
   - Include backup/restore procedures

4. Safety Improvements:
   - Add migrate:safe npm script for production use
   - Update wait-for-db.sh to use safe migrations in production
   - Add proper error handling and logging

This resolves the "relation already exists" error and prevents similar
issues in future deployments. The safe migration system can handle both
fresh installations and existing databases.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-13 22:57:22 +02:00
Gitea Actions Bot 279c70b3d6 chore: bump version to 1.0.9
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-07-13 20:46:24 +00:00
paul 5a73f6963f fix: correct YAML syntax in docker-compose.prod.yml
Test and Lint / backend-test (push) Successful in 1m10s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m13s
Version and Release / version-bump (push) Successful in 31s
Version and Release / trigger-drone (push) Successful in 3s
- Remove quotes from POSTGRES_INITDB_ARGS environment variable
- Fix YAML parsing error for PostgreSQL configuration
- Ensure proper formatting for all environment variables

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-13 22:42:27 +02:00
Gitea Actions Bot 5101a05bca chore: bump version to 1.0.8
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
2025-07-13 20:30:04 +00:00
paul c82caf6539 fix: resolve PostgreSQL connection authentication error
Test and Lint / backend-test (push) Successful in 1m8s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m15s
Version and Release / version-bump (push) Successful in 36s
Version and Release / trigger-drone (push) Successful in 2s
- Fix "no pg_hba.conf entry" error by disabling SSL for Docker network
- Use scram-sha-256 authentication method for better security
- Update knexfile.js to support SSL configuration via environment variable
- Add documentation about PostgreSQL connection requirements

The PostgreSQL container now accepts connections from the Docker network
without requiring SSL, which is appropriate for internal container communication.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-13 22:26:00 +02:00
Gitea Actions Bot ae1b508726 chore: bump version to 1.0.7
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-07-13 20:19:10 +00:00
paul 26c05912fc fix: resolve critical production deployment issues
Test and Lint / backend-test (push) Successful in 1m11s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m21s
Version and Release / version-bump (push) Successful in 33s
Version and Release / trigger-drone (push) Successful in 2s
- Fix database connection error "getaddrinfo ENOTFOUND postgres"
- Add wait-for-db.sh script to ensure PostgreSQL is ready before starting
- Fix email processor initialization timing issue
- Add missing storage path environment variables
- Add database dependency to backend service
- Enhance health check endpoint with database connectivity check
- Update production database defaults to match docker-compose
- Install postgresql-client in Docker image for health checks
- Document all required environment variables in .env.example

Fixes immediate production deployment failures and ensures proper service startup order.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-13 22:14:44 +02:00
Gitea Actions Bot d1033cb83a chore: bump version to 1.0.6
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-07-13 20:04:46 +00:00
paul b7458b5a37 fix: add missing pg dependency for PostgreSQL support
Test and Lint / backend-test (push) Successful in 1m10s
continuous-integration/drone/push Build is passing
Test and Lint / frontend-test (push) Successful in 2m7s
Version and Release / version-bump (push) Successful in 36s
Version and Release / trigger-drone (push) Successful in 3s
- Add pg package required for production PostgreSQL connections
- Fixes "Cannot find module 'pg'" error in production deployment

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-13 22:00:41 +02:00
17 changed files with 1086 additions and 56 deletions
+111
View File
@@ -0,0 +1,111 @@
# Quick Fix for Migration Error
## Immediate Fix
The error "relation photo_categories already exists" occurs because the database already has tables but the migration tracking doesn't know they were applied.
### Option 1: Use Safe Migration Runner (Recommended)
Update your `docker-compose.prod.yml` to use the safe migration command:
```yaml
backend:
environment:
- NODE_ENV=production
# ... other config ...
```
Then update the `wait-for-db.sh` (already done) to use `npm run migrate:safe` in production.
### Option 2: Quick Manual Fix
If you need to fix the running system immediately:
```bash
# 1. Enter the backend container
docker-compose -f docker-compose.prod.yml exec backend sh
# 2. Run the safe migration script
npm run migrate:safe
# 3. If that fails, manually mark migrations as applied:
docker-compose -f docker-compose.prod.yml exec db psql -U picpeak -d picpeak
# In PostgreSQL:
CREATE TABLE IF NOT EXISTS migrations (
id SERIAL PRIMARY KEY,
filename VARCHAR(255) UNIQUE NOT NULL,
applied_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
-- Mark existing migrations as applied
INSERT INTO migrations (filename) VALUES
('init.js'),
('004_add_categories_and_cms.js'),
('006_add_photo_counter_to_categories.js'),
('007_add_read_at_to_activity_logs.js'),
('008_add_language_support_to_email_templates.js'),
('009_update_german_email_templates.js'),
('010_add_missing_email_templates.js'),
('011_add_user_upload_settings.js'),
('012_add_hero_photo_id.js'),
('013_fix_email_links_and_date_format.js'),
('014_add_default_welcome_message.js'),
('014_add_host_name_to_events.js'),
('015_add_login_attempts_table.js'),
('016_add_auth_security_columns.js'),
('017_add_token_revocation_tables.js')
ON CONFLICT (filename) DO NOTHING;
\q
```
### Option 3: Fresh Start (Nuclear Option)
If you don't have important data yet:
```bash
# Stop everything
docker-compose -f docker-compose.prod.yml down
# Remove database volume
docker volume rm wedding-photo-sharing_postgres_data
# Start fresh
docker-compose -f docker-compose.prod.yml up -d
```
## Root Cause
The issue happens when:
1. Database volume persists between deployments
2. Migration tracking table gets out of sync
3. The original migration runner doesn't check for existing tables
## Permanent Solution
The new safe migration runner (`migrate:safe`) handles this by:
1. Checking if tables exist before creating them
2. Catching "already exists" errors gracefully
3. Auto-detecting existing schema and marking migrations as applied
## Next Steps
After fixing the migration issue:
1. Create admin user:
```bash
docker-compose -f docker-compose.prod.yml exec backend node scripts/create-admin.js \
--username admin \
--email admin@yourdomain.com
```
2. Check health:
```bash
curl http://yourdomain.com/api/health
```
3. Monitor logs:
```bash
docker-compose -f docker-compose.prod.yml logs -f backend
```
+100
View File
@@ -0,0 +1,100 @@
# Production Deployment Fixes
This document describes the fixes applied to resolve production deployment issues in Docker.
## Issues Fixed
### 1. Database Connection Error: "getaddrinfo ENOTFOUND postgres"
**Problem**: The backend was trying to connect to hostname "postgres" but the database service is named "db" in docker-compose.
**Solution**:
- Updated `knexfile.js` to use correct default host "db" instead of "postgres"
- Added `depends_on: db` to backend service in docker-compose.prod.yml
### 2. Backend Starting Before Database Ready
**Problem**: Backend service started before PostgreSQL was ready, causing connection failures.
**Solution**:
- Created `wait-for-db.sh` script that waits for PostgreSQL to be ready
- Updated Dockerfile to install postgresql-client and use the wait script
- Script also runs migrations automatically on startup
### 3. Email Processor Initialization Failure
**Problem**: Email processor tried to initialize on module load before database was available.
**Solution**:
- Modified `emailProcessor.js` to export initialization functions
- Updated `server.js` to call initialization after database is ready
- Added proper error handling for email service initialization
### 4. Missing Environment Variables
**Problem**: Critical storage path environment variables were missing.
**Solution**:
- Added STORAGE_PATH, EVENTS_PATH, and ARCHIVE_PATH to docker-compose.prod.yml
- Created `.env.example` documenting all required environment variables
### 5. Enhanced Health Check
**Problem**: Basic health check didn't verify database connectivity.
**Solution**:
- Updated `/api/health` endpoint to check database connection
- Returns proper HTTP 503 status when unhealthy
## Files Modified
1. **backend/knexfile.js** - Fixed production database defaults
2. **backend/wait-for-db.sh** - Created database wait script
3. **backend/Dockerfile** - Added postgresql-client and wait script
4. **docker-compose.prod.yml** - Added dependencies and environment variables
5. **backend/src/services/emailProcessor.js** - Disabled auto-initialization
6. **backend/server.js** - Added email initialization and improved health check
7. **backend/.env.example** - Created environment variable documentation
## Deployment Steps
1. Ensure all environment variables are set according to `.env.example`
2. Build and deploy with docker-compose:
```bash
docker-compose -f docker-compose.prod.yml build
docker-compose -f docker-compose.prod.yml up -d
```
3. The backend will now:
- Wait for PostgreSQL to be ready
- Run migrations automatically
- Initialize all services in proper order
- Provide health status at `/api/health`
## Verification
Check deployment health:
```bash
curl http://localhost/api/health
```
Expected response:
```json
{
"status": "ok",
"database": "connected",
"timestamp": "2025-07-13T20:30:00.000Z"
}
```
## Email Configuration
Email service requires configuration in the database. If email is not configured:
- The service will log a warning but continue running
- Emails will be queued but not sent
- Configure email settings in the admin panel after deployment
## PostgreSQL Connection Fix
### Issue: "no pg_hba.conf entry for host"
This error occurs when PostgreSQL requires SSL but the client connects without encryption.
### Solution:
- Disabled SSL requirement for PostgreSQL in Docker environment (`ssl=off`)
- Added proper authentication method (`scram-sha-256`)
- This is acceptable for internal Docker networks where all traffic is isolated
### Security Note:
For production deployments exposed to the internet:
1. Use SSL certificates for PostgreSQL
2. Or ensure the database is only accessible within the Docker network
3. Never expose PostgreSQL port (5432) directly to the internet
+312
View File
@@ -0,0 +1,312 @@
# Production Deployment Guide
This guide addresses all known production deployment issues and provides solutions.
## Pre-Deployment Checklist
### 1. Environment Variables
Create a `.env` file with ALL required variables:
```bash
# Required
JWT_SECRET=<generate-with-openssl-rand-base64-32>
DB_PASSWORD=<strong-password>
ADMIN_URL=https://yourdomain.com
FRONTEND_URL=https://yourdomain.com
# Database
DB_USER=picpeak
DB_NAME=picpeak
# Email (Optional but recommended)
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_SECURE=false
SMTP_USER=your-email@gmail.com
SMTP_PASS=your-app-password
EMAIL_FROM=noreply@yourdomain.com
# Umami Analytics (Optional)
UMAMI_URL=https://analytics.yourdomain.com
UMAMI_WEBSITE_ID=your-website-id
UMAMI_HASH_SALT=<generate-random-string>
```
### 2. Generate Secrets
```bash
# Generate JWT Secret
openssl rand -base64 32
# Generate Database Password
openssl rand -base64 24
# Generate Umami Hash Salt
openssl rand -hex 32
```
## Deployment Steps
### 1. Initial Setup
```bash
# Clone repository
git clone https://github.com/yourusername/wedding-photo-sharing.git
cd wedding-photo-sharing
# Create required directories
mkdir -p storage/events/active storage/events/archived storage/thumbnails storage/uploads
mkdir -p data logs
mkdir -p certbot/conf certbot/www
# Set permissions (important!)
chmod -R 755 storage data logs
```
### 2. Fix Docker Volume Permissions
Create `docker-compose.override.yml` for local volume configuration:
```yaml
version: '3.8'
services:
backend:
volumes:
- ./storage:/app/storage:delegated
- ./data:/app/data:delegated
- ./logs:/app/logs:delegated
user: "1001:1001" # nodejs user
db:
volumes:
- ./postgres-data:/var/lib/postgresql/data
```
### 3. Build and Deploy
```bash
# Build images
docker-compose -f docker-compose.prod.yml build
# Start services
docker-compose -f docker-compose.prod.yml up -d
# Check logs
docker-compose -f docker-compose.prod.yml logs -f backend
```
### 4. Create Admin User
After deployment, create the first admin user:
```bash
# Enter backend container
docker-compose -f docker-compose.prod.yml exec backend sh
# Create admin
node scripts/create-admin.js \
--username admin \
--email admin@yourdomain.com \
--password <your-secure-password>
# Exit container
exit
```
### 5. Configure Email (if using database config)
1. Login to admin panel: https://yourdomain.com/admin
2. Go to Settings > Email Configuration
3. Enter SMTP details
4. Test email sending
## Common Issues and Solutions
### Issue 1: Migration Failures
**Error**: "relation already exists"
**Solution**: The safe migration runner handles this automatically. If issues persist:
```bash
# Reset migrations tracking
docker-compose -f docker-compose.prod.yml exec db psql -U picpeak -d picpeak
# In PostgreSQL:
DROP TABLE IF EXISTS migrations;
\q
# Re-run migrations
docker-compose -f docker-compose.prod.yml exec backend npm run migrate:safe
```
### Issue 2: Permission Denied Errors
**Error**: "EACCES: permission denied"
**Solution**: Fix container permissions:
```bash
# Stop containers
docker-compose -f docker-compose.prod.yml down
# Fix permissions on host
sudo chown -R 1001:1001 storage data logs
# Restart
docker-compose -f docker-compose.prod.yml up -d
```
### Issue 3: Database Connection Failed
**Error**: "no pg_hba.conf entry"
**Solution**: Already fixed in docker-compose.prod.yml with:
- SSL disabled for internal Docker network
- Proper authentication method (scram-sha-256)
### Issue 4: Frontend Can't Connect to Backend
**Error**: CORS errors or connection refused
**Solution**: Ensure environment variables match:
- Backend: `FRONTEND_URL` must match your frontend URL
- Frontend: `VITE_API_URL` must be set during build
### Issue 5: Email Not Sending
**Solution**: Check email configuration:
```bash
# Check backend logs
docker-compose -f docker-compose.prod.yml logs backend | grep email
# Verify SMTP settings
# Gmail users: Use app password, not regular password
# Enable "Less secure app access" or use OAuth2
```
## SSL/HTTPS Setup
1. Update `nginx/sites-enabled/default` with your domain
2. Run certbot:
```bash
# Initial certificate
docker-compose -f docker-compose.prod.yml run --rm certbot certonly \
--webroot --webroot-path=/var/www/certbot \
-d yourdomain.com -d www.yourdomain.com
# Auto-renewal is handled by the certbot container
```
## Monitoring
### Health Checks
```bash
# Backend health
curl http://localhost/api/health
# Database connection
docker-compose -f docker-compose.prod.yml exec backend \
psql -U picpeak -d picpeak -c "SELECT 1"
```
### Logs
```bash
# All services
docker-compose -f docker-compose.prod.yml logs -f
# Specific service
docker-compose -f docker-compose.prod.yml logs -f backend
```
## Backup and Restore
### Backup
```bash
#!/bin/bash
# backup.sh
DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_DIR="./backups/$DATE"
mkdir -p $BACKUP_DIR
# Database
docker-compose -f docker-compose.prod.yml exec -T db \
pg_dump -U picpeak picpeak > $BACKUP_DIR/database.sql
# Files
tar -czf $BACKUP_DIR/storage.tar.gz storage/
echo "Backup completed: $BACKUP_DIR"
```
### Restore
```bash
# Database
docker-compose -f docker-compose.prod.yml exec -T db \
psql -U picpeak picpeak < ./backups/20240713_120000/database.sql
# Files
tar -xzf ./backups/20240713_120000/storage.tar.gz
```
## Production Best Practices
1. **Always use named volumes** in production for better data persistence
2. **Set up monitoring** with Prometheus/Grafana
3. **Enable backups** with automated scripts
4. **Use a reverse proxy** (Nginx) for SSL termination
5. **Implement rate limiting** at the Nginx level
6. **Regular updates** - Keep Docker images updated
7. **Log rotation** - Configure log rotation for application logs
## Troubleshooting Commands
```bash
# Check running containers
docker-compose -f docker-compose.prod.yml ps
# Restart a service
docker-compose -f docker-compose.prod.yml restart backend
# View real-time logs
docker-compose -f docker-compose.prod.yml logs -f --tail=100
# Execute commands in container
docker-compose -f docker-compose.prod.yml exec backend sh
# Database shell
docker-compose -f docker-compose.prod.yml exec db psql -U picpeak
# Clean restart
docker-compose -f docker-compose.prod.yml down
docker-compose -f docker-compose.prod.yml up -d
```
## Security Checklist
- [ ] Strong JWT_SECRET (min 32 chars)
- [ ] Strong database password
- [ ] SSL/HTTPS enabled
- [ ] Firewall configured (only 80/443 open)
- [ ] Regular security updates
- [ ] Backup encryption
- [ ] Access logs monitored
- [ ] Rate limiting enabled
- [ ] File upload restrictions configured
## Support
For issues not covered here:
1. Check application logs
2. Review error messages carefully
3. Ensure all environment variables are set
4. Verify file permissions
5. Check Docker daemon logs
+33 -34
View File
@@ -1,42 +1,41 @@
NODE_ENV=development
# Backend Environment Variables Example
# Copy this file to .env and update with your values
# Application
NODE_ENV=production
PORT=3000
# URLs
ADMIN_URL=http://localhost:3000
FRONTEND_URL=http://localhost:3001
# Security
JWT_SECRET=dev-secret-key
JWT_SECRET=your-very-secure-jwt-secret-at-least-32-characters-long
# Email Configuration
SMTP_HOST=mailhog
SMTP_PORT=1025
SMTP_SECURE=false
SMTP_USER=
SMTP_PASS=
EMAIL_FROM=noreply@localhost
# Storage Paths (relative to project root)
STORAGE_PATH=./storage
EVENTS_PATH=./storage/events
ARCHIVE_PATH=./storage/events/archived
# Logging
LOG_LEVEL=info
# URLs
ADMIN_URL=https://yourdomain.com
FRONTEND_URL=https://yourdomain.com
# Database Configuration
# Development: Use SQLite
DATABASE_CLIENT=sqlite3
DATABASE_PATH=./data/photo_sharing.db
DATABASE_CLIENT=pg
DB_HOST=db
DB_PORT=5432
DB_USER=picpeak
DB_PASSWORD=your-secure-database-password
DB_NAME=picpeak
# Production: Use PostgreSQL
# DATABASE_CLIENT=pg
# DB_HOST=localhost
# DB_PORT=5432
# DB_USER=picpeak
# DB_PASSWORD=your-secure-password
# DB_NAME=picpeak
# Email Configuration
SMTP_HOST=smtp.example.com
SMTP_PORT=587
SMTP_SECURE=false
SMTP_USER=your-smtp-username
SMTP_PASS=your-smtp-password
EMAIL_FROM=noreply@yourdomain.com
# Umami Analytics (optional)
UMAMI_URL=
UMAMI_WEBSITE_ID=
# Storage Paths (Docker)
STORAGE_PATH=/app/storage
EVENTS_PATH=/app/storage/events
ARCHIVE_PATH=/app/storage/events/archived
# Analytics (Optional)
UMAMI_URL=https://analytics.yourdomain.com
UMAMI_WEBSITE_ID=your-website-id
# Logging
LOG_LEVEL=info
+6 -3
View File
@@ -16,8 +16,8 @@ FROM node:18-alpine
WORKDIR /app
# Install dumb-init for proper signal handling
RUN apk add --no-cache dumb-init
# Install dumb-init for proper signal handling and postgresql-client for database checks
RUN apk add --no-cache dumb-init postgresql-client
# Create non-root user
RUN addgroup -g 1001 -S nodejs && adduser -S nodejs -u 1001
@@ -26,6 +26,9 @@ RUN addgroup -g 1001 -S nodejs && adduser -S nodejs -u 1001
COPY --from=builder --chown=nodejs:nodejs /app/node_modules ./node_modules
COPY --chown=nodejs:nodejs . .
# Make wait script executable
RUN chmod +x wait-for-db.sh
# Create necessary directories
RUN mkdir -p storage/events/active storage/events/archived storage/thumbnails data logs && \
chown -R nodejs:nodejs storage data logs
@@ -35,4 +38,4 @@ USER nodejs
EXPOSE 3000
ENTRYPOINT ["dumb-init", "--"]
CMD ["node", "server.js"]
CMD ["./wait-for-db.sh", "node", "server.js"]
+50
View File
@@ -0,0 +1,50 @@
#!/bin/sh
# init-production.sh - Production initialization script
set -e
echo "🚀 Initializing PicPeak Production Environment..."
# Wait for services to be ready
echo "⏳ Waiting for database to be fully ready..."
sleep 3
# Fix permissions if running as root (shouldn't happen with proper Dockerfile)
if [ "$(id -u)" = "0" ]; then
echo "🔧 Fixing file permissions..."
chown -R nodejs:nodejs /app/storage /app/data /app/logs 2>/dev/null || true
fi
# Create required directories
echo "📁 Creating required directories..."
mkdir -p /app/storage/events/active \
/app/storage/events/archived \
/app/storage/thumbnails \
/app/storage/uploads/logos \
/app/storage/uploads/favicons \
/app/data \
/app/logs
# Run migrations with safe runner
echo "🗄️ Running database migrations (safe mode)..."
NODE_ENV=production npm run migrate:safe
# Create admin user if environment variables are set
if [ -n "$ADMIN_EMAIL" ] && [ -n "$ADMIN_PASSWORD" ]; then
echo "👤 Creating admin user..."
node scripts/create-admin.js \
--email "$ADMIN_EMAIL" \
--username "${ADMIN_USERNAME:-admin}" \
--password "$ADMIN_PASSWORD" || echo "Admin user might already exist"
fi
# Initialize email configuration if variables are set
if [ -n "$SMTP_HOST" ]; then
echo "📧 Email configuration detected via environment variables"
fi
echo "✅ Production initialization complete!"
echo "🌐 Starting application server..."
# Start the application
exec node server.js
+5 -4
View File
@@ -27,11 +27,12 @@ const config = {
production: {
client: process.env.DATABASE_CLIENT || 'pg',
connection: {
host: process.env.DB_HOST || 'postgres',
host: process.env.DB_HOST || 'db',
port: process.env.DB_PORT || 5432,
user: process.env.DB_USER || 'postgres',
password: process.env.DB_PASSWORD || 'postgres',
database: process.env.DB_NAME || 'photo_sharing'
user: process.env.DB_USER || 'picpeak',
password: process.env.DB_PASSWORD,
database: process.env.DB_NAME || 'picpeak',
ssl: process.env.DB_SSL === 'true' ? { rejectUnauthorized: false } : false
},
pool: {
min: 2,
+83
View File
@@ -0,0 +1,83 @@
/**
* Migration helper functions for production-safe migrations
*/
/**
* Create a table only if it doesn't already exist
*/
async function createTableIfNotExists(knex, tableName, callback) {
const exists = await knex.schema.hasTable(tableName);
if (!exists) {
console.log(`Creating table: ${tableName}`);
return knex.schema.createTable(tableName, callback);
} else {
console.log(`Table ${tableName} already exists, skipping...`);
}
}
/**
* Add column to table only if it doesn't exist
*/
async function addColumnIfNotExists(knex, tableName, columnName, callback) {
const hasColumn = await knex.schema.hasColumn(tableName, columnName);
if (!hasColumn) {
console.log(`Adding column ${columnName} to table ${tableName}`);
return knex.schema.alterTable(tableName, (table) => {
callback(table);
});
} else {
console.log(`Column ${columnName} already exists in table ${tableName}, skipping...`);
}
}
/**
* Insert data only if it doesn't already exist
*/
async function insertIfNotExists(knex, tableName, data, uniqueField) {
const exists = await knex(tableName)
.where(uniqueField, data[uniqueField])
.first();
if (!exists) {
console.log(`Inserting ${uniqueField}: ${data[uniqueField]} into ${tableName}`);
return knex(tableName).insert(data);
} else {
console.log(`${uniqueField}: ${data[uniqueField]} already exists in ${tableName}, skipping...`);
}
}
/**
* Create index only if it doesn't exist
*/
async function createIndexIfNotExists(knex, tableName, columns, indexName) {
// This is database-specific, works for PostgreSQL
if (knex.client.config.client === 'pg') {
const result = await knex.raw(`
SELECT 1 FROM pg_indexes
WHERE tablename = ? AND indexname = ?
`, [tableName, indexName]);
if (result.rows.length === 0) {
console.log(`Creating index ${indexName} on ${tableName}`);
return knex.schema.alterTable(tableName, (table) => {
table.index(columns, indexName);
});
}
} else {
// For SQLite, just try to create and ignore errors
try {
await knex.schema.alterTable(tableName, (table) => {
table.index(columns, indexName);
});
} catch (error) {
// Index probably already exists
}
}
}
module.exports = {
createTableIfNotExists,
addColumnIfNotExists,
insertIfNotExists,
createIndexIfNotExists
};
+170
View File
@@ -0,0 +1,170 @@
const fs = require('fs').promises;
const path = require('path');
const { db } = require('../src/database/db');
/**
* Production-safe migration runner that handles existing schema
*/
// Create or verify migrations tracking table
async function ensureMigrationsTable() {
const tableExists = await db.schema.hasTable('migrations');
if (!tableExists) {
await db.schema.createTable('migrations', (table) => {
table.increments('id').primary();
table.string('filename').unique().notNullable();
table.timestamp('applied_at').defaultTo(db.fn.now());
});
console.log('Created migrations tracking table');
}
}
// Check if a migration has been applied
async function isMigrationApplied(filename) {
const result = await db('migrations').where('filename', filename).first();
return !!result;
}
// Mark migration as applied without running it (for existing schema)
async function markMigrationAsApplied(filename) {
await db('migrations').insert({ filename });
console.log(`Marked migration ${filename} as applied`);
}
// Detect existing schema and mark migrations as applied
async function detectExistingSchema() {
console.log('Detecting existing schema...');
const tableChecks = [
{ table: 'events', migration: 'init.js' },
{ table: 'photos', migration: 'init.js' },
{ table: 'photo_categories', migration: '004_add_categories_and_cms.js' },
{ table: 'cms_pages', migration: '004_add_categories_and_cms.js' },
{ table: 'login_attempts', migration: '015_add_login_attempts_table.js' },
{ table: 'token_blacklist', migration: '017_add_token_revocation_tables.js' },
];
for (const check of tableChecks) {
const exists = await db.schema.hasTable(check.table);
if (exists) {
const isApplied = await isMigrationApplied(check.migration);
if (!isApplied) {
await markMigrationAsApplied(check.migration);
}
}
}
}
// Run a single migration safely
async function runMigrationSafely(filename) {
try {
const migrationPath = path.join(__dirname, filename);
const migration = require(migrationPath);
if (migration.up) {
console.log(`Running migration: ${filename}`);
// Run migration in a transaction if possible
if (db.client.config.client === 'pg') {
await db.transaction(async (trx) => {
await migration.up(trx);
});
} else {
await migration.up(db);
}
await db('migrations').insert({ filename });
console.log(`Migration ${filename} completed successfully`);
}
} catch (error) {
// Check if error is because schema already exists
if (error.code === '42P07' || // PostgreSQL: relation already exists
error.code === 'SQLITE_ERROR' && error.message.includes('already exists')) {
console.log(`Migration ${filename} - schema already exists, marking as applied`);
await markMigrationAsApplied(filename);
} else {
throw error;
}
}
}
// Main migration runner
async function runMigrations() {
let connection;
try {
console.log('Starting production-safe database migrations...');
// Ensure database connection is ready
await db.raw('SELECT 1');
console.log('Database connection verified');
// Create migrations tracking table
await ensureMigrationsTable();
// Detect and mark existing schema
await detectExistingSchema();
// Get all migration files
const files = await fs.readdir(__dirname);
const migrationFiles = files
.filter(f => f.match(/^\d{3}_.*\.js$/) || f === 'init.js')
.sort((a, b) => {
// Ensure init.js runs first
if (a === 'init.js') return -1;
if (b === 'init.js') return 1;
return a.localeCompare(b);
});
// Run pending migrations
let pendingCount = 0;
let skippedCount = 0;
for (const file of migrationFiles) {
const isApplied = await isMigrationApplied(file);
if (!isApplied) {
await runMigrationSafely(file);
pendingCount++;
} else {
skippedCount++;
}
}
console.log(`\nMigration Summary:`);
console.log(`- Applied: ${pendingCount} migration(s)`);
console.log(`- Skipped: ${skippedCount} migration(s) (already applied)`);
console.log(`- Total: ${migrationFiles.length} migration(s)`);
console.log('\nAll migrations completed successfully');
// Close database connection
await db.destroy();
process.exit(0);
} catch (error) {
console.error('\n❌ Migration failed:', error.message);
console.error('Error details:', error);
// Close database connection on error
try {
await db.destroy();
} catch (e) {
// Ignore
}
process.exit(1);
}
}
// Add delay for database readiness in production
async function waitAndRun() {
if (process.env.NODE_ENV === 'production') {
console.log('Waiting 2 seconds for database readiness...');
await new Promise(resolve => setTimeout(resolve, 2000));
}
await runMigrations();
}
// Only run if called directly
if (require.main === module) {
waitAndRun();
}
module.exports = { runMigrations };
+140 -2
View File
@@ -1,12 +1,12 @@
{
"name": "picpeak-backend",
"version": "1.0.5",
"version": "1.0.10",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "picpeak-backend",
"version": "1.0.5",
"version": "1.0.10",
"dependencies": {
"adm-zip": "^0.5.16",
"archiver": "^5.3.1",
@@ -29,6 +29,7 @@
"multer": "^2.0.1",
"node-cron": "^3.0.2",
"nodemailer": "^6.9.1",
"pg": "^8.16.3",
"react-i18next": "^15.6.0",
"sharp": "^0.32.0",
"sqlite3": "^5.1.6",
@@ -6471,12 +6472,101 @@
"integrity": "sha512-RA1GjUVMnvYFxuqovrEqZoxxW5NUZqbwKtYz/Tt7nXerk0LbLblQmrsgdeOxV5SFHf0UDggjS/bSeOZwt1pmEQ==",
"license": "MIT"
},
"node_modules/pg": {
"version": "8.16.3",
"resolved": "https://registry.npmjs.org/pg/-/pg-8.16.3.tgz",
"integrity": "sha512-enxc1h0jA/aq5oSDMvqyW3q89ra6XIIDZgCX9vkMrnz5DFTw/Ny3Li2lFQ+pt3L6MCgm/5o2o8HW9hiJji+xvw==",
"license": "MIT",
"dependencies": {
"pg-connection-string": "^2.9.1",
"pg-pool": "^3.10.1",
"pg-protocol": "^1.10.3",
"pg-types": "2.2.0",
"pgpass": "1.0.5"
},
"engines": {
"node": ">= 16.0.0"
},
"optionalDependencies": {
"pg-cloudflare": "^1.2.7"
},
"peerDependencies": {
"pg-native": ">=3.0.1"
},
"peerDependenciesMeta": {
"pg-native": {
"optional": true
}
}
},
"node_modules/pg-cloudflare": {
"version": "1.2.7",
"resolved": "https://registry.npmjs.org/pg-cloudflare/-/pg-cloudflare-1.2.7.tgz",
"integrity": "sha512-YgCtzMH0ptvZJslLM1ffsY4EuGaU0cx4XSdXLRFae8bPP4dS5xL1tNB3k2o/N64cHJpwU7dxKli/nZ2lUa5fLg==",
"license": "MIT",
"optional": true
},
"node_modules/pg-connection-string": {
"version": "2.6.1",
"resolved": "https://registry.npmjs.org/pg-connection-string/-/pg-connection-string-2.6.1.tgz",
"integrity": "sha512-w6ZzNu6oMmIzEAYVw+RLK0+nqHPt8K3ZnknKi+g48Ak2pr3dtljJW3o+D/n2zzCG07Zoe9VOX3aiKpj+BN0pjg==",
"license": "MIT"
},
"node_modules/pg-int8": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/pg-int8/-/pg-int8-1.0.1.tgz",
"integrity": "sha512-WCtabS6t3c8SkpDBUlb1kjOs7l66xsGdKpIPZsg4wR+B3+u9UAum2odSsF9tnvxg80h4ZxLWMy4pRjOsFIqQpw==",
"license": "ISC",
"engines": {
"node": ">=4.0.0"
}
},
"node_modules/pg-pool": {
"version": "3.10.1",
"resolved": "https://registry.npmjs.org/pg-pool/-/pg-pool-3.10.1.tgz",
"integrity": "sha512-Tu8jMlcX+9d8+QVzKIvM/uJtp07PKr82IUOYEphaWcoBhIYkoHpLXN3qO59nAI11ripznDsEzEv8nUxBVWajGg==",
"license": "MIT",
"peerDependencies": {
"pg": ">=8.0"
}
},
"node_modules/pg-protocol": {
"version": "1.10.3",
"resolved": "https://registry.npmjs.org/pg-protocol/-/pg-protocol-1.10.3.tgz",
"integrity": "sha512-6DIBgBQaTKDJyxnXaLiLR8wBpQQcGWuAESkRBX/t6OwA8YsqP+iVSiond2EDy6Y/dsGk8rh/jtax3js5NeV7JQ==",
"license": "MIT"
},
"node_modules/pg-types": {
"version": "2.2.0",
"resolved": "https://registry.npmjs.org/pg-types/-/pg-types-2.2.0.tgz",
"integrity": "sha512-qTAAlrEsl8s4OiEQY69wDvcMIdQN6wdz5ojQiOy6YRMuynxenON0O5oCpJI6lshc6scgAY8qvJ2On/p+CXY0GA==",
"license": "MIT",
"dependencies": {
"pg-int8": "1.0.1",
"postgres-array": "~2.0.0",
"postgres-bytea": "~1.0.0",
"postgres-date": "~1.0.4",
"postgres-interval": "^1.1.0"
},
"engines": {
"node": ">=4"
}
},
"node_modules/pg/node_modules/pg-connection-string": {
"version": "2.9.1",
"resolved": "https://registry.npmjs.org/pg-connection-string/-/pg-connection-string-2.9.1.tgz",
"integrity": "sha512-nkc6NpDcvPVpZXxrreI/FOtX3XemeLl8E0qFr6F2Lrm/I8WOnaWNhIPK2Z7OHpw7gh5XJThi6j6ppgNoaT1w4w==",
"license": "MIT"
},
"node_modules/pgpass": {
"version": "1.0.5",
"resolved": "https://registry.npmjs.org/pgpass/-/pgpass-1.0.5.tgz",
"integrity": "sha512-FdW9r/jQZhSeohs1Z3sI1yxFQNFvMcnmfuj4WBMUTxOrAyLMaTcE1aAMBiTlbMNaXvBCQuVi0R7hd8udDSP7ug==",
"license": "MIT",
"dependencies": {
"split2": "^4.1.0"
}
},
"node_modules/picocolors": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz",
@@ -6575,6 +6665,45 @@
"node": ">=8"
}
},
"node_modules/postgres-array": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/postgres-array/-/postgres-array-2.0.0.tgz",
"integrity": "sha512-VpZrUqU5A69eQyW2c5CA1jtLecCsN2U/bD6VilrFDWq5+5UIEVO7nazS3TEcHf1zuPYO/sqGvUvW62g86RXZuA==",
"license": "MIT",
"engines": {
"node": ">=4"
}
},
"node_modules/postgres-bytea": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/postgres-bytea/-/postgres-bytea-1.0.0.tgz",
"integrity": "sha512-xy3pmLuQqRBZBXDULy7KbaitYqLcmxigw14Q5sj8QBVLqEwXfeybIKVWiqAXTlcvdvb0+xkOtDbfQMOf4lST1w==",
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/postgres-date": {
"version": "1.0.7",
"resolved": "https://registry.npmjs.org/postgres-date/-/postgres-date-1.0.7.tgz",
"integrity": "sha512-suDmjLVQg78nMK2UZ454hAG+OAW+HQPZ6n++TNDUX+L0+uUlLywnoxJKDou51Zm+zTCjrCl0Nq6J9C5hP9vK/Q==",
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/postgres-interval": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/postgres-interval/-/postgres-interval-1.2.0.tgz",
"integrity": "sha512-9ZhXKM/rw350N1ovuWHbGxnGh/SNJ4cnxHiM0rxE4VN41wsg8P8zWn9hv/buK00RP4WvlOyr/RBDiptyxVbkZQ==",
"license": "MIT",
"dependencies": {
"xtend": "^4.0.0"
},
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/prebuild-install": {
"version": "7.1.3",
"resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",
@@ -7494,6 +7623,15 @@
"source-map": "^0.6.0"
}
},
"node_modules/split2": {
"version": "4.2.0",
"resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz",
"integrity": "sha512-UcjcJOWknrNkF6PLX83qcHM6KHgVKNkV62Y8a5uYDVv9ydGQVwAHMKqHdJje1VTWpljG0WYpCDhrCdAOYH4TWg==",
"license": "ISC",
"engines": {
"node": ">= 10.x"
}
},
"node_modules/sprintf-js": {
"version": "1.1.3",
"resolved": "https://registry.npmjs.org/sprintf-js/-/sprintf-js-1.1.3.tgz",
+3 -1
View File
@@ -1,12 +1,13 @@
{
"name": "picpeak-backend",
"version": "1.0.5",
"version": "1.0.10",
"description": "Backend for PicPeak event photo sharing platform",
"main": "server.js",
"scripts": {
"start": "node server.js",
"dev": "nodemon server.js",
"migrate": "node migrations/run-migrations.js",
"migrate:safe": "node migrations/run-migrations-safe.js",
"test": "jest",
"lint": "eslint src/"
},
@@ -32,6 +33,7 @@
"multer": "^2.0.1",
"node-cron": "^3.0.2",
"nodemailer": "^6.9.1",
"pg": "^8.16.3",
"react-i18next": "^15.6.0",
"sharp": "^0.32.0",
"sqlite3": "^5.1.6",
+22 -4
View File
@@ -13,7 +13,7 @@ const path = require('path');
const { initializeDatabase } = require('./src/database/db');
const { startFileWatcher } = require('./src/services/fileWatcher');
const { startExpirationChecker } = require('./src/services/expirationChecker');
const { startEmailQueueProcessor } = require('./src/services/emailProcessor');
const { initializeTransporter, startEmailQueueProcessor } = require('./src/services/emailProcessor');
const { maintenanceMiddleware } = require('./src/middleware/maintenance');
const { sessionTimeoutMiddleware } = require('./src/middleware/sessionTimeout');
const logger = require('./src/utils/logger');
@@ -152,8 +152,25 @@ app.use('/thumbnails', require('./src/middleware/photoAuth'), setCorsHeaders, se
app.use('/uploads', setCorsHeaders, secureStatic(path.join(__dirname, 'storage/uploads')));
// Health check endpoint
app.get('/api/health', (req, res) => {
res.json({ status: 'ok', timestamp: new Date().toISOString() });
app.get('/api/health', async (req, res) => {
try {
// Check database connectivity
await db.raw('SELECT 1');
res.json({
status: 'ok',
database: 'connected',
timestamp: new Date().toISOString()
});
} catch (error) {
logger.error('Health check failed:', error);
res.status(503).json({
status: 'error',
database: 'disconnected',
error: error.message,
timestamp: new Date().toISOString()
});
}
});
// Routes
@@ -189,7 +206,8 @@ async function startServer() {
// Start expiration checker
startExpirationChecker();
// Start email queue processor
// Initialize email transporter and start queue processor
await initializeTransporter();
startEmailQueueProcessor();
app.listen(PORT, () => {
+7 -4
View File
@@ -395,12 +395,15 @@ function stopEmailQueueProcessor() {
}
}
// Initialize on module load
initializeTransporter().then(() => {
startEmailQueueProcessor();
});
// Initialize on module load - DISABLED for production startup
// This will be called from server.js after database is ready
// initializeTransporter().then(() => {
// startEmailQueueProcessor();
// });
module.exports = {
initializeTransporter,
startEmailQueueProcessor,
sendTemplateEmail,
processEmailQueue,
queueEmail,
+29
View File
@@ -0,0 +1,29 @@
#!/bin/sh
# wait-for-db.sh - Wait for PostgreSQL to be ready before starting the application
set -e
host="$DB_HOST"
port="${DB_PORT:-5432}"
user="${DB_USER:-picpeak}"
echo "Waiting for PostgreSQL at $host:$port..."
# Wait for PostgreSQL to be ready
until PGPASSWORD=$DB_PASSWORD psql -h "$host" -p "$port" -U "$user" -d "${DB_NAME:-picpeak}" -c '\q' 2>/dev/null; do
>&2 echo "PostgreSQL is unavailable - sleeping"
sleep 2
done
>&2 echo "PostgreSQL is up - executing command"
# Run migrations (use safe runner in production)
echo "Running database migrations..."
if [ "$NODE_ENV" = "production" ]; then
npm run migrate:safe
else
npm run migrate
fi
# Execute the main command
exec "$@"
+12 -1
View File
@@ -8,6 +8,8 @@ services:
context: ./backend
dockerfile: Dockerfile
restart: unless-stopped
depends_on:
- db
environment:
- NODE_ENV=production
- PORT=3000
@@ -31,6 +33,10 @@ services:
# Analytics
- UMAMI_URL=${UMAMI_URL}
- UMAMI_WEBSITE_ID=${UMAMI_WEBSITE_ID}
# Storage paths
- STORAGE_PATH=/app/storage
- EVENTS_PATH=/app/storage/events
- ARCHIVE_PATH=/app/storage/events/archived
volumes:
- ./storage:/app/storage
- ./data:/app/data
@@ -82,10 +88,15 @@ services:
- POSTGRES_USER=${DB_USER:-picpeak}
- POSTGRES_PASSWORD=${DB_PASSWORD}
- POSTGRES_DB=${DB_NAME:-picpeak}
# Allow connections from any host with password authentication
- POSTGRES_HOST_AUTH_METHOD=scram-sha-256
- POSTGRES_INITDB_ARGS=--auth-host=scram-sha-256 --auth-local=trust
volumes:
- postgres_data:/var/lib/postgresql/data
networks:
- picpeak
# Allow connections without SSL requirement from Docker network
command: postgres -c ssl=off
umami:
image: ghcr.io/umami-software/umami:postgresql-latest
@@ -104,4 +115,4 @@ networks:
driver: bridge
volumes:
postgres_data:
postgres_data:
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "picpeak-frontend",
"version": "1.0.5",
"version": "1.0.10",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "picpeak-frontend",
"version": "1.0.5",
"version": "1.0.10",
"dependencies": {
"@tanstack/react-query": "^5.0.0",
"@tiptap/extension-link": "^2.25.0",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "picpeak-frontend",
"private": true,
"version": "1.0.5",
"version": "1.0.10",
"type": "module",
"scripts": {
"dev": "vite",