Files

IGNY8 VPS (Salman) 128b186865 temproary docs uplaoded

2026-03-23 09:02:49 +00:00

45 KiB

Raw Blame History

IGNY8 Phase 0: Staging Environment Setup (Doc 00D)

Document Status: Build Specification Date Created: 2026-03-23 Target Phase: Phase 0 - Infrastructure & Deployment Related Docs: 00B Infrastructure Setup | 00C Production Migration | 00B Version Matrix (SINGLE SOURCE OF TRUTH for all versions)

Key Details:

Staging runs on the NEW VPS (from 00B Infrastructure Setup)
Separate database: igny8_staging_db on PostgreSQL 18 instance
Redis DB 1 (production uses DB 0)
Separate ports: Backend 8012, Frontend 8024, Marketing 8026
Separate compose file: docker-compose.staging.yml with project name igny8-staging
DNS coordination with 00C 3-stage migration flow

1. Current State

Staging Environment Location: On the NEW VPS, as provisioned in 00B Infrastructure Setup.

Note on Versions: For all component versions (PostgreSQL, Redis, Docker, etc.), refer to the Version Matrix in 00B Infrastructure Setup as the single source of truth. This document reflects those versions. All staging components use the latest versions matching production on the NEW VPS.

1.1 Infrastructure Baseline

Host Server: Single Linux VM running Docker on NEW VPS (from 00B Infrastructure Setup)
Base OS: Ubuntu 24.04 LTS
Shared Resources:
- PostgreSQL 18 server (port 5432)
- Redis 8 server (port 6379)
- Docker network: igny8_net
- Caddy 2.11 reverse proxy (port 80/443)
- Cloudflare DNS management (may or may not be active - dependent on 00C flow stage)
- Log directory: /data/app/logs/

1.2 Production Environment (Already Complete - Doc 00C)

Database: igny8_db (PostgreSQL)
Cache: Redis DB 0
Compose file: docker-compose.yml
Containers:
- igny8_backend (port 8010)
- igny8_frontend (port 5173)
- igny8_marketing_dev (port 5174)
- igny8_celery_worker
- igny8_celery_beat
Env file: .env (production settings)
Domains: igny8.com, api.igny8.com, marketing.igny8.com
Logs: /data/app/logs/production/

1.3 Staging Environment (To Be Built)

Does not yet exist. This document defines the complete staging setup.

2. What to Build

2.1 Staging Environment Architecture

A complete parallel environment sharing infrastructure with production:

┌─────────────────────────────────────────────────────┐
│         Docker Containers (Staging)                 │
├─────────────────────────────────────────────────────┤
│                                                     │
│  igny8_staging_backend:8012 → :8010 (Django)      │
│  igny8_staging_frontend:8024 → :5173 (Vue)        │
│  igny8_staging_marketing_dev:8026 → :5174 (Nuxt)  │
│  igny8_staging_celery_worker                       │
│  igny8_staging_celery_beat                         │
│                                                     │
└─────────────────────────────────────────────────────┘
                        ↓
            igny8_net (Docker network)
                        ↓
┌─────────────────────────────────────────────────────┐
│      Shared Infrastructure Services                 │
├─────────────────────────────────────────────────────┤
│                                                     │
│  PostgreSQL 18 (port 5432)                         │
│  ├── igny8_db (production)                          │
│  └── igny8_staging_db (staging)                     │
│                                                     │
│  Redis 8 (port 6379)                               │
│  ├── DB 0 (production)                              │
│  └── DB 1 (staging)                                 │
│                                                     │
│  Caddy 2.11 (port 80/443) → Routes staging domains│
│                                                     │
└─────────────────────────────────────────────────────┘
                        ↓
        Cloudflare DNS → staging.igny8.com
                        api.staging.igny8.com
                        marketing.staging.igny8.com

2.2 Deliverables

Staging PostgreSQL database (igny8_staging_db)
Docker Compose file: docker-compose.staging.yml
Environment file: .env.staging
Caddyfile additions for staging route configuration
Cloudflare DNS records for staging subdomains
Staging Docker images (tagged :staging)
Deployment script: deploy-staging.sh
Sync script: sync-prod-to-staging.sh
Logs directory: /data/app/logs/staging/

2.3 Key Characteristics

Independent Databases: Separate PostgreSQL DB and Redis DB
Shared Infrastructure: Same PostgreSQL server, Redis server, Docker network, Caddy
Isolated Configuration: Separate env file with test credentials, sandbox API keys
Feature Branch Testing: staging branch receives feature PRs before merging to main
Production Data Sync: Ability to copy production data to staging for realistic testing
Domain Isolation: Staging domains clearly identified (staging subdomain)

3. Data Models / APIs

3.1 PostgreSQL Database Setup

PostgreSQL Version: PostgreSQL 18 (postgres:18-alpine from 00B Version Matrix) Database Name: igny8_staging_db Owner: igny8_user (same as production) Encoding: UTF8 Locale: Same as production Extensions: Same as production (uuid-ossp, etc.)

Schema: Identical structure to production (migrations auto-generate on first startup)

3.2 Redis Setup

Redis Version: Redis 8 (redis:8-alpine from 00B Version Matrix) Instance: Same Redis server (port 6379) Database: DB 1 (production uses DB 0) Prefix: staging: (for key namespacing) Persistence: Same RDB/AOF configuration as production

3.3 Environment Variables

DJANGO_ENV: staging DEBUG: True (staging allows debug mode for testing) ALLOWED_HOSTS: staging.igny8.com,staging-api.igny8.com,localhost,127.0.0.1 API_KEYS: Test sandbox keys (different from production) PAYMENT_GATEWAY: Sandbox credentials (Stripe test keys, etc.) CELERY: Routes to staging Redis DB 1 CACHE: Routes to staging Redis DB 1 CORS: Allows staging frontend origins

3.4 API Endpoints

Component	Production	Staging
Frontend	`igny8.com`	`staging.igny8.com`
Backend API	`api.igny8.com`	`staging-api.igny8.com`
Marketing	`marketing.igny8.com`	`staging-marketing.igny8.com`

4. Implementation Steps

Version Requirements: All versions referenced below are from the 00B Version Matrix (source of truth for all versions). The staging environment uses identical versions to production on the NEW VPS:

PostgreSQL 18 (postgres:18-alpine)
Redis 8 (redis:8-alpine)
Caddy 2.11 (caddy:2-alpine)
Ubuntu 24.04 LTS (base OS)
Docker Engine 29.x
Python 3.14 (in backend container)
Node 24 LTS (in frontend and marketing containers)
Django 6.0
Vite 8
Gunicorn 25
Celery 5.6

Step 1: Create Staging PostgreSQL Database

File: SQL commands to run on the host PostgreSQL server

-- Connect as superuser
psql -U postgres -h localhost

-- Create staging database
CREATE DATABASE igny8_staging_db
  WITH OWNER igny8_user
  ENCODING 'UTF8'
  LOCALE 'en_US.UTF-8'
  TEMPLATE template0;

-- Verify creation
\l | grep igny8

-- Grant privileges
GRANT ALL PRIVILEGES ON DATABASE igny8_staging_db TO igny8_user;
GRANT ALL PRIVILEGES ON SCHEMA public TO igny8_user;

Execution:

# On host server
docker exec -i igny8_postgres psql -U postgres -d postgres << 'EOF'
CREATE DATABASE igny8_staging_db
  WITH OWNER igny8_user
  ENCODING 'UTF8'
  LOCALE 'en_US.UTF-8'
  TEMPLATE template0;

GRANT ALL PRIVILEGES ON DATABASE igny8_staging_db TO igny8_user;
GRANT ALL PRIVILEGES ON SCHEMA public TO igny8_user;
EOF

echo "Staging database created"

Step 2: Create `docker-compose.staging.yml`

Location: Root of project directory (alongside docker-compose.yml) Project Name: igny8-staging

version: '3.8'

services:
  # Backend API Service
  igny8_staging_backend:
    image: igny8-backend:staging
    container_name: igny8_staging_backend
    environment:
      - DJANGO_ENV=staging
      - DEBUG=True
      - ALLOWED_HOSTS=staging.igny8.com,staging-api.igny8.com,localhost,127.0.0.1
      - SECRET_KEY=${STAGING_SECRET_KEY}
      - DATABASE_URL=postgresql://igny8_user:${DB_PASSWORD}@igny8_postgres:5432/igny8_staging_db
      - REDIS_URL=redis://igny8_redis:6379/1
      - CELERY_BROKER_URL=redis://igny8_redis:6379/1
      - CELERY_RESULT_BACKEND=redis://igny8_redis:6379/1
      - CACHE_URL=redis://igny8_redis:6379/1
      - CORS_ALLOWED_ORIGINS=https://staging.igny8.com,https://staging-marketing.igny8.com
      - STRIPE_PUBLIC_KEY=${STAGING_STRIPE_PUBLIC_KEY}
      - STRIPE_SECRET_KEY=${STAGING_STRIPE_SECRET_KEY}
      - STRIPE_WEBHOOK_SECRET=${STAGING_STRIPE_WEBHOOK_SECRET}
      - API_BASE_URL=https://staging-api.igny8.com
      - FRONTEND_URL=https://staging.igny8.com
      - MARKETING_URL=https://staging-marketing.igny8.com
      - AWS_ACCESS_KEY_ID=${STAGING_AWS_ACCESS_KEY_ID}
      - AWS_SECRET_ACCESS_KEY=${STAGING_AWS_SECRET_ACCESS_KEY}
      - AWS_S3_BUCKET=${STAGING_AWS_S3_BUCKET}
      - AWS_REGION=${AWS_REGION}
      - SENTRY_DSN=${STAGING_SENTRY_DSN}
      - LOG_LEVEL=INFO
    ports:
      - "8012:8010"
    volumes:
      - ./backend:/app/backend
      - /data/app/logs/staging:/var/log/igny8
    networks:
      - igny8_net
    depends_on:
      - igny8_postgres
      - igny8_redis
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8010/health/"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s
    labels:
      - "com.igny8.component=backend"
      - "com.igny8.environment=staging"

  # Frontend Service
  igny8_staging_frontend:
    image: igny8-frontend-dev:staging
    container_name: igny8_staging_frontend
    environment:
      - NODE_ENV=staging
      - VITE_API_URL=https://staging-api.igny8.com
      - VITE_ENVIRONMENT=staging
    ports:
      - "8024:5173"
    volumes:
      - ./frontend:/app
      - /app/node_modules
    networks:
      - igny8_net
    restart: unless-stopped
    labels:
      - "com.igny8.component=frontend"
      - "com.igny8.environment=staging"

  # Marketing Site Service
  igny8_staging_marketing_dev:
    image: igny8-marketing-dev:staging
    container_name: igny8_staging_marketing_dev
    environment:
      - NODE_ENV=staging
      - NUXT_PUBLIC_API_URL=https://staging-api.igny8.com
      - NUXT_PUBLIC_ENVIRONMENT=staging
    ports:
      - "8026:5174"
    volumes:
      - ./marketing:/app
      - /app/.nuxt
      - /app/node_modules
    networks:
      - igny8_net
    restart: unless-stopped
    labels:
      - "com.igny8.component=marketing"
      - "com.igny8.environment=staging"

  # Celery Worker
  igny8_staging_celery_worker:
    image: igny8-backend:staging
    container_name: igny8_staging_celery_worker
    command: celery -A backend.celery worker --loglevel=info --concurrency=2
    environment:
      - DJANGO_ENV=staging
      - DEBUG=True
      - SECRET_KEY=${STAGING_SECRET_KEY}
      - DATABASE_URL=postgresql://igny8_user:${DB_PASSWORD}@igny8_postgres:5432/igny8_staging_db
      - REDIS_URL=redis://igny8_redis:6379/1
      - CELERY_BROKER_URL=redis://igny8_redis:6379/1
      - CELERY_RESULT_BACKEND=redis://igny8_redis:6379/1
      - AWS_ACCESS_KEY_ID=${STAGING_AWS_ACCESS_KEY_ID}
      - AWS_SECRET_ACCESS_KEY=${STAGING_AWS_SECRET_ACCESS_KEY}
      - AWS_S3_BUCKET=${STAGING_AWS_S3_BUCKET}
    volumes:
      - ./backend:/app/backend
      - /data/app/logs/staging:/var/log/igny8
    networks:
      - igny8_net
    depends_on:
      - igny8_postgres
      - igny8_redis
    restart: unless-stopped
    labels:
      - "com.igny8.component=celery-worker"
      - "com.igny8.environment=staging"

  # Celery Beat (Scheduler)
  igny8_staging_celery_beat:
    image: igny8-backend:staging
    container_name: igny8_staging_celery_beat
    command: celery -A backend.celery beat --loglevel=info --scheduler django_celery_beat.schedulers:DatabaseScheduler
    environment:
      - DJANGO_ENV=staging
      - DEBUG=True
      - SECRET_KEY=${STAGING_SECRET_KEY}
      - DATABASE_URL=postgresql://igny8_user:${DB_PASSWORD}@igny8_postgres:5432/igny8_staging_db
      - REDIS_URL=redis://igny8_redis:6379/1
      - CELERY_BROKER_URL=redis://igny8_redis:6379/1
      - CELERY_RESULT_BACKEND=redis://igny8_redis:6379/1
    volumes:
      - ./backend:/app/backend
      - /data/app/logs/staging:/var/log/igny8
    networks:
      - igny8_net
    depends_on:
      - igny8_postgres
      - igny8_redis
    restart: unless-stopped
    labels:
      - "com.igny8.component=celery-beat"
      - "com.igny8.environment=staging"

networks:
  igny8_net:
    external: true

volumes:
  # Data volumes referenced from external production infrastructure

Step 3: Create `.env.staging`

Location: Root of project directory Purpose: Staging-specific environment variables

# ==============================================================================
# IGNY8 STAGING ENVIRONMENT CONFIGURATION
# ==============================================================================

# ==============================================================================
# GENERAL
# ==============================================================================
ENVIRONMENT=staging
DJANGO_ENV=staging
DEBUG=True
LOG_LEVEL=INFO

# ==============================================================================
# SECRETS (STAGING VERSIONS)
# ==============================================================================
# Generate new secrets for staging (do NOT copy from production)
STAGING_SECRET_KEY=your-generated-staging-secret-key-here-minimum-50-chars-random
DB_PASSWORD=your-staging-db-password

# ==============================================================================
# DATABASE
# ==============================================================================
DATABASE_ENGINE=postgresql
DATABASE_HOST=igny8_postgres
DATABASE_PORT=5432
DATABASE_NAME=igny8_staging_db
DATABASE_USER=igny8_user
DATABASE_PASSWORD=${DB_PASSWORD}
# Full URL for Django
DATABASE_URL=postgresql://igny8_user:${DB_PASSWORD}@igny8_postgres:5432/igny8_staging_db

# ==============================================================================
# CACHE & QUEUE (REDIS DB 1 - Separate from Production)
# ==============================================================================
REDIS_HOST=igny8_redis
REDIS_PORT=6379
REDIS_DB=1
REDIS_PASSWORD=
REDIS_URL=redis://igny8_redis:6379/1
CACHE_URL=redis://igny8_redis:6379/1

# ==============================================================================
# CELERY (Uses Redis DB 1)
# ==============================================================================
CELERY_BROKER_URL=redis://igny8_redis:6379/1
CELERY_RESULT_BACKEND=redis://igny8_redis:6379/1
CELERY_ACCEPT_CONTENT=json
CELERY_TASK_SERIALIZER=json

# ==============================================================================
# HOSTS & DOMAINS (STAGING)
# ==============================================================================
ALLOWED_HOSTS=staging.igny8.com,staging-api.igny8.com,localhost,127.0.0.1
FRONTEND_URL=https://staging.igny8.com
API_BASE_URL=https://staging-api.igny8.com
MARKETING_URL=https://staging-marketing.igny8.com
CORS_ALLOWED_ORIGINS=https://staging.igny8.com,https://staging-marketing.igny8.com,http://localhost:8024,http://localhost:8026

# ==============================================================================
# PAYMENT GATEWAY (STRIPE - SANDBOX)
# ==============================================================================
STAGING_STRIPE_PUBLIC_KEY=pk_test_your_staging_stripe_public_key
STAGING_STRIPE_SECRET_KEY=sk_test_your_staging_stripe_secret_key
STAGING_STRIPE_WEBHOOK_SECRET=whsec_your_staging_webhook_secret

# ==============================================================================
# AWS / S3 (STAGING BUCKET)
# ==============================================================================
STAGING_AWS_ACCESS_KEY_ID=your-staging-aws-access-key
STAGING_AWS_SECRET_ACCESS_KEY=your-staging-aws-secret-key
STAGING_AWS_S3_BUCKET=igny8-staging-assets
AWS_REGION=us-east-1
AWS_S3_CUSTOM_DOMAIN=staging-assets.igny8.com

# ==============================================================================
# EXTERNAL SERVICES (STAGING / SANDBOX CREDENTIALS)
# ==============================================================================
# Email
MAILGUN_API_KEY=your-staging-mailgun-key
MAILGUN_DOMAIN=staging-mail.igny8.com

# Analytics
MIXPANEL_TOKEN=your-staging-mixpanel-token

# Error Tracking
STAGING_SENTRY_DSN=https://your-staging-sentry-dsn

# SMS
TWILIO_ACCOUNT_SID=your-staging-twilio-sid
TWILIO_AUTH_TOKEN=your-staging-twilio-token
TWILIO_PHONE_NUMBER=+15551234567

# ==============================================================================
# SECURITY (STAGING)
# ==============================================================================
SECURE_SSL_REDIRECT=True
SESSION_COOKIE_SECURE=True
CSRF_COOKIE_SECURE=True
SECURE_HSTS_SECONDS=31536000
SECURE_HSTS_INCLUDE_SUBDOMAINS=True

# ==============================================================================
# DJANGO SETTINGS
# ==============================================================================
DJANGO_SUPERUSER_USERNAME=admin
DJANGO_SUPERUSER_EMAIL=admin@staging.igny8.com
DJANGO_SUPERUSER_PASSWORD=your-staging-admin-password

# ==============================================================================
# FRONTEND / VUE
# ==============================================================================
VITE_API_URL=https://staging-api.igny8.com
VITE_ENVIRONMENT=staging

# ==============================================================================
# MARKETING / NUXT
# ==============================================================================
NUXT_PUBLIC_API_URL=https://staging-api.igny8.com
NUXT_PUBLIC_ENVIRONMENT=staging

Step 4: Update Caddyfile for Staging Routes

Location: /data/caddy/Caddyfile (append to existing production configuration)

# ==============================================================================
# STAGING ENVIRONMENT ROUTES
# ==============================================================================

# Staging API Backend
staging-api.igny8.com {
  reverse_proxy igny8_staging_backend:8010 {
    header_uri X-Forwarded-Proto https
    header_uri X-Forwarded-Host {host}
  }
  log {
    output file /data/app/logs/staging/staging-api-access.log {
      roll_size 100mb
      roll_keep 10
    }
  }
  encode gzip
}

# Staging Frontend
staging.igny8.com {
  reverse_proxy igny8_staging_frontend:5173 {
    header_uri X-Forwarded-Proto https
    header_uri X-Forwarded-Host {host}
  }
  log {
    output file /data/app/logs/staging/staging-frontend-access.log {
      roll_size 100mb
      roll_keep 10
    }
  }
  encode gzip
}

# Staging Marketing Site
staging-marketing.igny8.com {
  reverse_proxy igny8_staging_marketing_dev:5174 {
    header_uri X-Forwarded-Proto https
    header_uri X-Forwarded-Host {host}
  }
  log {
    output file /data/app/logs/staging/staging-marketing-access.log {
      roll_size 100mb
      roll_keep 10
    }
  }
  encode gzip
}

Reload Caddy:

# On host
docker exec igny8_caddy caddy reload --config /etc/caddy/Caddyfile

Step 5: Create DNS Records for Staging Subdomains

DNS Provider Timing: Staging setup is on the NEW VPS (from 00B). This assumes DNS infrastructure is at one of the stages in the 00C 3-stage migration flow:

00C Stage 1 (DNS Preparation): Old provider still active - use old provider for staging DNS
00C Stage 2 (DNS Flip): Migrating to new provider - add staging records to new provider
00C Stage 3 (Cloudflare Onboarding): Cloudflare now active - add staging records to Cloudflare

Current DNS Provider: Use whichever DNS provider is currently managing igny8.com per the 00C flow:

If 00C Stage 3 complete (Cloudflare active): Use Cloudflare
If 00C Stage 2 in progress or complete: Use the new DNS provider
Otherwise: Use the current active DNS provider

Note on Staging Domains: Staging may use test variants (e.g., test-staging.igny8.com) during the migration period if production domains are already in use. Coordinate with 00C DNS flow to avoid conflicts.

Add the following DNS records to your active provider:

Type	Name	Content	TTL	Proxy
CNAME	staging	igny8.com	3600	Proxied (if Cloudflare)
CNAME	staging-api	igny8.com	3600	Proxied (if Cloudflare)
CNAME	staging-marketing	igny8.com	3600	Proxied (if Cloudflare)

SSL/TLS Setup:

If using Cloudflare: Ensure staging subdomains are included in SSL certificate (auto-included if using wildcard or zone certificate)
If using another DNS provider: Ensure Caddy can obtain certificates for staging subdomains (will auto-issue via ACME)

Wait for propagation: DNS changes typically propagate within 5-10 minutes.

Step 6: Build Staging Docker Images

Backend Image

cd /path/to/backend
docker build -f Dockerfile -t igny8-backend:staging .

# Verify
docker images | grep igny8-backend

Frontend Image

cd /path/to/frontend
docker build -f Dockerfile.dev -t igny8-frontend-dev:staging .

# Verify
docker images | grep igny8-frontend-dev

Marketing Image

cd /path/to/marketing
docker build -f Dockerfile.dev -t igny8-marketing-dev:staging .

# Verify
docker images | grep igny8-marketing-dev

Tag images for registry (optional, if using Docker Hub/ECR):

# Example: tag for Docker Hub
docker tag igny8-backend:staging yourregistry/igny8-backend:staging
docker tag igny8-frontend-dev:staging yourregistry/igny8-frontend-dev:staging
docker tag igny8-marketing-dev:staging yourregistry/igny8-marketing-dev:staging

# Push to registry
docker push yourregistry/igny8-backend:staging
docker push yourregistry/igny8-frontend-dev:staging
docker push yourregistry/igny8-marketing-dev:staging

Step 7: Create Logs Directory

# On host server
sudo mkdir -p /data/app/logs/staging
sudo chown -R 1000:1000 /data/app/logs/staging
sudo chmod 755 /data/app/logs/staging

Step 8: Create `deploy-staging.sh` Script

Location: Root project directory or scripts/deploy-staging.sh

#!/bin/bash
# ==============================================================================
# IGNY8 Staging Environment Deployment Script
# ==============================================================================
# Usage: ./deploy-staging.sh [pull-images]
# ==============================================================================

set -e  # Exit on error

# Configuration
PROJECT_NAME="igny8-staging"
COMPOSE_FILE="docker-compose.staging.yml"
ENV_FILE=".env.staging"
LOG_DIR="/data/app/logs/staging"
PROD_COMPOSE_FILE="docker-compose.yml"
PROD_ENV_FILE=".env"

# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color

# ==============================================================================
# FUNCTIONS
# ==============================================================================

log_info() {
  echo -e "${BLUE}[INFO]${NC} $1"
}

log_success() {
  echo -e "${GREEN}[SUCCESS]${NC} $1"
}

log_warn() {
  echo -e "${YELLOW}[WARN]${NC} $1"
}

log_error() {
  echo -e "${RED}[ERROR]${NC} $1"
}

check_prerequisites() {
  log_info "Checking prerequisites..."

  if ! command -v docker &> /dev/null; then
    log_error "Docker not installed"
    exit 1
  fi

  if ! command -v docker-compose &> /dev/null; then
    log_error "docker-compose not installed"
    exit 1
  fi

  if [ ! -f "$COMPOSE_FILE" ]; then
    log_error "File not found: $COMPOSE_FILE"
    exit 1
  fi

  if [ ! -f "$ENV_FILE" ]; then
    log_warn "File not found: $ENV_FILE"
    log_info "Creating $ENV_FILE from template..."
    # You could create a template here
    exit 1
  fi

  log_success "Prerequisites check passed"
}

verify_network() {
  log_info "Verifying Docker network..."

  if ! docker network inspect igny8_net &> /dev/null; then
    log_error "Docker network 'igny8_net' not found"
    log_info "Expected to be created by production setup (doc 00B)"
    exit 1
  fi

  log_success "Docker network verified"
}

verify_shared_services() {
  log_info "Verifying shared infrastructure services..."

  # Check PostgreSQL
  if ! docker exec igny8_postgres pg_isready -U postgres &> /dev/null; then
    log_error "PostgreSQL service not running"
    exit 1
  fi
  log_success "PostgreSQL verified"

  # Check Redis
  if ! docker exec igny8_redis redis-cli ping &> /dev/null; then
    log_error "Redis service not running"
    exit 1
  fi
  log_success "Redis verified"

  # Check Caddy
  if ! docker ps | grep -q igny8_caddy; then
    log_error "Caddy service not running"
    exit 1
  fi
  log_success "Caddy verified"
}

create_staging_database() {
  log_info "Creating staging database..."

  # Check if database already exists
  if docker exec igny8_postgres psql -U postgres -lqt | cut -d \| -f 1 | grep -qw igny8_staging_db; then
    log_warn "Database 'igny8_staging_db' already exists, skipping creation"
    return 0
  fi

  docker exec -i igny8_postgres psql -U postgres -d postgres << 'EOF'
CREATE DATABASE igny8_staging_db
  WITH OWNER igny8_user
  ENCODING 'UTF8'
  LOCALE 'en_US.UTF-8'
  TEMPLATE template0;

GRANT ALL PRIVILEGES ON DATABASE igny8_staging_db TO igny8_user;
GRANT ALL PRIVILEGES ON SCHEMA public TO igny8_user;
EOF

  log_success "Staging database created"
}

pull_images() {
  log_info "Pulling latest staging images..."

  docker pull igny8-backend:staging 2>/dev/null || log_warn "Could not pull igny8-backend:staging (may not exist in registry yet)"
  docker pull igny8-frontend-dev:staging 2>/dev/null || log_warn "Could not pull igny8-frontend-dev:staging (may not exist in registry yet)"
  docker pull igny8-marketing-dev:staging 2>/dev/null || log_warn "Could not pull igny8-marketing-dev:staging (may not exist in registry yet)"

  log_info "Image pull complete"
}

start_staging_containers() {
  log_info "Starting staging containers..."

  docker-compose \
    --project-name "$PROJECT_NAME" \
    --file "$COMPOSE_FILE" \
    --env-file "$ENV_FILE" \
    up -d

  log_success "Staging containers started"
}

run_migrations() {
  log_info "Running database migrations..."

  docker-compose \
    --project-name "$PROJECT_NAME" \
    --file "$COMPOSE_FILE" \
    --env-file "$ENV_FILE" \
    exec -T igny8_staging_backend python manage.py migrate

  log_success "Database migrations completed"
}

create_superuser() {
  log_info "Checking for superuser..."

  SUPERUSER_EXISTS=$(docker-compose \
    --project-name "$PROJECT_NAME" \
    --file "$COMPOSE_FILE" \
    --env-file "$ENV_FILE" \
    exec -T igny8_staging_backend python manage.py shell << 'EOF'
from django.contrib.auth.models import User
print(User.objects.filter(is_superuser=True).exists())
EOF
  )

  if [ "$SUPERUSER_EXISTS" = "True" ]; then
    log_warn "Superuser already exists, skipping creation"
    return 0
  fi

  log_info "Creating superuser..."

  docker-compose \
    --project-name "$PROJECT_NAME" \
    --file "$COMPOSE_FILE" \
    --env-file "$ENV_FILE" \
    exec -T igny8_staging_backend python manage.py createsuperuser \
      --username admin \
      --email admin@staging.igny8.com \
      --noinput

  # Set password from env or prompt
  docker-compose \
    --project-name "$PROJECT_NAME" \
    --file "$COMPOSE_FILE" \
    --env-file "$ENV_FILE" \
    exec -T igny8_staging_backend python manage.py shell << 'EOF'
from django.contrib.auth.models import User
user = User.objects.get(username='admin')
user.set_password('$DJANGO_SUPERUSER_PASSWORD')
user.save()
EOF

  log_success "Superuser created"
}

health_check() {
  log_info "Running health checks..."

  RETRIES=30
  RETRY_COUNT=0

  while [ $RETRY_COUNT -lt $RETRIES ]; do
    if curl -s -f "http://localhost:8012/health/" &> /dev/null; then
      log_success "Backend health check passed"
      return 0
    fi

    RETRY_COUNT=$((RETRY_COUNT + 1))
    if [ $RETRY_COUNT -lt $RETRIES ]; then
      sleep 2
    fi
  done

  log_warn "Backend health check failed after $RETRIES retries"
  return 1
}

print_summary() {
  log_success "Staging environment deployment completed!"
  echo ""
  echo -e "${BLUE}========================================${NC}"
  echo -e "${BLUE}STAGING ENVIRONMENT READY${NC}"
  echo -e "${BLUE}========================================${NC}"
  echo ""
  echo "Frontend:   https://staging.igny8.com"
  echo "API:        https://staging-api.igny8.com"
  echo "Marketing:  https://staging-marketing.igny8.com"
  echo "Admin:      https://staging-api.igny8.com/admin/"
  echo ""
  echo "Container Status:"
  docker-compose \
    --project-name "$PROJECT_NAME" \
    --file "$COMPOSE_FILE" \
    ps
  echo ""
}

# ==============================================================================
# MAIN
# ==============================================================================

main() {
  log_info "Starting staging environment deployment..."
  echo ""

  check_prerequisites
  verify_network
  verify_shared_services
  create_staging_database

  if [ "$1" = "pull-images" ]; then
    pull_images
  fi

  start_staging_containers
  run_migrations
  create_superuser
  health_check

  print_summary
}

main "$@"

Make script executable:

chmod +x deploy-staging.sh

Step 9: Create `sync-prod-to-staging.sh` Script

Location: Root project directory or scripts/sync-prod-to-staging.sh

#!/bin/bash
# ==============================================================================
# IGNY8 Production to Staging Data Sync Script
# ==============================================================================
# Usage: ./sync-prod-to-staging.sh [--full] [--backup]
# --full: Sync entire database (slower, more comprehensive)
# --backup: Create backup of staging DB before sync
# ==============================================================================

set -e  # Exit on error

# Configuration
PROD_DB="igny8_db"
STAGING_DB="igny8_staging_db"
DB_HOST="igny8_postgres"
DB_USER="igny8_user"
BACKUP_DIR="/data/backups/staging"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)

# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color

# Flags
FULL_SYNC=false
CREATE_BACKUP=false

# ==============================================================================
# FUNCTIONS
# ==============================================================================

log_info() {
  echo -e "${BLUE}[INFO]${NC} $1"
}

log_success() {
  echo -e "${GREEN}[SUCCESS]${NC} $1"
}

log_warn() {
  echo -e "${YELLOW}[WARN]${NC} $1"
}

log_error() {
  echo -e "${RED}[ERROR]${NC} $1"
}

parse_args() {
  while [[ $# -gt 0 ]]; do
    case $1 in
      --full)
        FULL_SYNC=true
        shift
        ;;
      --backup)
        CREATE_BACKUP=true
        shift
        ;;
      *)
        log_error "Unknown option: $1"
        exit 1
        ;;
    esac
  done
}

check_prerequisites() {
  log_info "Checking prerequisites..."

  if ! command -v docker &> /dev/null; then
    log_error "Docker not installed"
    exit 1
  fi

  # Check PostgreSQL accessibility
  if ! docker exec igny8_postgres pg_isready -U "$DB_USER" -h "$DB_HOST" &> /dev/null; then
    log_error "Cannot connect to PostgreSQL"
    exit 1
  fi

  log_success "Prerequisites check passed"
}

backup_staging_db() {
  if [ "$CREATE_BACKUP" = false ]; then
    return 0
  fi

  log_info "Creating backup of staging database..."

  mkdir -p "$BACKUP_DIR"

  BACKUP_FILE="$BACKUP_DIR/igny8_staging_db_${TIMESTAMP}.sql.gz"

  docker exec igny8_postgres pg_dump \
    -U "$DB_USER" \
    --format=plain \
    "$STAGING_DB" | gzip > "$BACKUP_FILE"

  log_success "Backup created: $BACKUP_FILE"
}

truncate_staging_tables() {
  log_info "Truncating staging database tables..."

  docker exec -i igny8_postgres psql \
    -U "$DB_USER" \
    -d "$STAGING_DB" << 'EOF'
-- Get list of all tables
DO $$ DECLARE
    r RECORD;
BEGIN
    FOR r IN (
        SELECT table_name
        FROM information_schema.tables
        WHERE table_schema = 'public'
        AND table_type = 'BASE TABLE'
    )
    LOOP
        EXECUTE 'TRUNCATE TABLE "' || r.table_name || '" CASCADE';
    END LOOP;
END $$;
EOF

  log_success "Staging database truncated"
}

dump_production_data() {
  log_info "Dumping production database..."

  DUMP_FILE="/tmp/igny8_prod_dump_${TIMESTAMP}.sql"

  docker exec igny8_postgres pg_dump \
    -U "$DB_USER" \
    --format=plain \
    "$PROD_DB" > "$DUMP_FILE"

  log_success "Production database dumped: $DUMP_FILE"
  echo "$DUMP_FILE"
}

restore_to_staging() {
  local DUMP_FILE=$1

  log_info "Restoring data to staging database..."

  cat "$DUMP_FILE" | docker exec -i igny8_postgres psql \
    -U "$DB_USER" \
    -d "$STAGING_DB" \
    --quiet

  log_success "Data restored to staging database"
}

handle_sensitive_data() {
  log_info "Anonymizing/resetting sensitive data in staging..."

  docker exec -i ighty8_postgres psql \
    -U "$DB_USER" \
    -d "$STAGING_DB" << 'EOF'
-- Reset payment information
UPDATE billing_paymentmethod SET token = NULL WHERE token IS NOT NULL;

-- Reset API tokens
UPDATE api_token SET token = 'staging-token-' || id WHERE 1=1;

-- Reset user passwords (set to default)
UPDATE auth_user SET password = 'pbkdf2_sha256$600000$abcdefg$hashed' WHERE is_staff = false;

-- Reset email addresses for non-admin users (optional - for testing)
-- UPDATE auth_user SET email = CONCAT(username, '@staging-test.local') WHERE is_staff = false;

-- Clear sensitive logs
DELETE FROM audit_log WHERE action_type IN ('payment', 'user_data_export');

-- Clear transient data
DELETE FROM celery_taskmeta;
DELETE FROM django_session;

-- Reset any third-party API keys to staging versions
UPDATE integration_apikey SET secret = 'sk_staging_' || id WHERE 1=1;
EOF

  log_success "Sensitive data handled"
}

sync_redis_cache() {
  log_info "Clearing staging Redis cache (DB 1)..."

  docker exec igny8_redis redis-cli -n 1 FLUSHDB

  log_success "Staging Redis cache cleared"
}

cleanup() {
  log_info "Cleaning up temporary files..."

  # Clean up old dump files (keep last 5)
  find /tmp -name "igny8_prod_dump_*.sql" -mtime +1 -delete

  log_success "Cleanup completed"
}

print_summary() {
  log_success "Data sync completed successfully!"
  echo ""
  echo -e "${BLUE}========================================${NC}"
  echo -e "${BLUE}SYNC SUMMARY${NC}"
  echo -e "${BLUE}========================================${NC}"
  echo ""
  echo "Source Database:      $PROD_DB"
  echo "Target Database:      $STAGING_DB"
  echo "Sync Type:            $([ "$FULL_SYNC" = true ] && echo 'Full' || echo 'Standard')"
  echo "Backup Created:       $([ "$CREATE_BACKUP" = true ] && echo 'Yes' || echo 'No')"
  echo "Timestamp:            $TIMESTAMP"
  echo ""
  echo "Next steps:"
  echo "1. Verify staging data is correct"
  echo "2. Test data-dependent features"
  echo "3. Run staging tests: docker-compose -f docker-compose.staging.yml exec igny8_staging_backend pytest"
  echo ""
}

# ==============================================================================
# MAIN
# ==============================================================================

main() {
  parse_args "$@"

  log_info "Starting production to staging data sync..."
  echo ""

  check_prerequisites
  backup_staging_db
  truncate_staging_tables

  DUMP_FILE=$(dump_production_data)
  restore_to_staging "$DUMP_FILE"

  handle_sensitive_data
  sync_redis_cache

  cleanup
  print_summary

  # Cleanup dump file
  rm -f "$DUMP_FILE"
}

main "$@"

Make script executable:

chmod +x sync-prod-to-staging.sh

Step 10: Deploy Staging Environment

Execute deployment:

# From project root
./deploy-staging.sh

# With image pull (if using registry)
./deploy-staging.sh pull-images

Verify deployment:

# Check container status
docker-compose -f docker-compose.staging.yml -p igny8-staging ps

# Check logs
docker-compose -f docker-compose.staging.yml -p igny8-staging logs -f

# Test API endpoint
curl -v https://staging-api.igny8.com/health/

# Test frontend
curl -v https://staging.igny8.com

5. Acceptance Criteria

5.1 Infrastructure Acceptance

Staging PostgreSQL database igny8_staging_db created and accessible
Redis DB 1 operational and isolated from production (DB 0)
Docker network igny8_net shared between prod and staging containers
/data/app/logs/staging/ directory created with proper permissions
All five staging containers running without errors:
- igny8_staging_backend
- igny8_staging_frontend
- igny8_staging_marketing_dev
- igny8_staging_celery_worker
- igny8_staging_celery_beat

5.2 Configuration Acceptance

docker-compose.staging.yml properly configured with all services
.env.staging populated with test credentials and sandbox keys
Caddyfile updated with three staging domain routes
Caddy successfully reloaded without errors
Cloudflare DNS records created for:
- staging.igny8.com
- staging-api.igny8.com
- staging-marketing.igny8.com

5.3 Application Acceptance

Database migrations run successfully on igny8_staging_db
Staging superuser created and accessible via admin interface
Frontend loads at https://staging.igny8.com without SSL errors
API accessible at https://staging-api.igny8.com with proper CORS headers
Marketing site loads at https://staging-marketing.igny8.com
Health check endpoint returns 200 at /health/

5.4 Data Synchronization Acceptance

sync-prod-to-staging.sh script successfully copies production data
Sensitive data (passwords, API keys, payment tokens) properly anonymized
Redis cache (DB 1) cleared after sync
Staging database contains realistic test data from production
Database backup created and stored in /data/backups/staging/

5.5 Monitoring & Logging Acceptance

All container logs written to /data/app/logs/staging/
Caddy access logs written to /data/app/logs/staging/staging-*.log
No errors in container startup sequences
Health checks passing for backend container
Application errors properly captured in logs

5.6 Deployment Automation Acceptance

deploy-staging.sh script runs without manual intervention
Prerequisites validation prevents incomplete deployments
Database creation idempotent (safe to run multiple times)
Container restart behavior correct (unless-stopped policy)
Script provides clear success/failure feedback

5.7 Git Workflow Acceptance

Staging branch (staging) created in Git
Feature branches merge to staging for testing
staging branch merges to main for production deployment
Branch protection rules configured (code review requirements)
CI/CD pipeline builds staging images on commits to staging branch

6. Claude Code Instructions

6.1 Pre-Deployment Tasks

Execute these tasks before running deployment scripts:

Tasks:
1. Verify all files exist:
   - docker-compose.staging.yml
   - .env.staging
   - deploy-staging.sh
   - sync-prod-to-staging.sh
   - Scripts must be executable (chmod +x)

2. Verify NEW VPS and production environment (00B must be complete):
   - NEW VPS provisioned with Ubuntu 24.04 LTS, Docker, PostgreSQL 18, Redis 8, Caddy 2.11
   - PostgreSQL running with igny8_db on PostgreSQL 18
   - Redis running with DB 0 active
   - Caddy running
   - docker-compose.yml (production) up and stable

3. Create logs directory:
   - sudo mkdir -p /data/app/logs/staging
   - sudo chmod 755 /data/app/logs/staging

4. Verify DNS configuration (coordinate with 00C 3-stage migration flow):
   - Determine current 00C stage (1: old provider, 2: new provider, 3: Cloudflare)
   - CNAME records created for staging subdomains on the appropriate active DNS provider
   - Wait for DNS propagation (5-10 minutes if newly created)
   - If DNS not yet ready, coordinate with 00C flow to complete Stage 2 (DNS flip) or Stage 3 (Cloudflare onboarding)
   - May use test variants (e.g., `test-staging.igny8.com`) if standard staging domains conflict during migration

5. Update .env.staging with actual values:
   - STAGING_SECRET_KEY (generate new, min 50 chars)
   - DB_PASSWORD (must match DATABASE_PASSWORD)
   - STAGING_STRIPE_PUBLIC_KEY (test key from Stripe)
   - STAGING_STRIPE_SECRET_KEY (test key from Stripe)
   - AWS credentials (staging bucket)
   - All other external service credentials

6.2 Deployment Workflow

Step 1: Validate Prerequisites

# In project root
docker-compose ps  # Verify production running
docker network inspect igny8_net  # Verify network
docker exec igny8_postgres pg_isready -U postgres  # Verify PostgreSQL
docker exec igny8_redis redis-cli ping  # Verify Redis

Step 2: Create Staging Database

docker exec -i igny8_postgres psql -U postgres -d postgres << 'EOF'
CREATE DATABASE igny8_staging_db
  WITH OWNER igny8_user
  ENCODING 'UTF8'
  LOCALE 'en_US.UTF-8'
  TEMPLATE template0;

GRANT ALL PRIVILEGES ON DATABASE igny8_staging_db TO igny8_user;
GRANT ALL PRIVILEGES ON SCHEMA public TO igny8_user;
EOF

Step 3: Update Caddyfile

# Edit /data/caddy/Caddyfile and append staging routes
# Reload: docker exec igny8_caddy caddy reload --config /etc/caddy/Caddyfile

Step 4: Build Images

cd /path/to/backend && docker build -t igny8-backend:staging .
cd /path/to/frontend && docker build -t igny8-frontend-dev:staging .
cd /path/to/marketing && docker build -t igny8-marketing-dev:staging .

Step 5: Deploy Staging

./deploy-staging.sh
# or with image pulling:
./deploy-staging.sh pull-images

Step 6: Verify Deployment

# Check containers
docker-compose -f docker-compose.staging.yml -p igny8-staging ps

# Check health
curl https://staging-api.igny8.com/health/

# Check logs
docker-compose -f docker-compose.staging.yml -p igny8-staging logs -f igny8_staging_backend

6.3 Post-Deployment Tasks

Tasks:
1. Sync production data (optional, for testing with real data):
   ./sync-prod-to-staging.sh --backup
   (Creates backup before overwriting staging data)

2. Access staging environment:
   - Frontend: https://staging.igny8.com
   - API: https://staging-api.igny8.com
   - Admin: https://staging-api.igny8.com/admin/
   - Username: admin
   - Password: (from .env.staging DJANGO_SUPERUSER_PASSWORD)

3. Run tests:
   docker-compose -f docker-compose.staging.yml -p igny8-staging exec igny8_staging_backend pytest

4. Monitor logs:
   tail -f /data/app/logs/staging/*.log

5. Document any issues found and file tickets against staging branch

6.4 Troubleshooting Commands

Container won't start:

docker-compose -f docker-compose.staging.yml -p igny8-staging logs igny8_staging_backend

Database connection failing:

docker exec igny8_postgres psql -U igny8_user -d igny8_staging_db -c "SELECT 1"

Redis connection failing:

docker exec igny8_redis redis-cli -n 1 ping

DNS not resolving:

nslookup staging.igny8.com
dig +short staging.igny8.com

Caddy route not working:

docker exec igny8_caddy caddy list-config
docker exec igny8_caddy caddy reload --config /etc/caddy/Caddyfile -v

Restart entire staging environment:

docker-compose -f docker-compose.staging.yml -p igny8-staging down
./deploy-staging.sh

Reset staging database:

docker exec igny8_postgres dropdb -U igny8_user igny8_staging_db
./deploy-staging.sh  # Recreates and migrations

00B Infrastructure Setup: NEW VPS provisioning, Docker, PostgreSQL 18, Redis 8, Caddy 2.11 configuration
- Version Matrix (in 00B): SINGLE SOURCE OF TRUTH for all component versions (PostgreSQL 18, Redis 8, Caddy 2.11, Python 3.14, Node 24 LTS, Django 6.0, Vite 8, Gunicorn 25, Celery 5.6, etc.)
- Staging environment on NEW VPS uses identical versions
00C Production Migration: 3-stage migration flow (DNS Preparation, DNS Flip, Cloudflare Onboarding), production database setup and initial deployment
- DNS Reference: Staging setup coordinates with 00C Stage 1/2/3 to determine active DNS provider and domain naming (staging may use test variants during migration)
- Prerequisite: 00B must be complete to provision the NEW VPS where staging runs. 00C determines which DNS provider is active for staging domain records.

8. Sign-Off

This document is complete and ready for implementation.

Document Version: 1.0 Last Updated: 2026-03-23 Status: Ready for Deployment Author: IGNY8 Phase 0 Planning

45 KiB Raw Blame History