salman/igny8
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
e78a41f11c75098698654684d7b92f1d39b13249
igny8/v2/V2-Execution-Docs/00D-staging-environment.md
IGNY8 VPS (Salman) e78a41f11c v2-exece-docs
2026-03-23 10:30:51 +00:00

44 KiB
Raw Blame History

IGNY8 Phase 0: Staging Environment Setup (Doc 00D)

Document Status: Build Specification Version: 2.1 Date Created: 2026-03-23 Target Phase: Phase 0 - Infrastructure & Deployment Source of Truth: Codebase at /data/app/igny8/ Related Docs: 00B Infrastructure Setup | 00C Production Migration

Key Details:

  • Staging runs on the NEW VPS (from 00B Infrastructure Setup)
  • Separate database: igny8_staging_db on PostgreSQL 18 instance
  • Redis DB 1 (production uses DB 0)
  • Separate ports: Backend 8012, Frontend 8024, Marketing 8026
  • Separate compose file: docker-compose.staging.yml with project name igny8-staging
  • DNS coordination with 00C 3-stage migration flow

1. Current State

Staging Environment Location: On the NEW VPS, as provisioned in 00B Infrastructure Setup.

Note on Versions: For component versions, the codebase (requirements.txt, package.json, Dockerfiles) is the source of truth. Aspirational upgrade targets are in 00B but current verified versions are: Python 3.11, Django >=5.2.7, Node 18, React ^19, Vite ^6.1.0, Celery >=5.3.0.

1.1 Infrastructure Baseline

  • Host Server: Single Linux VM running Docker on NEW VPS (from 00B Infrastructure Setup)
  • Base OS: Ubuntu (version per 00B)
  • Shared Resources:
    • PostgreSQL server (port 5432) — version set by infra stack
    • Redis server (port 6379) — version set by infra stack
    • Docker network: igny8_net
    • Caddy reverse proxy (port 80/443)
    • Cloudflare DNS management (may or may not be active — dependent on 00C flow stage)
    • Log directory: /data/app/logs/ (production), /data/logs/staging/ (staging)

1.2 Production Environment (Already Complete - Doc 00C)

  • Database: igny8_db (PostgreSQL)
  • Cache: Redis DB 0
  • Compose file: docker-compose.app.yml (project name: igny8-app)
  • Containers (7):
    • igny8_backend (host port 8011, container port 8010)
    • igny8_frontend (host port 8021, container port 5173)
    • igny8_marketing_dev (host port 8023, container port 5174)
    • igny8_celery_worker
    • igny8_celery_beat
    • igny8_flower (port 5555)
  • Env file: .env (production settings) — NOT used inline; env vars set in compose
  • Domains: igny8.com, api.igny8.com, marketing.igny8.com
  • Logs: /data/app/logs/

1.3 Staging Environment (To Be Built)

Does not yet exist. This document defines the complete staging setup.

2. What to Build

2.1 Staging Environment Architecture

A complete parallel environment sharing infrastructure with production:

┌─────────────────────────────────────────────────────┐
│         Docker Containers (Staging)                 │
├─────────────────────────────────────────────────────┤
│                                                     │
│  igny8_staging_backend:8012 → :8010 (Django/Gunicorn)  │
│  igny8_staging_frontend:8024 → :5173 (React/Vite)      │
│  igny8_staging_marketing_dev:8026 → :5174 (Vite)       │
│  igny8_staging_celery_worker                            │
│  igny8_staging_celery_beat                              │
│                                                     │
└─────────────────────────────────────────────────────┘
                        ↓
            igny8_net (Docker network)
                        ↓
┌─────────────────────────────────────────────────────┐
│      Shared Infrastructure Services                 │
├─────────────────────────────────────────────────────┤
│                                                     │
│  PostgreSQL 18 (port 5432)                         │
│  ├── igny8_db (production)                          │
│  └── igny8_staging_db (staging)                     │
│                                                     │
│  Redis 8 (port 6379)                               │
│  ├── DB 0 (production)                              │
│  └── DB 1 (staging)                                 │
│                                                     │
│  Caddy 2.11 (port 80/443) → Routes staging domains│
│                                                     │
└─────────────────────────────────────────────────────┘
                        ↓
        Cloudflare DNS → staging.igny8.com
                        api.staging.igny8.com
                        marketing.staging.igny8.com

2.2 Deliverables

  1. Staging PostgreSQL database (igny8_staging_db)
  2. Docker Compose file: docker-compose.staging.yml
  3. Environment file: .env.staging
  4. Caddyfile additions for staging route configuration
  5. Cloudflare DNS records for staging subdomains
  6. Staging Docker images (tagged :staging)
  7. Deployment script: deploy-staging.sh
  8. Sync script: sync-prod-to-staging.sh
  9. Logs directory: /data/app/logs/staging/

2.3 Key Characteristics

  • Independent Databases: Separate PostgreSQL DB and Redis DB
  • Shared Infrastructure: Same PostgreSQL server, Redis server, Docker network, Caddy
  • Isolated Configuration: Separate env file with test credentials, sandbox API keys
  • Feature Branch Testing: staging branch receives feature PRs before merging to main
  • Production Data Sync: Ability to copy production data to staging for realistic testing
  • Domain Isolation: Staging domains clearly identified (staging subdomain)

3. Data Models / APIs

3.1 PostgreSQL Database Setup

PostgreSQL Version: PostgreSQL 18 (postgres:18-alpine from 00B Version Matrix) Database Name: igny8_staging_db Owner: igny8_user (same as production) Encoding: UTF8 Locale: Same as production Extensions: Same as production (uuid-ossp, etc.)

Schema: Identical structure to production (migrations auto-generate on first startup)

3.2 Redis Setup

Redis Version: Redis 8 (redis:8-alpine from 00B Version Matrix) Instance: Same Redis server (port 6379) Database: DB 1 (production uses DB 0) Prefix: staging: (for key namespacing) Persistence: Same RDB/AOF configuration as production

3.3 Environment Variables

DJANGO_ENV: staging DEBUG: True (staging allows debug mode for testing) ALLOWED_HOSTS: staging.igny8.com,staging-api.igny8.com,localhost,127.0.0.1 API_KEYS: Test sandbox keys (different from production) PAYMENT_GATEWAY: Sandbox credentials (Stripe test keys, etc.) CELERY: Routes to staging Redis DB 1 CACHE: Routes to staging Redis DB 1 CORS: Allows staging frontend origins

3.4 API Endpoints

Component Production Staging
Frontend igny8.com staging.igny8.com
Backend API api.igny8.com staging-api.igny8.com
Marketing marketing.igny8.com staging-marketing.igny8.com

4. Implementation Steps

Version Requirements: All versions below are verified from the codebase. The staging environment uses identical versions to production:

  • PostgreSQL (version set by infra stack)
  • Redis (version set by infra stack)
  • Caddy 2 (version set by infra stack)
  • Ubuntu base OS (set by infra stack)
  • Docker Engine (installed on VPS)
  • Python 3.11-slim (in backend Dockerfile)
  • Node 18-alpine (in frontend Dockerfile)
  • Django >=5.2.7 (requirements.txt)
  • Vite ^6.1.0 (package.json)
  • Gunicorn (requirements.txt)
  • Celery >=5.3.0 (requirements.txt)

Note: The actual docker-compose.staging.yml already exists in the repo and is the source of truth. The compose excerpt below is for reference only — always use the actual file.

Step 1: Create Staging PostgreSQL Database

File: SQL commands to run on the host PostgreSQL server

-- Connect as superuser
psql -U postgres -h localhost

-- Create staging database
CREATE DATABASE igny8_staging_db
  WITH OWNER igny8_user
  ENCODING 'UTF8'
  LOCALE 'en_US.UTF-8'
  TEMPLATE template0;

-- Verify creation
\l | grep igny8

-- Grant privileges
GRANT ALL PRIVILEGES ON DATABASE igny8_staging_db TO igny8_user;
GRANT ALL PRIVILEGES ON SCHEMA public TO igny8_user;

Execution:

# On host server
docker exec -i postgres psql -U postgres -d postgres << 'EOF'
CREATE DATABASE igny8_staging_db
  WITH OWNER igny8
  ENCODING 'UTF8'
  LOCALE 'en_US.UTF-8'
  TEMPLATE template0;

GRANT ALL PRIVILEGES ON DATABASE igny8_staging_db TO igny8;
GRANT ALL PRIVILEGES ON SCHEMA public TO igny8;
EOF

echo "Staging database created"

Step 2: Create docker-compose.staging.yml

Location: Root of project directory (alongside docker-compose.yml) Project Name: igny8-staging

# Actual file: docker-compose.staging.yml (already exists in repo)
# Key differences from this reference: the actual file uses env_file: .env.staging
# and individual env vars (DB_HOST, DB_NAME) rather than DATABASE_URL format.

name: igny8-staging

services:
  # Backend API Service
  igny8_staging_backend:
    image: igny8-backend:staging
    container_name: igny8_staging_backend
    restart: always
    working_dir: /app
    ports:
      - "0.0.0.0:8012:8010"
    environment:
      DJANGO_ENV: staging
      DB_HOST: postgres          # External infra container name (NOT igny8_postgres)
      DB_NAME: igny8_staging_db
      DB_USER: igny8
      DB_PASSWORD: igny8pass
      REDIS_HOST: redis           # External infra container name (NOT igny8_redis)
      REDIS_PORT: "6379"
      REDIS_DB: "1"              # DB 1 for staging (production uses DB 0)
      USE_SECURE_COOKIES: "True"
      USE_SECURE_PROXY_HEADER: "True"
      DEBUG: "False"
    volumes:
      - /data/app/igny8/backend:/app:rw
      - /data/app/igny8:/data/app/igny8:rw
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /data/logs/staging:/app/logs:rw
    env_file:
      - .env.staging
    healthcheck:
      test: ["CMD-SHELL", "python -c \"import urllib.request; urllib.request.urlopen('http://localhost:8010/api/v1/system/status/').read()\" || exit 1"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s
    command: ["gunicorn", "igny8_core.wsgi:application", "--bind", "0.0.0.0:8010", "--workers", "2", "--timeout", "120"]
    networks: [igny8_net]

  # Frontend Service (React + Vite)
  igny8_staging_frontend:
    image: igny8-frontend-dev:staging
    container_name: igny8_staging_frontend
    restart: always
    ports:
      - "0.0.0.0:8024:5173"
    environment:
      VITE_BACKEND_URL: "https://staging-api.igny8.com/api"
      VITE_ENV: "staging"
    volumes:
      - /data/app/igny8/frontend:/app:rw
    depends_on:
      igny8_staging_backend:
        condition: service_healthy
    networks: [igny8_net]

  # Marketing Site Service (Vite, NOT Nuxt — built from frontend/Dockerfile.marketing.dev)
  igny8_staging_marketing_dev:
    image: igny8-marketing-dev:staging
    container_name: igny8_staging_marketing_dev
    restart: always
    ports:
      - "0.0.0.0:8026:5174"
    environment:
      VITE_BACKEND_URL: "https://staging-api.igny8.com/api"
      VITE_ENV: "staging"
    volumes:
      - /data/app/igny8/frontend:/app:rw    # Same frontend dir — marketing is a Vite build mode
    networks: [igny8_net]

  # Celery Worker
  igny8_staging_celery_worker:
    image: igny8-backend:staging
    container_name: igny8_staging_celery_worker
    restart: always
    working_dir: /app
    environment:
      DJANGO_ENV: staging
      DB_HOST: postgres
      DB_NAME: igny8_staging_db
      DB_USER: igny8
      DB_PASSWORD: igny8pass
      REDIS_HOST: redis
      REDIS_PORT: "6379"
      REDIS_DB: "1"
      C_FORCE_ROOT: "true"
    volumes:
      - /data/app/igny8/backend:/app:rw
      - /data/logs/staging:/app/logs:rw
    env_file:
      - .env.staging
    command: ["celery", "-A", "igny8_core", "worker", "--loglevel=info", "--concurrency=2"]
    depends_on:
      igny8_staging_backend:
        condition: service_healthy
    networks: [igny8_net]

  # Celery Beat (Scheduler)
  igny8_staging_celery_beat:
    image: igny8-backend:staging
    container_name: igny8_staging_celery_beat
    restart: always
    working_dir: /app
    environment:
      DJANGO_ENV: staging
      DB_HOST: postgres
      DB_NAME: igny8_staging_db
      DB_USER: igny8
      DB_PASSWORD: igny8pass
      REDIS_HOST: redis
      REDIS_PORT: "6379"
      REDIS_DB: "1"
      C_FORCE_ROOT: "true"
    volumes:
      - /data/app/igny8/backend:/app:rw
      - /data/logs/staging:/app/logs:rw
    env_file:
      - .env.staging
    command: ["celery", "-A", "igny8_core", "beat", "--loglevel=info", "--scheduler", "django_celery_beat.schedulers:DatabaseScheduler"]
    depends_on:
      igny8_staging_backend:
        condition: service_healthy
    networks: [igny8_net]

networks:
  igny8_net:
    external: true

Note: The actual docker-compose.staging.yml in the repo is the definitive version. The above is aligned with it as of this writing.

Step 3: Create .env.staging

Location: Root of project directory Purpose: Staging-specific environment variables

# ==============================================================================
# IGNY8 STAGING ENVIRONMENT CONFIGURATION
# ==============================================================================

# ==============================================================================
# GENERAL
# ==============================================================================
ENVIRONMENT=staging
DJANGO_ENV=staging
DEBUG=True
LOG_LEVEL=INFO

# ==============================================================================
# SECRETS (STAGING VERSIONS)
# ==============================================================================
# Generate new secrets for staging (do NOT copy from production)
STAGING_SECRET_KEY=your-generated-staging-secret-key-here-minimum-50-chars-random
DB_PASSWORD=your-staging-db-password

# ==============================================================================
# DATABASE
# ==============================================================================
DATABASE_ENGINE=postgresql
DATABASE_HOST=postgres
DATABASE_PORT=5432
DATABASE_NAME=igny8_staging_db
DATABASE_USER=igny8
DATABASE_PASSWORD=${DB_PASSWORD}

# ==============================================================================
# CACHE & QUEUE (REDIS DB 1 - Separate from Production)
# ==============================================================================
REDIS_HOST=redis
REDIS_PORT=6379
REDIS_DB=1
REDIS_PASSWORD=
REDIS_URL=redis://redis:6379/1
CACHE_URL=redis://redis:6379/1

# ==============================================================================
# CELERY (Uses Redis DB 1)
# ==============================================================================
CELERY_BROKER_URL=redis://redis:6379/1
CELERY_RESULT_BACKEND=redis://redis:6379/1
CELERY_ACCEPT_CONTENT=json
CELERY_TASK_SERIALIZER=json

# ==============================================================================
# HOSTS & DOMAINS (STAGING)
# ==============================================================================
ALLOWED_HOSTS=staging.igny8.com,staging-api.igny8.com,localhost,127.0.0.1
FRONTEND_URL=https://staging.igny8.com
API_BASE_URL=https://staging-api.igny8.com
MARKETING_URL=https://staging-marketing.igny8.com
CORS_ALLOWED_ORIGINS=https://staging.igny8.com,https://staging-marketing.igny8.com,http://localhost:8024,http://localhost:8026

# ==============================================================================
# PAYMENT GATEWAY (STRIPE - SANDBOX)
# ==============================================================================
STAGING_STRIPE_PUBLIC_KEY=pk_test_your_staging_stripe_public_key
STAGING_STRIPE_SECRET_KEY=sk_test_your_staging_stripe_secret_key
STAGING_STRIPE_WEBHOOK_SECRET=whsec_your_staging_webhook_secret

# ==============================================================================
# AWS / S3 (STAGING BUCKET)
# ==============================================================================
STAGING_AWS_ACCESS_KEY_ID=your-staging-aws-access-key
STAGING_AWS_SECRET_ACCESS_KEY=your-staging-aws-secret-key
STAGING_AWS_S3_BUCKET=igny8-staging-assets
AWS_REGION=us-east-1
AWS_S3_CUSTOM_DOMAIN=staging-assets.igny8.com

# ==============================================================================
# EXTERNAL SERVICES (STAGING / SANDBOX CREDENTIALS)
# ==============================================================================
# Email (app uses Resend, not Mailgun)
# RESEND_API_KEY=your-staging-resend-key

# Error Tracking (optional)
# SENTRY_DSN=https://your-staging-sentry-dsn

# ==============================================================================
# SECURITY (STAGING)
# ==============================================================================
SECURE_SSL_REDIRECT=True
SESSION_COOKIE_SECURE=True
CSRF_COOKIE_SECURE=True
SECURE_HSTS_SECONDS=31536000
SECURE_HSTS_INCLUDE_SUBDOMAINS=True

# ==============================================================================
# DJANGO SETTINGS
# ==============================================================================
DJANGO_SUPERUSER_USERNAME=admin
DJANGO_SUPERUSER_EMAIL=admin@staging.igny8.com
DJANGO_SUPERUSER_PASSWORD=your-staging-admin-password

# ==============================================================================
# FRONTEND (React + Vite)
# ==============================================================================
VITE_BACKEND_URL=https://staging-api.igny8.com/api
VITE_ENV=staging

Step 4: Update Caddyfile for Staging Routes

Location: /data/caddy/Caddyfile (append to existing production configuration)

# ==============================================================================
# STAGING ENVIRONMENT ROUTES
# ==============================================================================

# Staging API Backend
staging-api.igny8.com {
  reverse_proxy igny8_staging_backend:8010 {
    header_uri X-Forwarded-Proto https
    header_uri X-Forwarded-Host {host}
  }
  log {
    output file /data/app/logs/staging/staging-api-access.log {
      roll_size 100mb
      roll_keep 10
    }
  }
  encode gzip
}

# Staging Frontend
staging.igny8.com {
  reverse_proxy igny8_staging_frontend:5173 {
    header_uri X-Forwarded-Proto https
    header_uri X-Forwarded-Host {host}
  }
  log {
    output file /data/app/logs/staging/staging-frontend-access.log {
      roll_size 100mb
      roll_keep 10
    }
  }
  encode gzip
}

# Staging Marketing Site
staging-marketing.igny8.com {
  reverse_proxy igny8_staging_marketing_dev:5174 {
    header_uri X-Forwarded-Proto https
    header_uri X-Forwarded-Host {host}
  }
  log {
    output file /data/app/logs/staging/staging-marketing-access.log {
      roll_size 100mb
      roll_keep 10
    }
  }
  encode gzip
}

Reload Caddy:

# On host
docker exec igny8_caddy caddy reload --config /etc/caddy/Caddyfile

Step 5: Create DNS Records for Staging Subdomains

DNS Provider Timing: Staging setup is on the NEW VPS (from 00B). This assumes DNS infrastructure is at one of the stages in the 00C 3-stage migration flow:

  • 00C Stage 1 (DNS Preparation): Old provider still active - use old provider for staging DNS
  • 00C Stage 2 (DNS Flip): Migrating to new provider - add staging records to new provider
  • 00C Stage 3 (Cloudflare Onboarding): Cloudflare now active - add staging records to Cloudflare

Current DNS Provider: Use whichever DNS provider is currently managing igny8.com per the 00C flow:

  • If 00C Stage 3 complete (Cloudflare active): Use Cloudflare
  • If 00C Stage 2 in progress or complete: Use the new DNS provider
  • Otherwise: Use the current active DNS provider

Note on Staging Domains: Staging may use test variants (e.g., test-staging.igny8.com) during the migration period if production domains are already in use. Coordinate with 00C DNS flow to avoid conflicts.

Add the following DNS records to your active provider:

Type Name Content TTL Proxy
CNAME staging igny8.com 3600 Proxied (if Cloudflare)
CNAME staging-api igny8.com 3600 Proxied (if Cloudflare)
CNAME staging-marketing igny8.com 3600 Proxied (if Cloudflare)

SSL/TLS Setup:

  • If using Cloudflare: Ensure staging subdomains are included in SSL certificate (auto-included if using wildcard or zone certificate)
  • If using another DNS provider: Ensure Caddy can obtain certificates for staging subdomains (will auto-issue via ACME)

Wait for propagation: DNS changes typically propagate within 5-10 minutes.

Step 6: Build Staging Docker Images

Backend Image

cd /data/app/igny8/backend
docker build -f Dockerfile -t igny8-backend:staging .

# Verify
docker images | grep igny8-backend

Frontend Image (React/Vite)

cd /data/app/igny8/frontend
docker build -f Dockerfile.dev -t igny8-frontend-dev:staging .

# Verify
docker images | grep igny8-frontend-dev

Marketing Image (built from frontend dir using Dockerfile.marketing.dev)

cd /data/app/igny8/frontend
docker build -f Dockerfile.marketing.dev -t igny8-marketing-dev:staging .

# Verify
docker images | grep igny8-marketing-dev

Tag images for registry (optional, if using Docker Hub/ECR):

# Example: tag for Docker Hub
docker tag igny8-backend:staging yourregistry/igny8-backend:staging
docker tag igny8-frontend-dev:staging yourregistry/igny8-frontend-dev:staging
docker tag igny8-marketing-dev:staging yourregistry/igny8-marketing-dev:staging

# Push to registry
docker push yourregistry/igny8-backend:staging
docker push yourregistry/igny8-frontend-dev:staging
docker push yourregistry/igny8-marketing-dev:staging

Step 7: Create Logs Directory

# On host server
sudo mkdir -p /data/app/logs/staging
sudo chown -R 1000:1000 /data/app/logs/staging
sudo chmod 755 /data/app/logs/staging

Step 8: Create deploy-staging.sh Script

Location: Root project directory or scripts/deploy-staging.sh

#!/bin/bash
# ==============================================================================
# IGNY8 Staging Environment Deployment Script
# ==============================================================================
# Usage: ./deploy-staging.sh [pull-images]
# ==============================================================================

set -e  # Exit on error

# Configuration
PROJECT_NAME="igny8-staging"
COMPOSE_FILE="docker-compose.staging.yml"
ENV_FILE=".env.staging"
LOG_DIR="/data/app/logs/staging"
PROD_COMPOSE_FILE="docker-compose.app.yml"
PROD_ENV_FILE=".env"

# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color

# ==============================================================================
# FUNCTIONS
# ==============================================================================

log_info() {
  echo -e "${BLUE}[INFO]${NC} $1"
}

log_success() {
  echo -e "${GREEN}[SUCCESS]${NC} $1"
}

log_warn() {
  echo -e "${YELLOW}[WARN]${NC} $1"
}

log_error() {
  echo -e "${RED}[ERROR]${NC} $1"
}

check_prerequisites() {
  log_info "Checking prerequisites..."

  if ! command -v docker &> /dev/null; then
    log_error "Docker not installed"
    exit 1
  fi

  if ! command -v docker-compose &> /dev/null; then
    log_error "docker-compose not installed"
    exit 1
  fi

  if [ ! -f "$COMPOSE_FILE" ]; then
    log_error "File not found: $COMPOSE_FILE"
    exit 1
  fi

  if [ ! -f "$ENV_FILE" ]; then
    log_warn "File not found: $ENV_FILE"
    log_info "Creating $ENV_FILE from template..."
    # You could create a template here
    exit 1
  fi

  log_success "Prerequisites check passed"
}

verify_network() {
  log_info "Verifying Docker network..."

  if ! docker network inspect igny8_net &> /dev/null; then
    log_error "Docker network 'igny8_net' not found"
    log_info "Expected to be created by production setup (doc 00B)"
    exit 1
  fi

  log_success "Docker network verified"
}

verify_shared_services() {
  log_info "Verifying shared infrastructure services..."

  # Check PostgreSQL
  if ! docker exec postgres pg_isready -U postgres &> /dev/null; then
    log_error "PostgreSQL service not running"
    exit 1
  fi
  log_success "PostgreSQL verified"

  # Check Redis
  if ! docker exec redis redis-cli ping &> /dev/null; then
    log_error "Redis service not running"
    exit 1
  fi
  log_success "Redis verified"

  # Check Caddy
  if ! docker ps | grep -q caddy; then
    log_error "Caddy service not running"
    exit 1
  fi
  log_success "Caddy verified"
}

create_staging_database() {
  log_info "Creating staging database..."

  # Check if database already exists
  if docker exec postgres psql -U postgres -lqt | cut -d \| -f 1 | grep -qw igny8_staging_db; then
    log_warn "Database 'igny8_staging_db' already exists, skipping creation"
    return 0
  fi

  docker exec -i postgres psql -U postgres -d postgres << 'EOF'
CREATE DATABASE igny8_staging_db
  WITH OWNER igny8
  ENCODING 'UTF8'
  LOCALE 'en_US.UTF-8'
  TEMPLATE template0;

GRANT ALL PRIVILEGES ON DATABASE igny8_staging_db TO igny8;
GRANT ALL PRIVILEGES ON SCHEMA public TO igny8;
EOF

  log_success "Staging database created"
}

pull_images() {
  log_info "Pulling latest staging images..."

  docker pull igny8-backend:staging 2>/dev/null || log_warn "Could not pull igny8-backend:staging (may not exist in registry yet)"
  docker pull igny8-frontend-dev:staging 2>/dev/null || log_warn "Could not pull igny8-frontend-dev:staging (may not exist in registry yet)"
  docker pull igny8-marketing-dev:staging 2>/dev/null || log_warn "Could not pull igny8-marketing-dev:staging (may not exist in registry yet)"

  log_info "Image pull complete"
}

start_staging_containers() {
  log_info "Starting staging containers..."

  docker-compose \
    --project-name "$PROJECT_NAME" \
    --file "$COMPOSE_FILE" \
    --env-file "$ENV_FILE" \
    up -d

  log_success "Staging containers started"
}

run_migrations() {
  log_info "Running database migrations..."

  docker-compose \
    --project-name "$PROJECT_NAME" \
    --file "$COMPOSE_FILE" \
    --env-file "$ENV_FILE" \
    exec -T igny8_staging_backend python manage.py migrate

  log_success "Database migrations completed"
}

create_superuser() {
  log_info "Checking for superuser..."

  SUPERUSER_EXISTS=$(docker-compose \
    --project-name "$PROJECT_NAME" \
    --file "$COMPOSE_FILE" \
    --env-file "$ENV_FILE" \
    exec -T igny8_staging_backend python manage.py shell << 'EOF'
from django.contrib.auth import get_user_model
User = get_user_model()
print(User.objects.filter(is_superuser=True).exists())
EOF
  )

  if [ "$SUPERUSER_EXISTS" = "True" ]; then
    log_warn "Superuser already exists, skipping creation"
    return 0
  fi

  log_info "Creating superuser..."

  docker-compose \
    --project-name "$PROJECT_NAME" \
    --file "$COMPOSE_FILE" \
    --env-file "$ENV_FILE" \
    exec -T igny8_staging_backend python manage.py createsuperuser \
      --username admin \
      --email admin@staging.igny8.com \
      --noinput

  # Set password from env or prompt
  docker-compose \
    --project-name "$PROJECT_NAME" \
    --file "$COMPOSE_FILE" \
    --env-file "$ENV_FILE" \
    exec -T igny8_staging_backend python manage.py shell << 'EOF'
from django.contrib.auth import get_user_model
User = get_user_model()
user = User.objects.get(username='admin')
user.set_password('$DJANGO_SUPERUSER_PASSWORD')
user.save()
EOF

  log_success "Superuser created"
}

health_check() {
  log_info "Running health checks..."

  RETRIES=30
  RETRY_COUNT=0

  while [ $RETRY_COUNT -lt $RETRIES ]; do
    if curl -s -f "http://localhost:8012/api/v1/system/status/" &> /dev/null; then
      log_success "Backend health check passed"
      return 0
    fi

    RETRY_COUNT=$((RETRY_COUNT + 1))
    if [ $RETRY_COUNT -lt $RETRIES ]; then
      sleep 2
    fi
  done

  log_warn "Backend health check failed after $RETRIES retries"
  return 1
}

print_summary() {
  log_success "Staging environment deployment completed!"
  echo ""
  echo -e "${BLUE}========================================${NC}"
  echo -e "${BLUE}STAGING ENVIRONMENT READY${NC}"
  echo -e "${BLUE}========================================${NC}"
  echo ""
  echo "Frontend:   https://staging.igny8.com"
  echo "API:        https://staging-api.igny8.com"
  echo "Marketing:  https://staging-marketing.igny8.com"
  echo "Admin:      https://staging-api.igny8.com/admin/"
  echo ""
  echo "Container Status:"
  docker-compose \
    --project-name "$PROJECT_NAME" \
    --file "$COMPOSE_FILE" \
    ps
  echo ""
}

# ==============================================================================
# MAIN
# ==============================================================================

main() {
  log_info "Starting staging environment deployment..."
  echo ""

  check_prerequisites
  verify_network
  verify_shared_services
  create_staging_database

  if [ "$1" = "pull-images" ]; then
    pull_images
  fi

  start_staging_containers
  run_migrations
  create_superuser
  health_check

  print_summary
}

main "$@"

Make script executable:

chmod +x deploy-staging.sh

Step 9: Create sync-prod-to-staging.sh Script

Location: Root project directory or scripts/sync-prod-to-staging.sh

#!/bin/bash
# ==============================================================================
# IGNY8 Production to Staging Data Sync Script
# ==============================================================================
# Usage: ./sync-prod-to-staging.sh [--full] [--backup]
# --full: Sync entire database (slower, more comprehensive)
# --backup: Create backup of staging DB before sync
# ==============================================================================

set -e  # Exit on error

# Configuration
PROD_DB="igny8_db"
STAGING_DB="igny8_staging_db"
DB_HOST="postgres"
DB_USER="igny8"
BACKUP_DIR="/data/backups/staging"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)

# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color

# Flags
FULL_SYNC=false
CREATE_BACKUP=false

# ==============================================================================
# FUNCTIONS
# ==============================================================================

log_info() {
  echo -e "${BLUE}[INFO]${NC} $1"
}

log_success() {
  echo -e "${GREEN}[SUCCESS]${NC} $1"
}

log_warn() {
  echo -e "${YELLOW}[WARN]${NC} $1"
}

log_error() {
  echo -e "${RED}[ERROR]${NC} $1"
}

parse_args() {
  while [[ $# -gt 0 ]]; do
    case $1 in
      --full)
        FULL_SYNC=true
        shift
        ;;
      --backup)
        CREATE_BACKUP=true
        shift
        ;;
      *)
        log_error "Unknown option: $1"
        exit 1
        ;;
    esac
  done
}

check_prerequisites() {
  log_info "Checking prerequisites..."

  if ! command -v docker &> /dev/null; then
    log_error "Docker not installed"
    exit 1
  fi

  # Check PostgreSQL accessibility (container name is 'postgres', not 'igny8_postgres')
  if ! docker exec postgres pg_isready -U "$DB_USER" &> /dev/null; then
    log_error "Cannot connect to PostgreSQL"
    exit 1
  fi

  log_success "Prerequisites check passed"
}

backup_staging_db() {
  if [ "$CREATE_BACKUP" = false ]; then
    return 0
  fi

  log_info "Creating backup of staging database..."

  mkdir -p "$BACKUP_DIR"

  BACKUP_FILE="$BACKUP_DIR/igny8_staging_db_${TIMESTAMP}.sql.gz"

  docker exec postgres pg_dump \
    -U "$DB_USER" \
    --format=plain \
    "$STAGING_DB" | gzip > "$BACKUP_FILE"

  log_success "Backup created: $BACKUP_FILE"
}

truncate_staging_tables() {
  log_info "Truncating staging database tables..."

  docker exec -i postgres psql \
    -U "$DB_USER" \
    -d "$STAGING_DB" << 'EOF'
-- Get list of all tables
DO $$ DECLARE
    r RECORD;
BEGIN
    FOR r IN (
        SELECT table_name
        FROM information_schema.tables
        WHERE table_schema = 'public'
        AND table_type = 'BASE TABLE'
    )
    LOOP
        EXECUTE 'TRUNCATE TABLE "' || r.table_name || '" CASCADE';
    END LOOP;
END $$;
EOF

  log_success "Staging database truncated"
}

dump_production_data() {
  log_info "Dumping production database..."

  DUMP_FILE="/tmp/igny8_prod_dump_${TIMESTAMP}.sql"

  docker exec postgres pg_dump \
    -U "$DB_USER" \
    --format=plain \
    "$PROD_DB" > "$DUMP_FILE"

  log_success "Production database dumped: $DUMP_FILE"
  echo "$DUMP_FILE"
}

restore_to_staging() {
  local DUMP_FILE=$1

  log_info "Restoring data to staging database..."

  cat "$DUMP_FILE" | docker exec -i postgres psql \
    -U "$DB_USER" \
    -d "$STAGING_DB" \
    --quiet

  log_success "Data restored to staging database"
}

handle_sensitive_data() {
  log_info "Anonymizing/resetting sensitive data in staging..."

  docker exec -i postgres psql \
    -U "$DB_USER" \
    -d "$STAGING_DB" << 'EOF'
-- Reset/anonymize sensitive data in staging using ACTUAL table names
-- (All tables prefixed with igny8_ — see 00C for full table list)

-- Reset user passwords for non-staff users (set to a known staging password hash)
UPDATE igny8_users SET password = 'pbkdf2_sha256$600000$stagingsalt$hashedvalue' WHERE is_staff = false;

-- Clear payment tokens (integration keys are in DB, not env vars)
-- Integration settings are in igny8_integration_settings and igny8_integration_providers
-- Do NOT delete these — just note they need to be updated to sandbox keys post-sync

-- Clear webhook event records (contain real payment data)
DELETE FROM igny8_webhook_events;

-- Clear transient data
DELETE FROM django_session;

-- Clear AI task logs (optional — may contain API call details)
-- DELETE FROM igny8_ai_task_logs;

-- NOTE: After sync, manually update igny8_integration_settings to use sandbox API keys
-- for openai, stripe, paypal, runware, resend providers
EOF

  log_success "Sensitive data handled"
}

sync_redis_cache() {
  log_info "Clearing staging Redis cache (DB 1)..."

  docker exec redis redis-cli -n 1 FLUSHDB

  log_success "Staging Redis cache cleared"
}

cleanup() {
  log_info "Cleaning up temporary files..."

  # Clean up old dump files (keep last 5)
  find /tmp -name "igny8_prod_dump_*.sql" -mtime +1 -delete

  log_success "Cleanup completed"
}

print_summary() {
  log_success "Data sync completed successfully!"
  echo ""
  echo -e "${BLUE}========================================${NC}"
  echo -e "${BLUE}SYNC SUMMARY${NC}"
  echo -e "${BLUE}========================================${NC}"
  echo ""
  echo "Source Database:      $PROD_DB"
  echo "Target Database:      $STAGING_DB"
  echo "Sync Type:            $([ "$FULL_SYNC" = true ] && echo 'Full' || echo 'Standard')"
  echo "Backup Created:       $([ "$CREATE_BACKUP" = true ] && echo 'Yes' || echo 'No')"
  echo "Timestamp:            $TIMESTAMP"
  echo ""
  echo "Next steps:"
  echo "1. Verify staging data is correct"
  echo "2. Test data-dependent features"
  echo "3. Run staging tests: docker-compose -f docker-compose.staging.yml exec igny8_staging_backend pytest"
  echo ""
}

# ==============================================================================
# MAIN
# ==============================================================================

main() {
  parse_args "$@"

  log_info "Starting production to staging data sync..."
  echo ""

  check_prerequisites
  backup_staging_db
  truncate_staging_tables

  DUMP_FILE=$(dump_production_data)
  restore_to_staging "$DUMP_FILE"

  handle_sensitive_data
  sync_redis_cache

  cleanup
  print_summary

  # Cleanup dump file
  rm -f "$DUMP_FILE"
}

main "$@"

Make script executable:

chmod +x sync-prod-to-staging.sh

Step 10: Deploy Staging Environment

Execute deployment:

# From project root
./deploy-staging.sh

# With image pull (if using registry)
./deploy-staging.sh pull-images

Verify deployment:

# Check container status
docker-compose -f docker-compose.staging.yml -p igny8-staging ps

# Check logs
docker-compose -f docker-compose.staging.yml -p igny8-staging logs -f

# Test API endpoint
curl -v https://staging-api.igny8.com/api/v1/system/status/

# Test frontend
curl -v https://staging.igny8.com

5. Acceptance Criteria

5.1 Infrastructure Acceptance

  • Staging PostgreSQL database igny8_staging_db created and accessible
  • Redis DB 1 operational and isolated from production (DB 0)
  • Docker network igny8_net shared between prod and staging containers
  • /data/app/logs/staging/ directory created with proper permissions
  • All five staging containers running without errors:
    • igny8_staging_backend
    • igny8_staging_frontend
    • igny8_staging_marketing_dev
    • igny8_staging_celery_worker
    • igny8_staging_celery_beat

5.2 Configuration Acceptance

  • docker-compose.staging.yml properly configured with all services
  • .env.staging populated with test credentials and sandbox keys
  • Caddyfile updated with three staging domain routes
  • Caddy successfully reloaded without errors
  • Cloudflare DNS records created for:
    • staging.igny8.com
    • staging-api.igny8.com
    • staging-marketing.igny8.com

5.3 Application Acceptance

  • Database migrations run successfully on igny8_staging_db
  • Staging superuser created and accessible via admin interface
  • Frontend loads at https://staging.igny8.com without SSL errors
  • API accessible at https://staging-api.igny8.com with proper CORS headers
  • Marketing site loads at https://staging-marketing.igny8.com
  • Health check endpoint returns 200 at /api/v1/system/status/

5.4 Data Synchronization Acceptance

  • sync-prod-to-staging.sh script successfully copies production data
  • Sensitive data (passwords, API keys, payment tokens) properly anonymized
  • Redis cache (DB 1) cleared after sync
  • Staging database contains realistic test data from production
  • Database backup created and stored in /data/backups/staging/

5.5 Monitoring & Logging Acceptance

  • All container logs written to /data/app/logs/staging/
  • Caddy access logs written to /data/app/logs/staging/staging-*.log
  • No errors in container startup sequences
  • Health checks passing for backend container
  • Application errors properly captured in logs

5.6 Deployment Automation Acceptance

  • deploy-staging.sh script runs without manual intervention
  • Prerequisites validation prevents incomplete deployments
  • Database creation idempotent (safe to run multiple times)
  • Container restart behavior correct (unless-stopped policy)
  • Script provides clear success/failure feedback

5.7 Git Workflow Acceptance

  • Staging branch (staging) created in Git
  • Feature branches merge to staging for testing
  • staging branch merges to main for production deployment
  • Branch protection rules configured (code review requirements)
  • CI/CD pipeline builds staging images on commits to staging branch

6. Claude Code Instructions

6.1 Pre-Deployment Tasks

Execute these tasks before running deployment scripts:

Tasks:
1. Verify all files exist:
   - docker-compose.staging.yml
   - .env.staging
   - deploy-staging.sh
   - sync-prod-to-staging.sh
   - Scripts must be executable (chmod +x)

2. Verify NEW VPS and production environment (00B must be complete):
   - NEW VPS provisioned with Ubuntu 24.04 LTS, Docker, PostgreSQL 18, Redis 8, Caddy 2.11
   - PostgreSQL running with igny8_db on PostgreSQL 18
   - Redis running with DB 0 active
   - Caddy running
   - docker-compose.yml (production) up and stable

3. Create logs directory:
   - sudo mkdir -p /data/app/logs/staging
   - sudo chmod 755 /data/app/logs/staging

4. Verify DNS configuration (coordinate with 00C 3-stage migration flow):
   - Determine current 00C stage (1: old provider, 2: new provider, 3: Cloudflare)
   - CNAME records created for staging subdomains on the appropriate active DNS provider
   - Wait for DNS propagation (5-10 minutes if newly created)
   - If DNS not yet ready, coordinate with 00C flow to complete Stage 2 (DNS flip) or Stage 3 (Cloudflare onboarding)
   - May use test variants (e.g., `test-staging.igny8.com`) if standard staging domains conflict during migration

5. Update .env.staging with actual values:
   - STAGING_SECRET_KEY (generate new, min 50 chars)
   - DB_PASSWORD (must match DATABASE_PASSWORD)
   - STAGING_STRIPE_PUBLIC_KEY (test key from Stripe)
   - STAGING_STRIPE_SECRET_KEY (test key from Stripe)
   - AWS credentials (staging bucket)
   - All other external service credentials

6.2 Deployment Workflow

Step 1: Validate Prerequisites

# In project root
docker-compose ps  # Verify production running
docker network inspect igny8_net  # Verify network
docker exec postgres pg_isready -U postgres  # Verify PostgreSQL
docker exec redis redis-cli ping  # Verify Redis

Step 2: Create Staging Database

docker exec -i postgres psql -U postgres -d postgres << 'EOF'
CREATE DATABASE igny8_staging_db
  WITH OWNER igny8_user
  ENCODING 'UTF8'
  LOCALE 'en_US.UTF-8'
  TEMPLATE template0;

GRANT ALL PRIVILEGES ON DATABASE igny8_staging_db TO igny8_user;
GRANT ALL PRIVILEGES ON SCHEMA public TO igny8_user;
EOF

Step 3: Update Caddyfile

# Edit /data/caddy/Caddyfile and append staging routes
# Reload: docker exec caddy caddy reload --config /etc/caddy/Caddyfile

Step 4: Build Images

cd /path/to/backend && docker build -t igny8-backend:staging .
cd /path/to/frontend && docker build -t igny8-frontend-dev:staging .
cd /path/to/marketing && docker build -t igny8-marketing-dev:staging .

Step 5: Deploy Staging

./deploy-staging.sh
# or with image pulling:
./deploy-staging.sh pull-images

Step 6: Verify Deployment

# Check containers
docker-compose -f docker-compose.staging.yml -p igny8-staging ps

# Check health
curl https://staging-api.igny8.com/api/v1/system/status/

# Check logs
docker-compose -f docker-compose.staging.yml -p igny8-staging logs -f igny8_staging_backend

6.3 Post-Deployment Tasks

Tasks:
1. Sync production data (optional, for testing with real data):
   ./sync-prod-to-staging.sh --backup
   (Creates backup before overwriting staging data)

2. Access staging environment:
   - Frontend: https://staging.igny8.com
   - API: https://staging-api.igny8.com
   - Admin: https://staging-api.igny8.com/admin/
   - Username: admin
   - Password: (from .env.staging DJANGO_SUPERUSER_PASSWORD)

3. Run tests:
   docker-compose -f docker-compose.staging.yml -p igny8-staging exec igny8_staging_backend pytest

4. Monitor logs:
   tail -f /data/app/logs/staging/*.log

5. Document any issues found and file tickets against staging branch

6.4 Troubleshooting Commands

Container won't start:

docker-compose -f docker-compose.staging.yml -p igny8-staging logs igny8_staging_backend

Database connection failing:

docker exec postgres psql -U igny8 -d igny8_staging_db -c "SELECT 1"

Redis connection failing:

docker exec redis redis-cli -n 1 ping

DNS not resolving:

nslookup staging.igny8.com
dig +short staging.igny8.com

Caddy route not working:

docker exec caddy caddy list-config
docker exec caddy caddy reload --config /etc/caddy/Caddyfile -v

Restart entire staging environment:

docker-compose -f docker-compose.staging.yml -p igny8-staging down
./deploy-staging.sh

Reset staging database:

docker exec postgres dropdb -U igny8 igny8_staging_db
./deploy-staging.sh  # Recreates and migrations

7. Related Documentation

  • 00B Infrastructure Setup: NEW VPS provisioning, Docker, PostgreSQL, Redis, Caddy configuration. Contains aspirational version targets; current versions verified from codebase (Python 3.11, Django >=5.2.7, Node 18, etc.)
  • 00C Production Migration: 3-stage migration flow (Deploy & Test, DNS Flip, Cloudflare Onboarding). DNS Reference: Staging setup coordinates with 00C stages to determine active DNS provider.
  • Codebase files: docker-compose.staging.yml (actual staging compose), docker-compose.app.yml (production compose), backend/requirements.txt (Python deps), frontend/package.json (JS deps).

8. Sign-Off

This document is complete and ready for implementation.

Document Version: 1.0 Last Updated: 2026-03-23 Status: Ready for Deployment Author: IGNY8 Phase 0 Planning

Reference in New Issue View Git Blame Copy Permalink