salman/igny8

Fork 0

Files

IGNY8 VPS (Salman) 128b186865 temproary docs uplaoded

2026-03-23 09:02:49 +00:00

28 KiB

Raw Blame History

IGNY8 Phase 0: VPS Provisioning (00B)

Status: In Development Date Created: 2026-03-23 Phase: 0 (Infrastructure Setup) Document ID: 00B

1. Current State

1.1 Migration Context

IGNY8 is transitioning from legacy infrastructure to a new KVM 4 VPS environment. The current production setup includes:

Production Domains: app.igny8.com, api.igny8.com, igny8.com (marketing)
Staging Domains: staging.igny8.com, staging-api.igny8.com, staging-marketing.igny8.com
Current DNS Provider: NOT Cloudflare — domains currently point to the OLD VPS via existing DNS provider
Current Stack Health: All services healthy on legacy infrastructure; zero downtime migration required

1.2 DNS Flow Clarification (CRITICAL)

This is NOT a Cloudflare-first migration. The correct approach:

Current State: Domains resolve via current DNS provider → OLD VPS (working, healthy)
During 00B (this doc):
- Provision new VPS infrastructure
- Create TEMPORARY TEST subdomains (test-app.igny8.com, test-api.igny8.com, test-marketing.igny8.com) at the CURRENT DNS provider pointing to NEW VPS IP
- Caddy on new server handles BOTH production domain names AND test subdomains (same containers, different hostnames)
- Real production domains still resolve to old VPS via existing DNS — no traffic shift yet
After 00B (covered in 00C): DNS cutover happens — production domains flipped to new VPS
Cloudflare Migration: Happens ONLY AFTER new server is fully healthy and production DNS has been migrated (end of 00C)

SSL Certificates: Caddy uses Let's Encrypt for auto-HTTPS on test subdomains. NO Cloudflare ACME tokens needed in this phase.

1.3 Previous Infrastructure Details

The legacy setup has these key characteristics:

Shared Services: PostgreSQL, Redis, Caddy reverse proxy, PgAdmin, FileBrowser, Portainer
Application Services: Defined in separate docker-compose.app.yml per environment
External Docker Network: igny8_net (connects all services across compose files)
Directory Structure: /data/stack/, /data/app/, /data/logs/, /data/backups/
Configuration: Caddyfile for SSL/TLS termination, utility scripts in igny8-stack repo
Installation Script: install.sh in igny8-stack repo handles automated setup

1.4 Known Constraints

OS must be Ubuntu 24.04 LTS or later
SSH access only (no root password login)
Firewall restrictions (ports 80, 443, 22)
Docker-first architecture (no bare-metal services)
Test subdomains are temporary — removed after successful production DNS migration

2. Target Version Matrix

This section is the single source of truth for all target versions across the entire V2 build.

2.1 Infrastructure & OS

Component	Version	Image/Package	Notes
Ubuntu	24.04 LTS (Noble Numbat)	Official ISO	VPS base OS
Docker Engine	29.x	docker-ce	Latest stable from Docker repo
Docker Compose	2.x+	docker-compose-plugin	Via Docker repo
UFW Firewall	Latest	ufw	System package

2.2 Shared Infrastructure Services

Service	Version	Image	Port (internal)	Port (public)	Volume
PostgreSQL	18	`postgres:18-alpine`	5432	N/A	`/data/backups/postgres/`
Redis	8.x	`redis:8-alpine`	6379	N/A	`/data/backups/redis/`
Caddy	2.11.x	`caddy:2-alpine`	80, 443	80, 443	`/data/backups/caddy/`
Portainer CE	2.27+	`portainer/portainer-ce:lts`	9000	Via Caddy	`/var/run/docker.sock`
PgAdmin	Latest	`dpage/pgadmin4:latest`	5050	Via Caddy	N/A
FileBrowser	Latest	`filebrowser/filebrowser:latest`	8080	Via Caddy	`/data/`

2.3 Application Stack (reference — installed during 00C/00D)

Component	Version	Notes
Python	3.14	For backend Dockerfile
Node.js	24 LTS	For frontend Dockerfile
Django	6.0	Backend framework
Django REST Framework	Latest	API serializers
Celery	5.6	Task queue
Gunicorn	25	WSGI application server
Vite	8	Frontend build tool
React	Latest	Frontend library

3. What to Build

3.1 Deliverables

A fully provisioned, hardened KVM 4 VPS that:

Runs Ubuntu 24.04 LTS with essential system tools
Has Docker Engine 29.x and Docker Compose 2.x+ installed and ready
Implements UFW firewall with minimal port exposure (22, 80, 443)
Enforces SSH key-based authentication (no password login)
Hosts all shared infrastructure services (PostgreSQL 18, Redis 8, Caddy 2.11, Portainer 2.27+)
Maintains /data/ directory structure for persistence
Caddy configured with routes for BOTH production AND test subdomains
Test subdomains resolve to new VPS and pass health checks
Passes health checks for all infrastructure services
Is ready for application deployment (see 00C for production, 00D for staging)

3.2 Out of Scope

Application container deployment (covered in 00C/00D)
Cloudflare DNS migration (covered at end of 00C)
Backup automation (post-Phase 0)
Monitoring/alerting setup (post-Phase 0)
Custom application configuration

4. Data Models / APIs

4.1 Service Specifications

PostgreSQL 18

Port: 5432 (internal, not exposed)
Image: postgres:18-alpine
Volumes: /data/backups/postgres/ (data persistence)
Environment Variables:
- POSTGRES_USER=igny8
- POSTGRES_PASSWORD=${DB_PASSWORD} (from .env)
- POSTGRES_INITDB_ARGS=-c shared_preload_libraries=pg_stat_statements

Redis 8.x

Port: 6379 (internal, not exposed)
Image: redis:8-alpine
Volumes: /data/backups/redis/ (persistence)
Command: redis-server --appendonly yes

Caddy 2.11.x

Ports: 80/443 (exposed)
Image: caddy:2-alpine
Config: /data/stack/Caddyfile
Volumes: /data/backups/caddy/ (SSL certs)
Auto-HTTPS: Enabled (via Let's Encrypt, not Cloudflare)
Routes: Production domains + test subdomains (same backend services)

Portainer CE 2.27+

Port: 9000 (internal, not exposed; proxied via Caddy)
Image: portainer/portainer-ce:lts
Volumes: /var/run/docker.sock (Docker daemon access)

PgAdmin

Port: 5050 (internal, not exposed; proxied via Caddy)
Image: dpage/pgadmin4:latest
Environment:
- PGADMIN_DEFAULT_EMAIL=admin@igny8.com
- PGADMIN_DEFAULT_PASSWORD=${PGADMIN_PASSWORD} (from .env)

FileBrowser

Port: 8080 (internal, not exposed; proxied via Caddy)
Image: filebrowser/filebrowser:latest
Root: /data/

4.2 Docker Network

Network Name: igny8_net
Type: External bridge network
Purpose: Allows both docker-compose.infra.yml and docker-compose.app.yml to communicate
All infrastructure services connected to igny8_net

4.3 Environment Variables

Create .env file at /data/stack/.env:

# PostgreSQL
DB_PASSWORD=<strong-random-password>
DB_USER=igny8

# PgAdmin
PGADMIN_PASSWORD=<strong-random-password>

# Portainer
PORTAINER_PASSWORD=<strong-random-password>

# Application Domains (for reference in Caddyfile)
PROD_DOMAIN=igny8.com
PROD_APP_DOMAIN=app.igny8.com
PROD_API_DOMAIN=api.igny8.com

STAGING_DOMAIN=staging.igny8.com
STAGING_APP_DOMAIN=staging.igny8.com
STAGING_API_DOMAIN=staging-api.igny8.com

# Test Domains (temporary, point to new VPS during migration)
TEST_DOMAIN=test-marketing.igny8.com
TEST_APP_DOMAIN=test-app.igny8.com
TEST_API_DOMAIN=test-api.igny8.com

NOTE: No CF_API_TOKEN or CF_ZONE_ID in this phase. Cloudflare integration happens in 00C.

4.4 Docker Compose Files

docker-compose.infra.yml (shared infrastructure):

PostgreSQL 18
Redis 8.x
Caddy 2.11.x
Portainer CE 2.27+
PgAdmin
FileBrowser

docker-compose.app.yml (application services, per environment):

Production application containers (deployed in 00C)
Staging application containers (deployed in 00D)
Linked to igny8_net external network

5. Implementation Steps

5.1 Pre-Provisioning Checklist

Before starting, have ready:

VPS login credentials (IP address, initial SSH access)
Current DNS provider credentials (to add test subdomains)
Strong passwords for DB_PASSWORD, PGADMIN_PASSWORD, PORTAINER_PASSWORD
SSH public key for hardening
igny8-stack repository cloned locally or accessible
Plan for test subdomain creation at current DNS provider

5.2 Phase 0: System Provisioning

Step 0.1: Initial Access & Updates

# SSH into VPS as root (initial access)
ssh root@<VPS_IP>

# Update system packages
apt-get update
apt-get upgrade -y

# Install essential utilities
apt-get install -y \
  curl \
  wget \
  git \
  vim \
  htop \
  net-tools \
  ufw \
  openssh-server \
  openssh-client

# Set timezone
timedatectl set-timezone UTC

# Verify Ubuntu 24.04 LTS
lsb_release -a
# Should output: Release: 24.04

# Verify kernel
uname -r

Step 0.2: Create Non-Root User

# Create new user (replace 'igny8' with preferred username)
adduser --disabled-password --gecos "" igny8

# Add user to sudoers
usermod -aG sudo igny8

# Create SSH directory for user
mkdir -p /home/igny8/.ssh
chmod 700 /home/igny8/.ssh

Step 0.3: SSH Hardening

# (From local machine) Copy public SSH key to VPS
ssh-copy-id -i ~/.ssh/id_rsa.pub igny8@<VPS_IP>

# SSH into VPS as new user
ssh igny8@<VPS_IP>

# Edit SSH daemon config (as igny8 with sudo)
sudo vi /etc/ssh/sshd_config

# Make these changes:
# PermitRootLogin no
# PasswordAuthentication no
# PubkeyAuthentication yes
# X11Forwarding no
# PrintMotd no
# AcceptEnv LANG LC_*
# Subsystem sftp /usr/lib/openssh/sftp-server

# Restart SSH daemon
sudo systemctl restart sshd

# Test from another terminal to ensure key-based auth works
ssh igny8@<VPS_IP>  # Should connect without password

Step 0.4: Configure UFW Firewall

# Enable UFW
sudo ufw enable

# Set default policies
sudo ufw default deny incoming
sudo ufw default allow outgoing

# Allow SSH (critical before locking down)
sudo ufw allow 22/tcp

# Allow HTTP/HTTPS (for Caddy)
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

# Verify rules
sudo ufw status verbose

# Output should show:
# To            Action      From
# --            ------      ----
# 22/tcp        ALLOW       Anywhere
# 80/tcp        ALLOW       Anywhere
# 443/tcp       ALLOW       Anywhere
# 22/tcp (v6)   ALLOW       Anywhere (v6)
# 80/tcp (v6)   ALLOW       Anywhere (v6)
# 443/tcp (v6)  ALLOW       Anywhere (v6)

Step 0.5: Install Docker & Docker Compose

# Add Docker repository
sudo apt-get update
sudo apt-get install -y \
  apt-transport-https \
  ca-certificates \
  curl \
  gnupg \
  lsb-release

# Add Docker GPG key
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \
  sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

# Add Docker repository
echo \
  "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] \
  https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Install Docker Engine 29.x and Docker Compose 2.x
sudo apt-get update
sudo apt-get install -y \
  docker-ce \
  docker-ce-cli \
  containerd.io \
  docker-compose-plugin

# Verify Docker installation
sudo docker --version
# Should show: Docker version 29.x.x

sudo docker compose version
# Should show: Docker Compose version 2.x.x

# Add user to docker group (logout & login to take effect)
sudo usermod -aG docker igny8

# Test Docker without sudo (after logout/login)
docker run hello-world

Step 0.6: Create Directory Structure

# Create /data hierarchy
sudo mkdir -p /data/stack
sudo mkdir -p /data/app
sudo mkdir -p /data/logs
sudo mkdir -p /data/backups/postgres
sudo mkdir -p /data/backups/redis
sudo mkdir -p /data/backups/caddy

# Set ownership and permissions
sudo chown -R igny8:igny8 /data
sudo chmod -R 755 /data

Step 0.7: Clone igny8-stack Repository

# Navigate to /data/stack
cd /data/stack

# Clone repository (adjust URL as needed)
git clone https://github.com/igny8/igny8-stack.git .

# Verify repository contents
ls -la /data/stack
# Should contain: docker-compose.infra.yml, Caddyfile, install.sh, etc.

5.3 Phase 1: Docker Network & Infrastructure Setup

Step 1.1: Create External Docker Network

# Create the igny8_net external bridge network
docker network create igny8_net

# Verify network creation
docker network ls | grep igny8_net
docker network inspect igny8_net

Step 1.2: Configure Environment Variables

# Create .env file at /data/stack/.env
cd /data/stack
cat > .env << 'EOF'
# PostgreSQL
DB_PASSWORD=<GENERATE_STRONG_PASSWORD>
DB_USER=igny8

# PgAdmin
PGADMIN_PASSWORD=<GENERATE_STRONG_PASSWORD>

# Portainer
PORTAINER_PASSWORD=<GENERATE_STRONG_PASSWORD>

# Application Domains (for reference in Caddyfile)
PROD_DOMAIN=igny8.com
PROD_APP_DOMAIN=app.igny8.com
PROD_API_DOMAIN=api.igny8.com

STAGING_DOMAIN=staging.igny8.com
STAGING_APP_DOMAIN=staging.igny8.com
STAGING_API_DOMAIN=staging-api.igny8.com

# Test Domains (temporary, point to new VPS during migration)
TEST_DOMAIN=test-marketing.igny8.com
TEST_APP_DOMAIN=test-app.igny8.com
TEST_API_DOMAIN=test-api.igny8.com
EOF

# Secure .env file permissions
chmod 600 .env

Step 1.3: Configure Caddyfile

# Edit /data/stack/Caddyfile
cat > /data/stack/Caddyfile << 'EOF'
# Global settings
{
  email admin@igny8.com
}

# Production domains (DNS still on old VPS — these won't receive traffic yet)
app.igny8.com {
    reverse_proxy igny8_backend:8010
}

api.igny8.com {
    reverse_proxy igny8_backend:8010
}

igny8.com, www.igny8.com {
    reverse_proxy igny8_marketing:3000
}

# Test domains (DNS points to THIS new VPS — use these for validation)
test-app.igny8.com {
    reverse_proxy igny8_backend:8010
}

test-api.igny8.com {
    reverse_proxy igny8_backend:8010
}

test-marketing.igny8.com {
    reverse_proxy igny8_marketing:3000
}

# Staging domains
staging.igny8.com, www.staging.igny8.com {
    reverse_proxy igny8_staging_marketing:3000
}

staging-app.igny8.com {
    reverse_proxy igny8_staging_backend:8010
}

staging-api.igny8.com {
    reverse_proxy igny8_staging_backend:8010
}

# Management services
portainer.igny8.com {
    reverse_proxy portainer:9000
}

pgadmin.igny8.com {
    reverse_proxy pgadmin:5050
}

filebrowser.igny8.com {
    reverse_proxy filebrowser:8080
}
EOF

# Verify Caddyfile syntax
docker run --rm -v /data/stack/Caddyfile:/etc/caddy/Caddyfile caddy:2-alpine caddy validate

Key Points:

Production domains (app.igny8.com, api.igny8.com, igny8.com) are configured but DNS still points to old VPS
Test subdomains (test-app.igny8.com, test-api.igny8.com, test-marketing.igny8.com) route to SAME backend containers — they're just different hostnames
Caddy uses Let's Encrypt for automatic HTTPS on test domains
No Cloudflare ACME tokens required in this phase

Step 1.4: Run Docker Compose (Infrastructure)

# Navigate to stack directory
cd /data/stack

# Start infrastructure services
docker compose -f docker-compose.infra.yml up -d

# Verify containers are running
docker ps

# Expected output should show:
# - postgres (postgres:18-alpine)
# - redis (redis:8-alpine)
# - caddy (caddy:2-alpine)
# - portainer (portainer/portainer-ce:lts)
# - pgadmin (dpage/pgadmin4:latest)
# - filebrowser (filebrowser/filebrowser:latest)

# Check container logs for errors
docker compose -f docker-compose.infra.yml logs -f

# Common issues:
# - Port 5432 already in use: Check existing postgres instances
# - Network error: Ensure igny8_net was created (Step 1.1)
# - Permission denied on volumes: Check /data ownership (Step 0.6)

Step 1.5: Test Subdomain DNS Configuration

At your CURRENT DNS provider (NOT Cloudflare):

Create A records for test subdomains pointing to NEW VPS IP:
- test-app.igny8.com → <NEW_VPS_IP> (not proxied, DNS only)
- test-api.igny8.com → <NEW_VPS_IP> (not proxied, DNS only)
- test-marketing.igny8.com → <NEW_VPS_IP> (not proxied, DNS only)
Leave production domains unchanged:
- app.igny8.com → OLD VPS IP (still healthy)
- api.igny8.com → OLD VPS IP (still healthy)
- igny8.com → OLD VPS IP (still healthy)
Verify DNS propagation:

nslookup test-app.igny8.com
nslookup test-api.igny8.com
nslookup test-marketing.igny8.com
# All should resolve to NEW_VPS_IP

5.4 Phase 2: Verification & Health Checks

Step 2.1: Container Health Verification

# Check all infrastructure containers
docker ps --format "table {{.Names}}\t{{.Status}}"

# Expected output:
# NAMES                 STATUS
# caddy                 Up X minutes
# portainer             Up X minutes
# pgadmin               Up X minutes
# filebrowser           Up X minutes
# postgres              Up X minutes
# redis                 Up X minutes

# Check logs for errors
docker compose -f docker-compose.infra.yml logs | grep -i "error"
# Should return no results or only informational messages

Step 2.2: Network Connectivity Tests

# Verify igny8_net connectivity
docker network inspect igny8_net

# Test PostgreSQL connectivity from another container
docker run --rm --network igny8_net postgres:18-alpine \
  psql -h postgres -U igny8 -c "SELECT version();"
# Prompt for password (DB_PASSWORD from .env)

# Test Redis connectivity
docker run --rm --network igny8_net redis:8-alpine \
  redis-cli -h redis PING
# Should return "PONG"

Step 2.3: Test Subdomain HTTPS Tests

# Test Let's Encrypt certificate provisioning on test domains
curl -I https://test-app.igny8.com
# Expected: 200 or 502 (container not deployed yet, which is fine)
# Important: Should NOT show certificate errors

curl -I https://test-api.igny8.com
# Expected: 200 or 502

curl -I https://test-marketing.igny8.com
# Expected: 200 or 502

# Verify certificate details
openssl s_client -connect test-app.igny8.com:443 -servername test-app.igny8.com < /dev/null | grep -A 2 "Issuer"
# Should show: Issuer: C = US, O = Let's Encrypt, CN = R3 (or similar)

# Check Caddy logs for certificate provisioning
docker logs caddy | grep -i "test-app\|test-api\|test-marketing"

Step 2.4: Production Domain Verification

# Verify production domains still resolve to OLD VPS (they should not change yet)
nslookup app.igny8.com
nslookup api.igny8.com
nslookup igny8.com
# Should show OLD VPS IP, not NEW VPS IP

# This confirms zero impact on production during provisioning

Step 2.5: System Resource Verification

# Check disk usage
df -h /data

# Check memory usage
free -h

# Check Docker disk usage
docker system df

# Monitor running processes
top -b -n 1 | head -n 20

Step 2.6: Backup & Persistence Verification

# Verify database file exists
ls -lh /data/backups/postgres/

# Verify Redis persistence
ls -lh /data/backups/redis/

# Check Caddy certificate storage
ls -lh /data/backups/caddy/

# All directories should contain actual data files

6. Acceptance Criteria

The VPS provisioning is complete when ALL of the following are satisfied:

6.1 System Requirements

Ubuntu 24.04 LTS confirmed via lsb_release -a
SSH access via key-based authentication only
Root password login disabled
UFW firewall enabled with only ports 22, 80, 443 open
System time synchronized (UTC timezone)
Non-root user (igny8) created with sudo access

6.2 Docker & Containerization

Docker Engine 29.x running (docker --version)
Docker Compose 2.x+ installed (docker compose version)
User igny8 can run docker commands without sudo
External network igny8_net exists and is accessible

6.3 Directory Structure

/data/stack/ contains igny8-stack repository with docker-compose.infra.yml and Caddyfile
/data/app/ exists and is owned by igny8:igny8
/data/logs/ exists and is writable
/data/backups/postgres/ contains PostgreSQL data
/data/backups/redis/ contains Redis persistence files
/data/backups/caddy/ contains Caddy certificates and Let's Encrypt data

6.4 Infrastructure Services

PostgreSQL 18 container running and healthy
- Verification: docker exec postgres psql -U igny8 -c "SELECT 1"
Redis 8.x container running and healthy
- Verification: docker exec redis redis-cli PING returns PONG
Caddy 2.11.x container running and listening on 80/443
- Verification: curl -I https://test-app.igny8.com succeeds (no cert errors)
Portainer container running and accessible
- Verification: curl -I https://portainer.igny8.com succeeds
PgAdmin container running and accessible
- Verification: curl -I https://pgadmin.igny8.com succeeds
FileBrowser container running and accessible
- Verification: curl -I https://filebrowser.igny8.com succeeds

6.5 DNS & Test Subdomains

Test subdomains created at CURRENT DNS provider pointing to NEW VPS IP
Production domains UNCHANGED at DNS provider (still point to OLD VPS)
DNS propagates successfully: nslookup test-app.igny8.com resolves to NEW VPS IP
HTTPS certificates auto-provisioned by Caddy via Let's Encrypt on test subdomains
Zero impact on production: all production domains still accessible on old VPS

6.6 Security & Hardening

SSH daemon configured for key-only authentication
Root account password login disabled
UFW firewall enforced with minimal rule set
.env file has 600 permissions (readable only by owner)
/data/ directory owned by igny8:igny8
No exposed ports beyond 22, 80, 443

6.7 Documentation & Handoff

This document (00B) completed and validated
Next phases (00C production migration, 00D staging setup) have actionable prerequisites
All credentials stored securely (not in git, only in .env with 600 permissions)
Docker Compose files checked into source control

7. Claude Code Instructions

These instructions enable Claude Code to provision the VPS or verify existing provisioning.

7.1 Provisioning Automation

Objective: Automate VPS setup from SSH access

Files to Create/Modify:

/data/stack/.env (environment variables — NO Cloudflare tokens)
/data/stack/docker-compose.infra.yml (infrastructure services)
/data/stack/Caddyfile (reverse proxy config with test subdomains)

Shell Scripts to Execute:

System update & security setup (Steps 0.1-0.4)
Docker installation (Step 0.5)
Directory structure creation (Step 0.6)
Repository clone (Step 0.7)
Network & Docker Compose startup (Steps 1.1-1.4)

Inputs Needed:

VPS IP address
SSH key path
Strong random passwords for services
Current DNS provider credentials (for test subdomain creation)

Expected Outputs:

Fully running infrastructure stack with PostgreSQL 18, Redis 8, Caddy 2.11
All containers healthy
Test subdomains accessible via HTTPS (Let's Encrypt certs)
Production domains unchanged on old VPS
No errors in logs
Passes all acceptance criteria (Section 6)

7.2 Verification Automation

Objective: Verify existing provisioning is healthy

Checks to Perform:

SSH connectivity to VPS
UFW firewall rules (22, 80, 443 only)
Docker services running count
Network igny8_net exists
PostgreSQL 18 responsive
Redis 8 responsive
Caddy 2.11 listening on 80/443
DNS resolution for test subdomains (NEW VPS)
DNS resolution for production domains (OLD VPS — should not change)
HTTPS certificate provisioning on test subdomains
No Cloudflare API tokens in .env (should not exist yet)

Expected Outcomes:

All checks pass
No orphaned containers
No disk space warnings
Logs show no errors
Test subdomains fully functional
Production domains unaffected

7.3 Integration with Downstream Phases

Dependencies for 00C (Production Migration):

Infrastructure provisioning complete (this phase)
docker-compose.app.yml ready in /data/app/
Production application images built and available
Test subdomains validated and working
Plan for DNS cutover (moving production domains to new VPS)

Dependencies for 00D (Staging Setup):

Infrastructure provisioning complete (this phase)
Staging application images available
docker-compose.app.yml configured for staging
Staging domains configured in Caddyfile (already done in this phase)

7.4 Troubleshooting Runbook

If PostgreSQL 18 fails to start:

# Check volume permissions
ls -lh /data/backups/postgres/

# Check logs
docker logs postgres | tail -20

# Verify image version
docker images | grep postgres
# Should show postgres:18-alpine

# Recreate volume if corrupted
docker compose -f docker-compose.infra.yml down
docker compose -f docker-compose.infra.yml up -d postgres

If Caddy fails to provision Let's Encrypt certificates on test subdomains:

# Verify DNS resolution for test subdomains
nslookup test-app.igny8.com
# Should resolve to NEW_VPS_IP

# Check Caddy logs
docker logs caddy | grep -i "test-app\|error\|acme"

# Verify test subdomain A records exist at current DNS provider
# (NOT Cloudflare — your current DNS provider)

# Force certificate renewal
docker compose -f docker-compose.infra.yml restart caddy

If Docker network unreachable:

# Verify network exists
docker network ls | grep igny8_net

# Recreate if missing
docker network create igny8_net

# Restart services
docker compose -f docker-compose.infra.yml restart

If SSH key auth fails:

# Verify SSH config
sudo cat /etc/ssh/sshd_config | grep -i "pubkey\|password"

# Check authorized_keys
cat ~/.ssh/authorized_keys

# Ensure correct permissions
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

If production domains become inaccessible:

# Verify production domains still resolve to OLD VPS IP
nslookup app.igny8.com
nslookup api.igny8.com
nslookup igny8.com

# These should NOT resolve to NEW VPS IP at this stage
# If they do, reverse the DNS change immediately

# Verify old VPS is still running (contact infrastructure team)

8. Cross-References

00A: IGNY8 Phase 0 Overview (prerequisites, timeline, team)
00C: Production Migration (deploying production apps + DNS cutover)
00D: Staging Setup (deploying staging environment)
Appendix: igny8-stack repository documentation

9. Sign-Off

Document Owner: Infrastructure Team Last Updated: 2026-03-23 Approval Status: Pending Review

Role	Name	Signature	Date
Infrastructure Lead	—	—	—
DevOps Engineer	—	—	—
Project Manager	—	—	—

Appendix: Quick Reference Commands

Essential Docker Commands

# View all services
docker ps -a

# View infrastructure logs
docker compose -f docker-compose.infra.yml logs -f

# Restart a service
docker compose -f docker-compose.infra.yml restart <service_name>

# Execute command in container
docker exec <container_name> <command>

# Connect to PostgreSQL
docker exec -it postgres psql -U igny8 -d igny8

# Check network connectivity
docker network inspect igny8_net

Essential System Commands

# Check disk usage
df -h /data

# Check memory
free -h

# Monitor processes
htop

# View firewall rules
sudo ufw status verbose

# Check SSH connectivity
ssh -v igny8@<VPS_IP>

# DNS propagation check
dig test-app.igny8.com +short
dig app.igny8.com +short

Backup & Recovery

# Backup PostgreSQL data
docker exec postgres pg_dump -U igny8 igny8 > backup.sql

# Backup Redis data
docker exec redis redis-cli --rdb /data/backups/redis/dump-$(date +%s).rdb

# List all backups
ls -lh /data/backups/

DNS Verification (Critical During Migration)

# Verify test subdomains point to NEW VPS (should work)
nslookup test-app.igny8.com
nslookup test-api.igny8.com
nslookup test-marketing.igny8.com

# Verify production domains point to OLD VPS (should NOT change yet)
nslookup app.igny8.com
nslookup api.igny8.com
nslookup igny8.com

# Test HTTPS on test subdomains
curl -I https://test-app.igny8.com

# Do NOT test production domains on new VPS yet (they're still on old VPS)

28 KiB Raw Blame History