Files

alorig 6b738d1ab7 remove venv

2025-11-09 18:08:23 +05:00

32 KiB

Raw Blame History

IGNY8 Infrastructure & Deployment Documentation

Version: 1.0
Last Updated: 2025-01-XX
Purpose: Complete infrastructure and deployment documentation covering Docker setup, containers, networks, volumes, ports, environment configuration, and deployment procedures.

Infrastructure Overview
Docker Architecture
Infrastructure Stack (igny8-infra)
Application Stack (igny8-app)
Network Configuration
Volume Management
Port Allocation
Environment Variables
Docker Images
Deployment Procedures
Backup & Restore
Monitoring & Health Checks
Troubleshooting

Infrastructure Overview

The IGNY8 platform runs on a Docker-based infrastructure with two main stacks:

Infrastructure Stack (igny8-infra): Shared services used by all applications
Application Stack (igny8-app): IGNY8-specific application services

All containers run on a single Docker network (igny8_net) for inter-container communication.

Infrastructure Components

Infrastructure Stack Services:

PostgreSQL 15 (Database)
Redis 7 (Cache & Celery Broker)
pgAdmin 4 (Database Administration)
FileBrowser (File Management)
Caddy (Reverse Proxy & HTTPS)
setup-helper (Utility Container)

Application Stack Services:

igny8_backend (Django Backend with Gunicorn)
igny8_frontend (React Frontend with Vite)
igny8_celery_worker (Celery Worker)
igny8_celery_beat (Celery Beat Scheduler)

External Services:

Portainer (Container Management - runs separately)

Docker Architecture

Stack Organization

Two-Stack Architecture:

Infrastructure Stack: Shared services (database, cache, reverse proxy)
Application Stack: Application-specific services (backend, frontend, workers)

Benefits:

Separation of concerns
Independent scaling
Shared infrastructure across multiple apps
Easier maintenance and updates

File Structure

/data/app/
├── docker-compose.yml          # Infrastructure stack (igny8-infra)
└── igny8/
    └── docker-compose.app.yml  # Application stack (igny8-app)

Network Architecture

Single Network: igny8_net (bridge network)

Network Type: External bridge network (created manually)

All Containers: Connected to igny8_net for inter-container communication

Infrastructure Stack (igny8-infra)

File: /data/app/docker-compose.yml

Stack Name: igny8-infra

Purpose: Shared infrastructure services used by all applications

Services

1. PostgreSQL Database

Container Name: igny8_postgres

Image: postgres:15

Purpose: Primary database for all applications

Configuration:

User: igny8
Password: igny8pass
Database: igny8_db
Port: Internal only (no external port mapping)
Volume: pgdata:/var/lib/postgresql/data

Health Check:

Command: pg_isready -U $POSTGRES_USER -d $POSTGRES_DB
Interval: 20s
Timeout: 5s
Retries: 5
Start Period: 15s

Network: igny8_net

Labels:

com.docker.compose.project=igny8-infra
com.docker.compose.service=postgres

2. Redis Cache & Broker

Container Name: igny8_redis

Image: redis:7

Purpose:

Celery task broker
Celery result backend
Application caching

Configuration:

Port: Internal only (6379, no external mapping)
Volume: redisdata:/data
Command: redis-server --save 60 1 --loglevel warning

Health Check:

Command: redis-cli ping | grep -q PONG
Interval: 20s
Timeout: 3s
Retries: 5

Network: igny8_net

Labels:

com.docker.compose.project=igny8-infra
com.docker.compose.service=redis

3. pgAdmin

Container Name: igny8_pgadmin

Image: dpage/pgadmin4

Purpose: PostgreSQL database administration interface

Configuration:

Port: 5050:80 (external:internal)
Email: admin@igny8.com
Password: admin123
Volume: pgadmin_data:/var/lib/pgadmin

Access: http://localhost:5050 or http://<server-ip>:5050

Network: igny8_net

Labels:

com.docker.compose.project=igny8-infra
com.docker.compose.service=pgadmin

4. FileBrowser

Container Name: igny8_filebrowser

Image: filebrowser/filebrowser:v2.25.0

Purpose: Web-based file management interface

Configuration:

Port: 8080:80 (external:internal)
Timezone: Asia/Karachi
Volumes:
- /data:/srv (read-write)
- /backups:/srv/backups (read-write)
- filebrowser_db:/database (database)

Access: http://localhost:8080 or http://<server-ip>:8080

Network: igny8_net

Labels:

com.docker.compose.project=igny8-infra
com.docker.compose.service=filebrowser

5. Caddy Reverse Proxy

Container Name: igny8_caddy

Image: caddy:latest

Purpose:

Reverse proxy for all applications
HTTPS/SSL termination
Request routing

Configuration:

Ports:
- 80:80 (HTTP)
- 443:443 (HTTPS)
Volumes:
- caddy_data:/data (SSL certificates)
- caddy_config:/config (configuration)
- /var/lib/docker/volumes/portainer_data/_data/caddy/Caddyfile:/etc/caddy/Caddyfile (routing config)

Caddyfile Location: /var/lib/docker/volumes/portainer_data/_data/caddy/Caddyfile

Network: igny8_net

Labels:

com.docker.compose.project=igny8-infra
com.docker.compose.service=caddy

6. Setup Helper

Container Name: setup-helper

Image: alpine:3.20

Purpose: Utility container for setup and maintenance tasks

Configuration:

Command: sh -c "sleep infinity" (keeps container running)
Volumes:
- /data/backups:/backups:rw (backup directory)
- /scripts:/scripts:ro (utility scripts)

Network: igny8_net

Labels:

com.docker.compose.project=igny8-infra
com.docker.compose.service=setup-helper

Volumes

Infrastructure Stack Volumes:

pgdata: PostgreSQL database data
redisdata: Redis data
pgadmin_data: pgAdmin configuration
filebrowser_db: FileBrowser database
caddy_data: Caddy SSL certificates and data
caddy_config: Caddy configuration

Network

Network Name: igny8_net

Type: External bridge network

Creation: Must be created manually before starting stacks

Command: docker network create igny8_net --driver bridge

Application Stack (igny8-app)

File: /data/app/igny8/docker-compose.app.yml

Stack Name: igny8-app

Purpose: IGNY8 application-specific services

Services

1. Backend Service

Container Name: igny8_backend

Image: igny8-backend:latest

Purpose: Django REST API backend

Configuration:

Port: 8011:8010 (external:internal)
Working Directory: /app
Command: gunicorn igny8_core.wsgi:application --bind 0.0.0.0:8010 --workers 4 --timeout 120

Environment Variables:

DB_HOST: postgres
DB_NAME: igny8_db
DB_USER: igny8
DB_PASSWORD: igny8pass
REDIS_HOST: redis
REDIS_PORT: 6379
USE_SECURE_COOKIES: True
USE_SECURE_PROXY_HEADER: True
DEBUG: False
SECRET_KEY: (should be set via environment variable in production)

Volumes:

/data/app/igny8/backend:/app:rw (application code)
/data/app/igny8:/data/app/igny8:ro (read-only access to project root)
/var/run/docker.sock:/var/run/docker.sock:ro (Docker socket for container management)
/data/app/logs:/app/logs:rw (application logs)

Health Check:

Command: python -c "import urllib.request; urllib.request.urlopen('http://localhost:8010/api/v1/system/status/').read()"
Interval: 30s
Timeout: 10s
Retries: 3
Start Period: 40s

Network: igny8_net

Dependencies:

Requires postgres and redis from infrastructure stack (external services)

Labels:

com.docker.compose.project=igny8-app
com.docker.compose.service=igny8_backend

2. Frontend Service

Container Name: igny8_frontend

Image: igny8-frontend-dev:latest

Purpose: React frontend with Vite dev server

Configuration:

Port: 8021:5173 (external:internal)
Command: npm run dev -- --host 0.0.0.0 --port 5173

Environment Variables:

VITE_BACKEND_URL: https://api.igny8.com/api

Volumes:

/data/app/igny8/frontend:/app:rw (application code)

Dependencies:

igny8_backend (waits for backend to be healthy)

Network: igny8_net

Labels:

com.docker.compose.project=igny8-app
com.docker.compose.service=igny8_frontend

3. Celery Worker

Container Name: igny8_celery_worker

Image: igny8-backend:latest (same as backend)

Purpose: Celery worker for asynchronous task processing

Configuration:

Working Directory: /app
Command: celery -A igny8_core worker --loglevel=info --concurrency=4

Environment Variables:

DB_HOST: postgres
DB_NAME: igny8_db
DB_USER: igny8
DB_PASSWORD: igny8pass
REDIS_HOST: redis
REDIS_PORT: 6379
DEBUG: False

Volumes:

/data/app/igny8/backend:/app:rw (application code)
/data/app/igny8:/data/app/igny8:ro (read-only access to project root)
/data/app/logs:/app/logs:rw (application logs)

Network: igny8_net

Dependencies:

Requires postgres and redis from infrastructure stack (external services)

Labels:

com.docker.compose.project=igny8-app
com.docker.compose.service=igny8_celery_worker

4. Celery Beat

Container Name: igny8_celery_beat

Image: igny8-backend:latest (same as backend)

Purpose: Celery beat scheduler for periodic tasks

Configuration:

Working Directory: /app
Command: celery -A igny8_core beat --loglevel=info

Environment Variables:

DB_HOST: postgres
DB_NAME: igny8_db
DB_USER: igny8
DB_PASSWORD: igny8pass
REDIS_HOST: redis
REDIS_PORT: 6379
DEBUG: False

Volumes:

/data/app/igny8/backend:/app:rw (application code)
/data/app/igny8:/data/app/igny8:ro (read-only access to project root)
/data/app/logs:/app/logs:rw (application logs)

Network: igny8_net

Dependencies:

Requires postgres and redis from infrastructure stack (external services)

Labels:

com.docker.compose.project=igny8-app
com.docker.compose.service=igny8_celery_beat

Network

Network Name: igny8_net

Type: External (uses network from infrastructure stack)

Configuration: external: true

Network Configuration

Network Details

Network Name: igny8_net

Type: Bridge network

Driver: bridge

Scope: External (shared across stacks)

Network Creation

Manual Creation Required:

docker network create igny8_net --driver bridge

Verification:

docker network inspect igny8_net

Container IP Addresses

Infrastructure Stack (172.18.0.x subnet):

igny8_postgres: 172.18.0.6
igny8_redis: 172.18.0.2
igny8_pgadmin: 172.18.0.4
igny8_filebrowser: 172.18.0.8
igny8_caddy: 172.18.0.7
setup-helper: 172.18.0.5

Application Stack (172.18.0.x subnet):

igny8_backend: 172.18.0.3
igny8_frontend: 172.18.0.11
igny8_celery_worker: 172.18.0.9
igny8_celery_beat: 172.18.0.10

Note: IP addresses are dynamically assigned by Docker and may vary.

Inter-Container Communication

Service Discovery: Containers can communicate using service names as hostnames

Examples:

Backend → PostgreSQL: postgres:5432
Backend → Redis: redis:6379
Frontend → Backend: igny8_backend:8010 (internal) or https://api.igny8.com/api (external via Caddy)

Volume Management

Infrastructure Stack Volumes

pgdata

Purpose: PostgreSQL database data

Location: /var/lib/docker/volumes/igny8-infra_pgdata/_data

Backup: Critical - contains all database data

redisdata

Purpose: Redis data persistence

Location: /var/lib/docker/volumes/igny8-infra_redisdata/_data

Backup: Optional - cache data can be regenerated

pgadmin_data

Purpose: pgAdmin configuration and settings

Location: /var/lib/docker/volumes/igny8-infra_pgadmin_data/_data

Backup: Optional - configuration can be recreated

filebrowser_db

Purpose: FileBrowser database

Location: /var/lib/docker/volumes/igny8-infra_filebrowser_db/_data

Backup: Optional - file browser state

caddy_data

Purpose: Caddy SSL certificates and data

Location: /var/lib/docker/volumes/igny8-infra_caddy_data/_data

Backup: Important - contains SSL certificates

caddy_config

Purpose: Caddy configuration

Location: /var/lib/docker/volumes/igny8-infra_caddy_config/_data

Backup: Important - contains Caddy configuration

Host Volume Mounts

Application Code

Path: /data/app/igny8/backend:/app:rw

Purpose: Backend application code (read-write for development)

Application Root

Path: /data/app/igny8:/data/app/igny8:ro

Purpose: Read-only access to project root

Logs

Path: /data/app/logs:/app/logs:rw

Purpose: Application logs directory

Docker Socket

Path: /var/run/docker.sock:/var/run/docker.sock:ro

Purpose: Read-only access to Docker socket for container management

FileBrowser Data

Path: /data:/srv (read-write)

Purpose: FileBrowser access to host filesystem

FileBrowser Backups

Path: /backups:/srv/backups (read-write)

Purpose: FileBrowser access to backup directory

Setup Helper Scripts

Path: /scripts:/scripts:ro

Purpose: Read-only access to utility scripts

Setup Helper Backups

Path: /data/backups:/backups:rw

Purpose: Backup directory access

Volume Backup

Docker Volumes:

# Backup all Docker volumes
tar -czf docker-volumes-backup-$(date +%Y%m%d).tar.gz \
  -C /var/lib/docker volumes/

Host Mounts:

# Backup application code
tar -czf app-backup-$(date +%Y%m%d).tar.gz -C /data app/

# Backup logs
tar -czf logs-backup-$(date +%Y%m%d).tar.gz -C /data logs/

Port Allocation

External Ports (Host → Container)

Infrastructure Stack:

5050 → pgAdmin (80)
8080 → FileBrowser (80)
80 → Caddy (80) - HTTP
443 → Caddy (443) - HTTPS

Application Stack:

8011 → Backend (8010)
8021 → Frontend (5173)

Portainer (External):

8000 → Portainer (8000) - HTTP
9443 → Portainer (9443) - HTTPS

Internal Ports (Container → Container)

Standardized Internal Ports:

Backend: 8010 (Gunicorn)
Frontend Dev: 5173 (Vite dev server)
Frontend Prod: 8020 (Caddy)
PostgreSQL: 5432
Redis: 6379
pgAdmin: 80
FileBrowser: 80
Caddy: 80 (HTTP), 443 (HTTPS)

Port Allocation Strategy

Pattern:

Backend External: XX11 (e.g., 8011)
Frontend External: XX21 (e.g., 8021)

IGNY8 Port Range: 8000-8099

Backend: 8011
Frontend: 8021

Environment Variables

Backend Environment Variables

Database Configuration:

DB_HOST: PostgreSQL hostname (default: postgres)
DB_NAME: Database name (default: igny8_db)
DB_USER: Database user (default: igny8)
DB_PASSWORD: Database password (default: igny8pass)
DB_PORT: Database port (default: 5432)

Redis Configuration:

REDIS_HOST: Redis hostname (default: redis)
REDIS_PORT: Redis port (default: 6379)

Django Configuration:

SECRET_KEY: Django secret key (REQUIRED in production)
DEBUG: Debug mode (default: False)
ALLOWED_HOSTS: Comma-separated list of allowed hosts
DJANGO_SETTINGS_MODULE: Django settings module (default: igny8_core.settings)

Security Settings:

USE_SECURE_COOKIES: Enable secure cookies (default: True in production)
USE_SECURE_PROXY_HEADER: Enable secure proxy header (default: True in production)

JWT Configuration:

JWT_SECRET_KEY: JWT secret key (defaults to SECRET_KEY)

Celery Configuration:

CELERY_BROKER_URL: Celery broker URL (default: redis://redis:6379/0)
CELERY_RESULT_BACKEND: Celery result backend (default: redis://redis:6379/0)

Frontend Environment Variables

API Configuration:

VITE_BACKEND_URL: Backend API URL (default: https://api.igny8.com/api)

Build Configuration:

VITE_APP_NAME: Application name

Infrastructure Environment Variables

PostgreSQL:

POSTGRES_USER: Database user (default: igny8)
POSTGRES_PASSWORD: Database password (default: igny8pass)
POSTGRES_DB: Database name (default: igny8_db)

pgAdmin:

PGADMIN_DEFAULT_EMAIL: Admin email (default: admin@igny8.com)
PGADMIN_DEFAULT_PASSWORD: Admin password (default: admin123)

FileBrowser:

TZ: Timezone (default: Asia/Karachi)

Docker Images

Image Building

Backend Image:

cd /data/app/igny8/backend
docker build -t igny8-backend:latest -f Dockerfile .

Frontend Dev Image:

cd /data/app/igny8/frontend
docker build -t igny8-frontend-dev:latest -f Dockerfile.dev .

Frontend Production Image:

cd /data/app/igny8/frontend
docker build -t igny8-frontend:latest -f Dockerfile .

Image Details

igny8-backend:latest

Base Image: python:3.11-slim

Dependencies:

System: gcc, libpq-dev, curl, git
Python: See requirements.txt

Exposed Port: 8010

Default Command: gunicorn igny8_core.wsgi:application --bind 0.0.0.0:8010

Build Context: /data/app/igny8/backend

igny8-frontend-dev:latest

Base Image: node:18-alpine

Dependencies: See package.json

Exposed Port: 5173

Default Command: npm run dev -- --host 0.0.0.0 --port 5173

Build Context: /data/app/igny8/frontend

igny8-frontend:latest

Base Image: Multi-stage build

Stage 1: node:18-alpine (builder)
Stage 2: caddy:latest (server)

Exposed Port: 8020

Default Command: caddy run --config /etc/caddy/Caddyfile

Build Context: /data/app/igny8/frontend

External Images

Infrastructure Images:

postgres:15 - PostgreSQL database
redis:7 - Redis cache and broker
dpage/pgadmin4 - pgAdmin interface
filebrowser/filebrowser:v2.25.0 - FileBrowser
caddy:latest - Caddy reverse proxy
alpine:3.20 - Setup helper utility

Management Images:

portainer/portainer-ce:latest - Portainer container management

Deployment Procedures

Initial Setup

1. Prerequisites

System Requirements:

Docker 20.10+
Docker Compose 2.0+
Sufficient disk space (minimum 20GB)
Network access for pulling images

Directory Structure:

/data/app/
├── docker-compose.yml          # Infrastructure stack
└── igny8/
    ├── docker-compose.app.yml  # Application stack
    ├── backend/
    └── frontend/

2. Create Docker Network

docker network create igny8_net --driver bridge

3. Build Docker Images

Backend Image:

cd /data/app/igny8/backend
docker build -t igny8-backend:latest -f Dockerfile .

Frontend Dev Image:

cd /data/app/igny8/frontend
docker build -t igny8-frontend-dev:latest -f Dockerfile.dev .

4. Start Infrastructure Stack

cd /data/app
docker compose -f docker-compose.yml -p igny8-infra up -d

Verify:

docker ps --filter "label=com.docker.compose.project=igny8-infra"

5. Start Application Stack

cd /data/app/igny8
docker compose -f docker-compose.app.yml -p igny8-app up -d

Verify:

docker ps --filter "label=com.docker.compose.project=igny8-app"

Database Initialization

1. Run Migrations

docker exec -it igny8_backend python manage.py migrate

2. Create Superuser

docker exec -it igny8_backend python manage.py createsuperuser

3. Load Initial Data (Optional)

docker exec -it igny8_backend python manage.py loaddata initial_data.json

Caddy Configuration

Caddyfile Location: /var/lib/docker/volumes/portainer_data/_data/caddy/Caddyfile

Configuration: Managed via Portainer or manually

Example Configuration:

api.igny8.com {
    reverse_proxy igny8_backend:8010
}

app.igny8.com {
    reverse_proxy igny8_frontend:5173
}

Reload Caddy:

docker exec igny8_caddy caddy reload --config /etc/caddy/Caddyfile

Production Deployment

1. Set Environment Variables

Create .env file or set via docker-compose:

SECRET_KEY=<generate-secure-key>
JWT_SECRET_KEY=<generate-secure-key>
DEBUG=False
USE_SECURE_COOKIES=True
USE_SECURE_PROXY_HEADER=True

2. Update Caddyfile

Configure domain names and SSL certificates in Caddyfile

3. Build Production Images

Backend (already built):

cd /data/app/igny8/backend
docker build -t igny8-backend:latest -f Dockerfile .

Frontend Production:

cd /data/app/igny8/frontend
docker build -t igny8-frontend:latest -f Dockerfile .

4. Update docker-compose.app.yml

Change frontend image from igny8-frontend-dev:latest to igny8-frontend:latest

5. Restart Stacks

# Restart application stack
cd /data/app/igny8
docker compose -f docker-compose.app.yml -p igny8-app restart

Updates and Maintenance

Update Application Code

Backend:

cd /data/app/igny8/backend
git pull  # or copy new files
docker compose -f docker-compose.app.yml -p igny8-app restart igny8_backend

Frontend:

cd /data/app/igny8/frontend
git pull  # or copy new files
docker compose -f docker-compose.app.yml -p igny8-app restart igny8_frontend

Rebuild Images

Backend:

cd /data/app/igny8/backend
docker build -t igny8-backend:latest -f Dockerfile .
docker compose -f docker-compose.app.yml -p igny8-app up -d --force-recreate igny8_backend igny8_celery_worker igny8_celery_beat

Frontend:

cd /data/app/igny8/frontend
docker build -t igny8-frontend-dev:latest -f Dockerfile.dev .
docker compose -f docker-compose.app.yml -p igny8-app up -d --force-recreate igny8_frontend

Database Migrations

docker exec -it igny8_backend python manage.py migrate

Collect Static Files

docker exec -it igny8_backend python manage.py collectstatic --noinput

Backup & Restore

Backup Procedures

1. Database Backup

PostgreSQL Dump:

docker exec igny8_postgres pg_dump -U igny8 igny8_db > backup-$(date +%Y%m%d).sql

With Compression:

docker exec igny8_postgres pg_dump -U igny8 igny8_db | gzip > backup-$(date +%Y%m%d).sql.gz

2. Docker Volumes Backup

All Volumes:

tar -czf docker-volumes-backup-$(date +%Y%m%d).tar.gz \
  -C /var/lib/docker volumes/

Individual Volumes:

# PostgreSQL data
tar -czf pgdata-backup-$(date +%Y%m%d).tar.gz \
  -C /var/lib/docker/volumes/igny8-infra_pgdata/_data .

# Redis data
tar -czf redisdata-backup-$(date +%Y%m%d).tar.gz \
  -C /var/lib/docker/volumes/igny8-infra_redisdata/_data .

# Caddy data (SSL certificates)
tar -czf caddy-data-backup-$(date +%Y%m%d).tar.gz \
  -C /var/lib/docker/volumes/igny8-infra_caddy_data/_data .

3. Application Code Backup

tar -czf app-backup-$(date +%Y%m%d).tar.gz -C /data app/

4. Logs Backup

tar -czf logs-backup-$(date +%Y%m%d).tar.gz -C /data logs/

5. Complete System Backup

#!/bin/bash
BACKUP_DIR="/backups/complete-backup-$(date +%Y%m%d)"
mkdir -p "$BACKUP_DIR"

# Database
docker exec igny8_postgres pg_dump -U igny8 igny8_db | gzip > "$BACKUP_DIR/database.sql.gz"

# Docker volumes
tar -czf "$BACKUP_DIR/docker-volumes.tar.gz" -C /var/lib/docker volumes/

# Application code
tar -czf "$BACKUP_DIR/app.tar.gz" -C /data app/

# Logs
tar -czf "$BACKUP_DIR/logs.tar.gz" -C /data logs/

# Caddyfile
cp /var/lib/docker/volumes/portainer_data/_data/caddy/Caddyfile "$BACKUP_DIR/Caddyfile"

Restore Procedures

1. Database Restore

From SQL Dump:

docker exec -i igny8_postgres psql -U igny8 igny8_db < backup-YYYYMMDD.sql

From Compressed Dump:

gunzip -c backup-YYYYMMDD.sql.gz | docker exec -i igny8_postgres psql -U igny8 igny8_db

2. Docker Volumes Restore

Stop Services:

docker compose -f docker-compose.app.yml -p igny8-app down
docker compose -f docker-compose.yml -p igny8-infra down

Restore Volumes:

# Extract volume backups
tar -xzf docker-volumes-backup-YYYYMMDD.tar.gz -C /var/lib/docker/

# Or restore individual volumes
tar -xzf pgdata-backup-YYYYMMDD.tar.gz -C /var/lib/docker/volumes/igny8-infra_pgdata/_data/

Start Services:

docker compose -f docker-compose.yml -p igny8-infra up -d
docker compose -f docker-compose.app.yml -p igny8-app up -d

3. Application Code Restore

tar -xzf app-backup-YYYYMMDD.tar.gz -C /data/

4. Complete System Restore

#!/bin/bash
BACKUP_DIR="/backups/complete-backup-YYYYMMDD"

# Restore database
gunzip -c "$BACKUP_DIR/database.sql.gz" | docker exec -i igny8_postgres psql -U igny8 igny8_db

# Restore volumes
tar -xzf "$BACKUP_DIR/docker-volumes.tar.gz" -C /var/lib/docker/

# Restore application code
tar -xzf "$BACKUP_DIR/app.tar.gz" -C /data/

# Restore Caddyfile
cp "$BACKUP_DIR/Caddyfile" /var/lib/docker/volumes/portainer_data/_data/caddy/Caddyfile

# Restart services
docker compose -f docker-compose.yml -p igny8-infra restart caddy
docker compose -f docker-compose.app.yml -p igny8-app restart

Monitoring & Health Checks

Health Check Endpoints

Backend Health Check

Endpoint: http://localhost:8011/api/v1/system/status/

Method: GET

Expected Response: HTTP 200

Implementation: Django view that checks database connectivity

Frontend Health Check

Endpoint: http://localhost:8021/

Method: GET

Expected Response: HTTP 200 (HTML page)

Container Health Checks

PostgreSQL

Check: pg_isready -U igny8 -d igny8_db

Interval: 20s

Timeout: 5s

Retries: 5

Redis

Check: redis-cli ping | grep -q PONG

Interval: 20s

Timeout: 3s

Retries: 5

Backend

Check: python -c "import urllib.request; urllib.request.urlopen('http://localhost:8010/api/v1/system/status/').read()"

Interval: 30s

Timeout: 10s

Retries: 3

Start Period: 40s

Monitoring Script

File: /data/app/igny8/check-status.sh

Purpose: Quick status check for all stacks and containers

Usage:

cd /data/app/igny8
./check-status.sh

Output:

Container status for both stacks
Network connectivity
Service health checks (Backend API, Frontend, PostgreSQL, Redis)
All IGNY8 containers list

Log Monitoring

View Container Logs

Backend:

docker logs -f igny8_backend

Frontend:

docker logs -f igny8_frontend

Celery Worker:

docker logs -f igny8_celery_worker

Celery Beat:

docker logs -f igny8_celery_beat

PostgreSQL:

docker logs -f igny8_postgres

Redis:

docker logs -f igny8_redis

Application Logs

Location: /data/app/logs/

Backend Logs: /data/app/logs/backend/

Frontend Logs: /data/app/logs/frontend/

Troubleshooting

Common Issues

1. Network Not Found

Error: network igny8_net not found

Solution:

docker network create igny8_net --driver bridge

2. Database Connection Failed

Error: could not connect to server: Connection refused

Check:

# Verify PostgreSQL is running
docker ps | grep igny8_postgres

# Check PostgreSQL logs
docker logs igny8_postgres

# Test connection
docker exec igny8_postgres pg_isready -U igny8

Solution: Ensure infrastructure stack is running before application stack

3. Redis Connection Failed

Error: Error connecting to Redis

Check:

# Verify Redis is running
docker ps | grep igny8_redis

# Check Redis logs
docker logs igny8_redis

# Test connection
docker exec igny8_redis redis-cli ping

Solution: Ensure infrastructure stack is running before application stack

4. Port Already in Use

Error: Bind for 0.0.0.0:8011 failed: port is already allocated

Check:

# Find process using port
sudo lsof -i :8011
# or
sudo netstat -tulpn | grep 8011

Solution: Stop conflicting service or change port in docker-compose

5. Container Health Check Failing

Error: Container shows as unhealthy

Check:

# View container health status
docker inspect igny8_backend | grep -A 10 Health

# View container logs
docker logs igny8_backend

# Test health check manually
docker exec igny8_backend python -c "import urllib.request; urllib.request.urlopen('http://localhost:8010/api/v1/system/status/').read()"

Solution: Check application logs, verify dependencies are running

6. Celery Tasks Not Processing

Check:

# Verify Celery worker is running
docker ps | grep igny8_celery_worker

# Check Celery worker logs
docker logs igny8_celery_worker

# Verify Redis connection
docker exec igny8_celery_worker celery -A igny8_core inspect active

Solution: Restart Celery worker, verify Redis connectivity

7. Static Files Not Loading

Error: 404 for static files

Solution:

# Collect static files
docker exec igny8_backend python manage.py collectstatic --noinput

# Restart backend
docker compose -f docker-compose.app.yml -p igny8-app restart igny8_backend

8. CORS Errors

Error: CORS policy blocked request

Check: Verify CORS_ALLOWED_ORIGINS in Django settings

Solution: Add frontend domain to CORS_ALLOWED_ORIGINS in settings.py

Debugging Commands

Container Inspection

# Inspect container configuration
docker inspect igny8_backend

# View container environment variables
docker exec igny8_backend env

# View container network configuration
docker network inspect igny8_net

Database Debugging

# Connect to PostgreSQL
docker exec -it igny8_postgres psql -U igny8 igny8_db

# List databases
docker exec igny8_postgres psql -U igny8 -l

# Check database size
docker exec igny8_postgres psql -U igny8 -c "SELECT pg_size_pretty(pg_database_size('igny8_db'));"

Redis Debugging

# Connect to Redis CLI
docker exec -it igny8_redis redis-cli

# Check Redis info
docker exec igny8_redis redis-cli INFO

# List all keys
docker exec igny8_redis redis-cli KEYS '*'

Application Debugging

# Django shell
docker exec -it igny8_backend python manage.py shell

# Check Django settings
docker exec igny8_backend python manage.py diffsettings

# View Django URLs
docker exec igny8_backend python manage.py show_urls

Summary

The IGNY8 infrastructure is built on:

Two-Stack Architecture: Infrastructure stack (shared services) and Application stack (app-specific services)
Single Network: All containers on igny8_net bridge network
Docker Compose: Separate compose files for infrastructure and application
Volume Management: Docker volumes for persistent data, host mounts for code
Health Checks: Built-in health checks for critical services
Port Allocation: Standardized port allocation (8011 backend, 8021 frontend)
Environment Variables: Configuration via environment variables
Backup & Restore: Comprehensive backup procedures for database, volumes, and code
Monitoring: Health checks, logging, and status scripts
Troubleshooting: Common issues and debugging commands

This infrastructure ensures reliability, scalability, and maintainability while providing a solid foundation for the IGNY8 application.

32 KiB Raw Blame History

IGNY8 Infrastructure & Deployment Documentation

Table of Contents

Infrastructure Overview

Infrastructure Components

Docker Architecture

Stack Organization

File Structure

Network Architecture

Infrastructure Stack (igny8-infra)

Services

1. PostgreSQL Database

2. Redis Cache & Broker

3. pgAdmin

4. FileBrowser

5. Caddy Reverse Proxy

6. Setup Helper

Volumes

Network

Application Stack (igny8-app)

Services

1. Backend Service

2. Frontend Service

3. Celery Worker

4. Celery Beat

Network

Network Configuration

Network Details

Network Creation

Container IP Addresses

Inter-Container Communication

Volume Management

Infrastructure Stack Volumes

pgdata

redisdata

pgadmin_data

filebrowser_db

caddy_data

caddy_config

Host Volume Mounts

Application Code

Application Root

Logs

Docker Socket

FileBrowser Data

FileBrowser Backups

Setup Helper Scripts

Setup Helper Backups

Volume Backup

Port Allocation

External Ports (Host → Container)

Internal Ports (Container → Container)

Port Allocation Strategy

Environment Variables

Backend Environment Variables

Frontend Environment Variables

Infrastructure Environment Variables

Docker Images

Image Building

Image Details

igny8-backend:latest

igny8-frontend-dev:latest

igny8-frontend:latest

External Images

Deployment Procedures

Initial Setup

1. Prerequisites

2. Create Docker Network

3. Build Docker Images

4. Start Infrastructure Stack

5. Start Application Stack

Database Initialization

1. Run Migrations

2. Create Superuser

3. Load Initial Data (Optional)

Caddy Configuration

Production Deployment

1. Set Environment Variables

2. Update Caddyfile

3. Build Production Images

32 KiB

Raw Blame History