salman/igny8
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v2-exece-docs

Browse Source
This commit is contained in:
IGNY8 VPS (Salman)
2026-03-23 10:30:51 +00:00
parent b94d41b7f6
commit e78a41f11c
15 changed files with 2218 additions and 707 deletions
Download Patch File Download Diff File Expand all files Collapse all files

124
v2/V2-Execution-Docs/00-MASTER-EXECUTION-PLAN.md
View File

@@ -1,9 +1,10 @@
# IGNY8 V2 — Master Execution Plan # IGNY8 V2 — Master Execution Plan
**Version:** 1.0 | March 23, 2026 **Version:** 1.1 | March 23, 2026
**Status:** Active — Execution Reference **Status:** Active — Execution Reference
**Author:** Salman (Alorig Systems) + Claude Opus **Author:** Salman (Alorig Systems) + Claude Opus
**Execution Tool:** Claude Code (SSH to VPS) **Execution Tool:** Claude Code (SSH to VPS)
**Source of Truth:** Codebase at `/data/app/igny8/` — all technical claims verified against actual code
--- ---
@@ -11,10 +12,12 @@
This is the single master document governing the complete IGNY8 V2 build — from infrastructure migration through SAG engine, all modules, WordPress ecosystem, business layer, and multi-app deployment. Every sub-phase references a dedicated build doc in this folder that Claude Code can pick up and execute independently. This is the single master document governing the complete IGNY8 V2 build — from infrastructure migration through SAG engine, all modules, WordPress ecosystem, business layer, and multi-app deployment. Every sub-phase references a dedicated build doc in this folder that Claude Code can pick up and execute independently.
## 2. Current State (Confirmed March 23, 2026) ## 2. Current State (Verified Against Codebase — March 23, 2026)
**IGNY8 v1.8.4** is healthy and functionally production-ready. **IGNY8 v1.8.4** is healthy and functionally production-ready.
### 2.1 Functional Status
| Area | Status | | Area | Status |
|------|--------| |------|--------|
| Settings save (content, publishing, profile) | ✅ Working | | Settings save (content, publishing, profile) | ✅ Working |
@@ -27,26 +30,107 @@ This is the single master document governing the complete IGNY8 V2 build — fro
| `/writer/tasks/{id}/brief/` | No current use case — v2 scope | | `/writer/tasks/{id}/brief/` | No current use case — v2 scope |
| Taxonomy sync, Linker/Optimizer, webhooks | Correctly scoped as v2 features, not bugs | | Taxonomy sync, Linker/Optimizer, webhooks | Correctly scoped as v2 features, not bugs |
### 2.2 Verified Codebase Baseline
| Component | Verified Value |
|-----------|---------------|
| Django | >=5.2.7 (requirements.txt) |
| Python | 3.11-slim (Dockerfile) |
| Node | 18-alpine (Dockerfile.dev) |
| React | ^19.0.0 |
| TypeScript | ~5.7.2 |
| Vite | ^6.1.0 |
| Zustand | ^5.0.8 |
| Tailwind CSS | ^4.0.8 |
| Celery | >=5.3.0 |
| WP Plugin | IGNY8 WordPress Bridge v1.5.2 |
| Primary Key Strategy | BigAutoField (integer, NOT UUID) |
| AUTH_USER_MODEL | igny8_core_auth.User |
| DEFAULT_AUTO_FIELD | django.db.models.BigAutoField |
| Installed Apps | 34 Django apps |
| Middleware Stack | 13 middleware classes |
| Celery Beat Tasks | 14 scheduled tasks |
| AI Functions | 7 (auto_cluster, generate_ideas, generate_content, generate_images, generate_image_prompts, optimize_content, generate_site_structure) |
### 2.3 Container Inventory (docker-compose.app.yml — 7 containers)
| Container | Image | Host Port | Role |
|-----------|-------|-----------|------|
| igny8_backend | igny8-backend:latest | 8011 | Django + Gunicorn (4 workers, 120s timeout) |
| igny8_frontend | igny8-frontend-dev:latest | 8021 | Vite dev server (port 5173 internal) |
| igny8_marketing_dev | igny8-marketing-dev:latest | 8023 | Marketing site dev server (port 5174 internal) |
| igny8_celery_worker | igny8-backend:latest | — | Celery worker (concurrency=4) |
| igny8_celery_beat | igny8-backend:latest | — | Celery beat scheduler |
| igny8_flower | igny8-backend:latest | 5555 | Celery monitoring |
*Plus shared infra containers (external to app compose): postgres, redis, caddy, portainer, pgadmin, filebrowser*
### 2.4 Existing Django Apps (34 in INSTALLED_APPS)
**Business layer:** automation, notifications, optimization, publishing, integration
**Module layer:** planner (keywords/clusters/ideas), writer (tasks/content/images), billing, system, linker (inactive), optimizer (inactive), publisher, integration
**Auth & core:** auth (Account, Site, Sector, User, Plan), ai, plugins, admin
### 2.5 Existing Models (key entities)
| App | Models |
|-----|--------|
| auth | Account, Plan, Subscription, Site, Sector, Industry, IndustrySector, SeedKeyword, User, SiteUserAccess |
| planning | Clusters (status: new/mapped), Keywords, ContentIdeas |
| content | Tasks, Content (content_type: post/page/product/taxonomy), ContentTaxonomy, ContentTaxonomyRelation, Images, ImagePrompts |
| automation | DefaultAutomationConfig, AutomationConfig (per-site), AutomationRun |
| integration | SiteIntegration, SyncEvent, PublishingSettings |
| publishing | PublishingRecord, DeploymentRecord |
| billing | CreditTransaction, CreditUsageLog, CreditCostConfig, AccountPaymentMethod, Payment, Invoice |
| system | IntegrationProvider, AIPrompt, IntegrationSettings, AuthorProfile |
| ai | AITaskLog |
| plugins | Plugin, PluginVersion, PluginDownload |
| notifications | Notification |
| optimization | OptimizationTask |
### 2.6 7-Stage Automation Pipeline
| Stage | Function | AI | Batch Size |
|-------|----------|-----|-----------|
| 1 | Keywords → Clusters | Yes (auto_cluster) | 50 |
| 2 | Clusters → Ideas | Yes (generate_ideas) | 1 |
| 3 | Ideas → Tasks | No | 20 |
| 4 | Tasks → Content | Yes (generate_content) | 1 |
| 5 | Content → Image Prompts | Yes (generate_image_prompts) | 1 |
| 6 | Image Prompts → Images | Yes (generate_images) | 1 |
| 7 | Auto-approval → Publish | No | — |
### 2.7 What Does NOT Exist (common misconceptions from planning docs)
- **No `sag/` app** — no SAGBlueprint, SAGAttribute, SAGCluster, or SectorAttributeTemplate models
- **No UUID primary keys** — all models use BigAutoField (integer)
- **No `sag_blueprint` field on Site model**
- **No `blueprint_context` field on Content or Tasks models**
- **No `self_hosted_ai` provider** in IntegrationProvider
- **No `/sag/site-analysis` endpoint** in the WordPress plugin
- **Content already has** `content_type` (post/page/product/taxonomy) and `content_structure` (article/guide/comparison/review/listicle/landing_page/etc) — these are not new fields
- **Linker & Optimizer modules** exist in code but are **inactive** (behind feature flags)
**Conclusion:** Phase 0 is pure migration. No bug-fixing sprint needed. Current environment stays untouched — all new work on new server with zero downtime. **Conclusion:** Phase 0 is pure migration. No bug-fixing sprint needed. Current environment stays untouched — all new work on new server with zero downtime.
## 3. Architecture Overview ## 3. Architecture Overview
**Current:** Single VPS, dev environment running as production, 14 Docker containers (6 unnecessary), Gitea self-hosted, no staging. **Current:** Single VPS running IGNY8 app containers + shared Alorig infrastructure containers. App-level: 7 containers in `docker-compose.app.yml` (backend, frontend, marketing_dev, celery_worker, celery_beat, flower) + shared infra containers (postgres, redis, caddy, portainer, pgadmin, filebrowser). Of the 7 app containers, `marketing_dev` and `flower` are non-essential for production. Gitea self-hosted for git, no staging environment, no GitHub.
**Target:** New Hostinger KVM 4 (4 vCPU, 16GB RAM, 200GB NVMe), shared Alorig infrastructure, production + staging environments, GitHub for all repos, Cloudflare DNS, self-hosted AI on Vast.ai GPU. **Target:** New Hostinger KVM 4 (4 vCPU, 16GB RAM, 200GB NVMe) with shared Alorig infrastructure stack (PG, Redis, Caddy, Portainer). IGNY8 app runs 3 core containers (backend, celery_worker, celery_beat) + frontend served via Caddy. Same pattern for all other Alorig apps. Production + staging environments, GitHub for all repos, Cloudflare DNS, self-hosted AI on Vast.ai GPU.
**IGNY8 v2 Transformation:** From keyword-driven content generator → structure-first SAG-powered site architecture engine. Attributes first, not keywords first. Keywords emerge from attribute intersections across 45 industries, 449 sectors. **IGNY8 v2 Transformation:** From keyword-driven content generator → structure-first SAG-powered site architecture engine. Attributes first, not keywords first. Keywords emerge from attribute intersections across 45 industries, 449 sectors.
## 4. Technology Stack ## 4. Technology Stack
| Layer | Current (v1.8.4) | V2 Addition | | Layer | Current (v1.8.4) — Verified | V2 Addition |
|-------|-------------------|-------------| |-------|-------------------|-------------|
| Backend | Django 5.1, DRF, PostgreSQL, Redis, Celery | SAG models, new module APIs | | Backend | Django >=5.2.7, DRF, PostgreSQL (external), Redis (external), Celery >=5.3.0, Python 3.11 | SAG models, new module APIs |
| Frontend | React 19, TypeScript, Zustand, Tailwind | Blueprint UI, wizard, dashboards | | Frontend | React ^19.0.0, TypeScript ~5.7.2, Zustand ^5.0.8, Tailwind ^4.0.8, Vite ^6.1.0, Node 18 | Blueprint UI, wizard, dashboards |
| AI (Cloud) | OpenAI GPT/DALL-E, Anthropic Claude, ElevenLabs | — | | AI (Cloud) | OpenAI (via IntegrationProvider), Anthropic (via IntegrationSettings), Runware (images), DALL-E (images) | — |
| AI (Self-hosted) | — | Qwen3 (text), FLUX/SD (images), Wan 2.1 (video) via Vast.ai | | AI (Self-hosted) | — | Qwen3 (text), FLUX/SD (images), Wan 2.1 (video) via Vast.ai |
| WordPress | Bridge plugin v1.3.3 | Plugin v2 (14 modules), Companion Theme, Toolkit | | WordPress | IGNY8 WordPress Bridge v1.5.2 | Plugin v2 (14 modules), Companion Theme, Toolkit |
| Infrastructure | Single VPS, Gitea, no staging | KVM 4 + Vast.ai GPU, GitHub, Cloudflare, prod + staging | | Infrastructure | Single VPS, Gitea self-hosted, no staging, Caddy reverse proxy | KVM 4 + Vast.ai GPU, GitHub, Cloudflare, prod + staging |
| DevOps | Manual | Claude Code via SSH | | DevOps | Manual | Claude Code via SSH |
## 5. Complete Execution Map ## 5. Complete Execution Map
@@ -59,8 +143,8 @@ This is the single master document governing the complete IGNY8 V2 build — fro
| 0A | `00A-github-repo-consolidation.md` | All repos → 1 GitHub account, linked to Source-Codes/, remove Gitea | — | | 0A | `00A-github-repo-consolidation.md` | All repos → 1 GitHub account, linked to Source-Codes/, remove Gitea | — |
| 0B | `00B-vps-provisioning.md` | New KVM 4, Cloudflare DNS, shared Docker infra (PG/Redis/Caddy/Portainer) | 0A | | 0B | `00B-vps-provisioning.md` | New KVM 4, Cloudflare DNS, shared Docker infra (PG/Redis/Caddy/Portainer) | 0A |
| 0C | `00C-igny8-production-migration.md` | pg_dump → new server, Docker Compose, DNS cutover, zero downtime | 0B | | 0C | `00C-igny8-production-migration.md` | pg_dump → new server, Docker Compose, DNS cutover, zero downtime | 0B |
| 0D | `00D-staging-environment.md` | Identical 3-container staging, separate DB + Redis prefix | 0C | | 0D | `00D-staging-environment.md` | Staging environment: backend + celery_worker + celery_beat + frontend, separate DB (`igny8_staging_db`) + Redis DB 1 | 0C |
| 0E | `00E-legacy-cleanup.md` | Kill Gitea + 5 containers (frontend-dev, marketing, pgadmin, filebrowser, setup-helper), ~1.5GB freed | 0C | | 0E | `00E-legacy-cleanup.md` | Kill Gitea + non-essential containers (marketing_dev, flower, pgadmin, filebrowser), decommission old VPS | 0C |
| 0F | `00F-self-hosted-ai-infra.md` | Vast.ai GPU (2×RTX 3090) + SSH tunnel + LiteLLM + Ollama/Qwen3 + ComfyUI | 0B | | 0F | `00F-self-hosted-ai-infra.md` | Vast.ai GPU (2×RTX 3090) + SSH tunnel + LiteLLM + Ollama/Qwen3 + ComfyUI | 0B |
### Phase 1 — SAG Core Engine ### Phase 1 — SAG Core Engine
@@ -168,13 +252,15 @@ After all V2-Execution-Docs are built, the following source locations get archiv
## 8. Key Principles ## 8. Key Principles
1. **Nothing working breaks** — nullable fields, feature flags, staging first 1. **Codebase is the single source of truth** — every technical claim in execution docs verified against actual code, not planning/reference docs
2. **SAG is attribute-first** — keywords are output, not input 2. **Nothing working breaks** — nullable fields, feature flags, staging first
3. **Same container pattern everywhere** — backend + celery_worker + celery_beat 3. **SAG is attribute-first** — keywords are output, not input
4. **Current environment never touched** — all new work on new server 4. **Same container pattern everywhere** — backend + celery_worker + celery_beat per app, shared infra (PG/Redis/Caddy) across all Alorig apps
5. **All development via Claude Code** — SSH to VPS, timelines compressed vs manual dev 5. **Current environment never touched** — all new work on new server
6. **Each doc is self-contained** — Claude Code executes one doc at a time without losing context 6. **All development via Claude Code** — SSH to VPS, timelines compressed vs manual dev
7. **Monitor real usage** — upgrade decisions are data-driven, not speculative 7. **Each doc is self-contained** — Claude Code executes one doc at a time without losing context
8. **Coexistence with existing models** — new SAG models must define migration path for existing Clusters/Keywords/Content, not ignore them
9. **Monitor real usage** — upgrade decisions are data-driven, not speculative
## 9. Timeline Estimate (Claude Code Execution) ## 9. Timeline Estimate (Claude Code Execution)

19
v2/V2-Execution-Docs/00A-github-repo-consolidation.md
View File

@@ -1,11 +1,12 @@
# IGNY8 Phase 0: GitHub Repository Consolidation # IGNY8 Phase 0: GitHub Repository Consolidation
## Document 00A: Complete GitHub Repo Consolidation Strategy ## Document 00A: Complete GitHub Repo Consolidation Strategy
**Document Version:** 1.0 **Document Version:** 1.1
**Last Updated:** 2026-03-23 **Last Updated:** 2026-03-23
**Status:** In Development **Status:** In Development
**Phase:** Phase 0 - Infrastructure Setup **Phase:** Phase 0 - Infrastructure Setup
**Priority:** High (blocking all other development) **Priority:** High (blocking all other development)
**Source of Truth:** Codebase at `/data/app/igny8/`
--- ---
@@ -35,15 +36,17 @@
- `igny8-app` contains Django/React application code - `igny8-app` contains Django/React application code
- Both typically cloned/mounted in development containers - Both typically cloned/mounted in development containers
### 1.2 Current Stack Versions ### 1.2 Current Stack Versions (Verified from codebase)
``` ```
Backend: Django 5.1 Backend: Django >=5.2.7 (requirements.txt), Python 3.11-slim (Dockerfile)
Frontend: React 19 Frontend: React ^19.0.0, TypeScript ~5.7.2, Vite ^6.1.0, Node 18-alpine (Dockerfile.dev)
Database: PostgreSQL 16 Database: PostgreSQL (external container, version set by infra stack)
Cache: Redis 7 Cache: Redis (external container, version set by infra stack)
Proxy: Caddy 2 Proxy: Caddy 2 (external container)
Task Queue: Celery 5.4 Task Queue: Celery >=5.3.0 (requirements.txt)
State: Zustand ^5.0.8
CSS: Tailwind ^4.0.8
Orchestration: Docker Compose Orchestration: Docker Compose
External Network: igny8_net External Network: igny8_net
``` ```

26
v2/V2-Execution-Docs/00B-vps-provisioning.md
View File

@@ -4,6 +4,7 @@
**Date Created:** 2026-03-23 **Date Created:** 2026-03-23
**Phase:** 0 (Infrastructure Setup) **Phase:** 0 (Infrastructure Setup)
**Document ID:** 00B **Document ID:** 00B
**Source of Truth:** Codebase at `/data/app/igny8/`
--- ---
@@ -80,16 +81,21 @@ This section is the single source of truth for all target versions across the en
### 2.3 Application Stack (reference — installed during 00C/00D) ### 2.3 Application Stack (reference — installed during 00C/00D)
| Component | Version | Notes | **IMPORTANT:** These are the versions the current codebase actually runs. Any version upgrades (e.g., Python 3.14, Django 6.0, Node 24) are separate upgrade tasks that require code changes, dependency testing, and migration work — not just a Dockerfile change. Phase 0 migrates the app as-is.
|-----------|---------|-------|
| Python | 3.14 | For backend Dockerfile | | Component | Current (Verified) | Upgrade Target (Separate Task) | Notes |
| Node.js | 24 LTS | For frontend Dockerfile | |-----------|-------------------|-------------------------------|-------|
| Django | 6.0 | Backend framework | | Python | 3.11-slim | TBD (3.13+ when deps support it) | Dockerfile: `python:3.11-slim` |
| Django REST Framework | Latest | API serializers | | Node.js | 18-alpine | TBD (20 LTS or 22 LTS) | Dockerfile.dev: `node:18-alpine` |
| Celery | 5.6 | Task queue | | Django | >=5.2.7 | TBD (6.0 when stable) | requirements.txt constraint |
| Gunicorn | 25 | WSGI application server | | Django REST Framework | Latest (unpinned) | Same | requirements.txt |
| Vite | 8 | Frontend build tool | | Celery | >=5.3.0 | Same | requirements.txt |
| React | Latest | Frontend library | | Gunicorn | Latest (unpinned) | Same | requirements.txt |
| Vite | ^6.1.0 | Same | package.json |
| React | ^19.0.0 | Same | package.json |
| TypeScript | ~5.7.2 | Same | package.json |
| Zustand | ^5.0.8 | Same | package.json |
| Tailwind CSS | ^4.0.8 | Same | package.json |
--- ---

137
v2/V2-Execution-Docs/00C-igny8-production-migration.md
View File

@@ -2,9 +2,10 @@
**Document ID:** 00C-igny8-production-migration **Document ID:** 00C-igny8-production-migration
**Phase:** Phase 0: Production Migration **Phase:** Phase 0: Production Migration
**Version:** 2.0 **Version:** 2.1
**Date:** 2026-03-23 **Date:** 2026-03-23
**Status:** In Progress **Status:** In Progress
**Source of Truth:** Codebase at `/data/app/igny8/`
**Related Docs:** **Related Docs:**
- 00A: GitHub Repository Consolidation (completed) - 00A: GitHub Repository Consolidation (completed)
@@ -23,18 +24,29 @@
**Project Name:** igny8-app **Project Name:** igny8-app
**Compose File:** docker-compose.app.yml **Compose File:** docker-compose.app.yml
#### Active Containers #### Active Containers (Verified from docker-compose.app.yml)
**App containers (7 in docker-compose.app.yml):**
| Container | Service | Port (Host:Container) | Technology |
|-----------|---------|----------------------|------------|
| igny8_backend | REST API | 8011:8010 | Django >=5.2.7 + Gunicorn (4 workers, 120s timeout) |
| igny8_frontend | Web UI | 8021:5173 | Vite dev server (React ^19, Node 18) |
| igny8_marketing_dev | Marketing site | 8023:5174 | Vite dev server |
| igny8_celery_worker | Task worker | — | Celery (concurrency=4) |
| igny8_celery_beat | Task scheduler | — | Celery Beat |
| igny8_flower | Celery monitor | 5555:5555 | Flower |
**Shared infra containers (external to app compose, on igny8_net):**
| Container | Service | Port (Internal) | Technology | | Container | Service | Port (Internal) | Technology |
|-----------|---------|----------------|------------| |-----------|---------|----------------|------------|
| igny8_backend | REST API | 8010 | Django 4.2 + Gunicorn (4 workers, 120s timeout) | | postgres | Database | 5432 | PostgreSQL (version set by infra stack) |
| igny8_frontend | Web UI | 5173 → 8021 | Vite dev server | | redis | Cache/Broker | 6379 | Redis (DB 0 for production) |
| igny8_celery_worker | Task worker | N/A | Celery |
| igny8_celery_beat | Task scheduler | N/A | Celery Beat |
| igny8_postgres | Database | 5432 | PostgreSQL 16 |
| igny8_redis | Cache/Broker | 6379 | Redis 7 (DB 0) |
| caddy | Reverse proxy/SSL | 80, 443 | Caddy 2 | | caddy | Reverse proxy/SSL | 80, 443 | Caddy 2 |
| marketing | Render service | 8023 | Custom service | | portainer | Docker management | 9000 | Portainer CE |
| sites | Render service | 8024 | Custom service | | pgadmin | DB admin | 5050 | PgAdmin 4 |
| filebrowser | File management | 8080 | FileBrowser |
#### Database #### Database
- **Database Name:** igny8_db - **Database Name:** igny8_db
@@ -61,7 +73,7 @@
- CELERY_BROKER_URL - CELERY_BROKER_URL
- Django DEBUG, ALLOWED_HOSTS - Django DEBUG, ALLOWED_HOSTS
**Important:** AI integration keys stored in database (GlobalIntegrationSettings table), NOT in env vars. **Important:** AI integration keys stored in database (`IntegrationProvider` table: `igny8_integration_providers`, and `IntegrationSettings` table: `igny8_integration_settings`), NOT in env vars.
#### Networking #### Networking
- **Primary Domain:** app.igny8.com (frontend) - **Primary Domain:** app.igny8.com (frontend)
@@ -77,9 +89,11 @@
- **Backup Automation:** Cron jobs on old VPS (backup-db.sh, backup-full.sh) - **Backup Automation:** Cron jobs on old VPS (backup-db.sh, backup-full.sh)
#### Health Check #### Health Check
- **Endpoint:** http://localhost:8010/api/v1/system/status/ - **Endpoint:** http://localhost:8010/api/v1/system/status/ (inside container) or http://localhost:8011/api/v1/system/status/ (from host)
- **Expected Response:** 200 OK with system status JSON - **Expected Response:** 200 OK with system status JSON (timestamp, system resources, database, Redis, Celery status)
- **Permission:** AllowAny (public endpoint)
- **Frequency:** Manual or via monitoring - **Frequency:** Manual or via monitoring
- **Docker healthcheck:** Configured in compose: 30s interval, 10s timeout, 3 retries
--- ---
@@ -167,30 +181,60 @@ This migration is **not a direct cutover**. Instead, we run both VPS in parallel
The database schema itself does not change during migration. We use pg_dump and pg_restore to move the entire database from old VPS (PG 16) to new VPS (PG 18). The database schema itself does not change during migration. We use pg_dump and pg_restore to move the entire database from old VPS (PG 16) to new VPS (PG 18).
**Key Tables (not exhaustive):** **Key Tables (verified from codebase — all use `igny8_` prefix convention):**
- `users` — User accounts
- `projects` — Projects/sites | Table | Purpose |
- `stripe_subscriptions` — Payment records |-------|---------|
- `integration_settings` — AI integration keys (GlobalIntegrationSettings) | `igny8_users` | User accounts (AUTH_USER_MODEL) |
- `wordpress_sync_logs` — Plugin sync history | `igny8_tenants` | Multi-tenant accounts |
- `celery_*` — Celery task tables | `igny8_sites` | Sites within accounts |
| `igny8_subscriptions` | Subscription records |
| `igny8_plans` | Plan definitions |
| `igny8_content` | Content items |
| `igny8_tasks` | Writer tasks |
| `igny8_clusters` | Keyword clusters |
| `igny8_keywords` | Keywords |
| `igny8_content_ideas` | Content ideas |
| `igny8_images` | Generated images |
| `igny8_invoices` | Billing invoices |
| `igny8_payments` | Payment records |
| `igny8_webhook_events` | Stripe/PayPal webhooks |
| `igny8_site_integrations` | WordPress site connections |
| `igny8_sync_events` | WordPress sync history |
| `igny8_publishing_records` | Publish records |
| `igny8_ai_task_logs` | AI task audit trail |
| `igny8_automation_configs` | Automation settings |
| `igny8_automation_runs` | Automation run history |
| `plugins` / `plugin_versions` / `plugin_installations` / `plugin_downloads` | Plugin system |
| `igny8_integration_providers` / `igny8_integration_settings` | AI/payment provider keys |
**Note:** There is NO `stripe_subscriptions` table, NO `wordpress_sync_logs` table, NO `GlobalIntegrationSettings` table. These were errors in earlier doc versions.
**Important:** Do NOT manually migrate tables. Use pg_dump/pg_restore with custom format. **Important:** Do NOT manually migrate tables. Use pg_dump/pg_restore with custom format.
### 3.2 Health Check API ### 3.2 Health Check API
**Endpoint:** `GET http://localhost:8010/api/v1/system/status/` **Endpoint:** `GET /api/v1/system/status/` (AllowAny — no auth required)
**Expected Response:**
**Actual response format** (verified from `igny8_core/modules/system/views.py:system_status`):
```json ```json
{ {
"status": "ok", "timestamp": "2026-03-23T12:00:00.000000+00:00",
"version": "1.8.4", "system": {
"database": "connected", "cpu": {"usage_percent": 12.5, "cores": 4, "status": "healthy"},
"redis": "connected", "memory": {"total_gb": 8.0, "used_gb": 3.2, "available_gb": 4.8, "usage_percent": 40.0, "status": "healthy"},
"celery": "ok" "disk": {"total_gb": 100.0, "used_gb": 35.0, "free_gb": 65.0, "usage_percent": 35.0, "status": "healthy"}
},
"database": {"connected": true, "version": "PostgreSQL 16.x ...", "size": "256 MB", "active_connections": 5, "status": "healthy"},
"redis": {"connected": true, "status": "healthy", "info": {}},
"celery": {"workers": ["celery@igny8_celery_worker"], "worker_count": 1, "tasks": {"active": 0, "scheduled": 0, "reserved": 0}, "status": "healthy"},
"processes": {},
"modules": {}
} }
``` ```
**Healthy indicators:** All `status` fields should be `"healthy"`, `database.connected` and `redis.connected` should be `true`, `celery.worker_count` should be ≥ 1.
Use this endpoint to verify both old and new VPS health before/after migration. Use this endpoint to verify both old and new VPS health before/after migration.
--- ---
@@ -222,11 +266,11 @@ git checkout main # or appropriate branch
cp .env.example .env cp .env.example .env
# Update .env for new VPS: # Update .env for new VPS:
# - DB_HOST=igny8_postgres (Docker internal hostname) # - DB_HOST=postgres (Docker service name on igny8_net — infra container, not app container)
# - DB_NAME=igny8_db # - DB_NAME=igny8_db
# - DB_USER=igny8 # - DB_USER=igny8
# - DB_PASSWORD=<secure password> # - DB_PASSWORD=<secure password>
# - REDIS_HOST=igny8_redis # - REDIS_URL=redis://redis:6379/0 (Redis is also an infra container on igny8_net)
# - SECRET_KEY=<generate new> # - SECRET_KEY=<generate new>
# - ALLOWED_HOSTS=test-app.igny8.com,test-api.igny8.com,app.igny8.com,api.igny8.com,igny8.com # - ALLOWED_HOSTS=test-app.igny8.com,test-api.igny8.com,app.igny8.com,api.igny8.com,igny8.com
@@ -282,7 +326,7 @@ PGPASSWORD=<db-password> pg_restore --format=custom \
/tmp/igny8_db_backup.dump /tmp/igny8_db_backup.dump
# Verify restore completed # Verify restore completed
PGPASSWORD=<db-password> psql --host=localhost --username=igny8 --dbname=igny8_db -c "SELECT COUNT(*) FROM users;" PGPASSWORD=<db-password> psql --host=localhost --username=igny8 --dbname=igny8_db -c "SELECT COUNT(*) FROM igny8_users;"
# Run ANALYZE on all tables to update statistics # Run ANALYZE on all tables to update statistics
PGPASSWORD=<db-password> psql --host=localhost --username=igny8 --dbname=igny8_db -c "ANALYZE;" PGPASSWORD=<db-password> psql --host=localhost --username=igny8 --dbname=igny8_db -c "ANALYZE;"
@@ -315,16 +359,15 @@ On new VPS:
```bash ```bash
cd /data/app/igny8 cd /data/app/igny8
# Copy docker-compose file (create if needed) # docker-compose.app.yml should contain these app containers:
# Ensure it contains: # - igny8_backend (Django + Gunicorn)
# - igny8_backend (Django) # - igny8_frontend (Vite/React)
# - igny8_frontend (Vite) # - igny8_marketing_dev (Vite marketing site)
# - igny8_celery_worker # - igny8_celery_worker (Celery)
# - igny8_celery_beat # - igny8_celery_beat (Celery Beat)
# - igny8_postgres (PostgreSQL 18) # - igny8_flower (Flower monitor)
# - igny8_redis # NOTE: postgres, redis, caddy are INFRA containers — NOT in app compose.
# - caddy # They must already be running on igny8_net (provisioned in 00B).
# - marketing, sites (if needed)
# Build and start # Build and start
docker compose -f docker-compose.app.yml build docker compose -f docker-compose.app.yml build
@@ -355,22 +398,14 @@ curl -I https://test-api.igny8.com
#### Step 1.7: Run Health Checks on Test Subdomains #### Step 1.7: Run Health Checks on Test Subdomains
```bash ```bash
# Health check via test API subdomain # Health check via test API subdomain (port 8011 is the host-mapped backend port)
curl -H "Host: test-api.igny8.com" http://localhost:8010/api/v1/system/status/ curl -H "Host: test-api.igny8.com" http://localhost:8011/api/v1/system/status/
# Or if DNS is live # Or if DNS is live
curl https://test-api.igny8.com/api/v1/system/status/ curl https://test-api.igny8.com/api/v1/system/status/
``` ```
**Expected Response:** **Verify response:** `database.connected` = true, `redis.connected` = true, `celery.worker_count` ≥ 1, all `status` fields = "healthy". See Section 3.2 for full response format.
```json
{
"status": "ok",
"version": "1.8.4",
"database": "connected",
"redis": "connected",
"celery": "ok"
}
``` ```
#### Step 1.8: Manual Testing on Test Subdomains #### Step 1.8: Manual Testing on Test Subdomains

437
v2/V2-Execution-Docs/00D-staging-environment.md
View File

@@ -1,9 +1,11 @@
# IGNY8 Phase 0: Staging Environment Setup (Doc 00D) # IGNY8 Phase 0: Staging Environment Setup (Doc 00D)
**Document Status:** Build Specification **Document Status:** Build Specification
**Version:** 2.1
**Date Created:** 2026-03-23 **Date Created:** 2026-03-23
**Target Phase:** Phase 0 - Infrastructure & Deployment **Target Phase:** Phase 0 - Infrastructure & Deployment
**Related Docs:** [00B Infrastructure Setup](00B-infrastructure-setup.md) | [00C Production Migration](00C-production-migration.md) | [00B Version Matrix](00B-infrastructure-setup.md#version-matrix) (SINGLE SOURCE OF TRUTH for all versions) **Source of Truth:** Codebase at `/data/app/igny8/`
**Related Docs:** [00B Infrastructure Setup](00B-infrastructure-setup.md) | [00C Production Migration](00C-production-migration.md)
**Key Details:** **Key Details:**
- Staging runs on the NEW VPS (from 00B Infrastructure Setup) - Staging runs on the NEW VPS (from 00B Infrastructure Setup)
@@ -19,32 +21,33 @@
**Staging Environment Location:** On the NEW VPS, as provisioned in 00B Infrastructure Setup. **Staging Environment Location:** On the NEW VPS, as provisioned in 00B Infrastructure Setup.
**Note on Versions:** For all component versions (PostgreSQL, Redis, Docker, etc.), refer to the **Version Matrix in 00B Infrastructure Setup** as the single source of truth. This document reflects those versions. All staging components use the latest versions matching production on the NEW VPS. **Note on Versions:** For component versions, the codebase (requirements.txt, package.json, Dockerfiles) is the source of truth. Aspirational upgrade targets are in 00B but current verified versions are: Python 3.11, Django >=5.2.7, Node 18, React ^19, Vite ^6.1.0, Celery >=5.3.0.
### 1.1 Infrastructure Baseline ### 1.1 Infrastructure Baseline
- **Host Server:** Single Linux VM running Docker on NEW VPS (from 00B Infrastructure Setup) - **Host Server:** Single Linux VM running Docker on NEW VPS (from 00B Infrastructure Setup)
- **Base OS:** Ubuntu 24.04 LTS - **Base OS:** Ubuntu (version per 00B)
- **Shared Resources:** - **Shared Resources:**
- PostgreSQL 18 server (port 5432) - PostgreSQL server (port 5432) — version set by infra stack
- Redis 8 server (port 6379) - Redis server (port 6379) — version set by infra stack
- Docker network: `igny8_net` - Docker network: `igny8_net`
- Caddy 2.11 reverse proxy (port 80/443) - Caddy reverse proxy (port 80/443)
- Cloudflare DNS management (may or may not be active - dependent on 00C flow stage) - Cloudflare DNS management (may or may not be active — dependent on 00C flow stage)
- Log directory: `/data/app/logs/` - Log directory: `/data/app/logs/` (production), `/data/logs/staging/` (staging)
### 1.2 Production Environment (Already Complete - Doc 00C) ### 1.2 Production Environment (Already Complete - Doc 00C)
- **Database:** `igny8_db` (PostgreSQL) - **Database:** `igny8_db` (PostgreSQL)
- **Cache:** Redis DB 0 - **Cache:** Redis DB 0
- **Compose file:** `docker-compose.yml` - **Compose file:** `docker-compose.app.yml` (project name: `igny8-app`)
- **Containers:** - **Containers (7):**
- `igny8_backend` (port 8010) - `igny8_backend` (host port 8011, container port 8010)
- `igny8_frontend` (port 5173) - `igny8_frontend` (host port 8021, container port 5173)
- `igny8_marketing_dev` (port 5174) - `igny8_marketing_dev` (host port 8023, container port 5174)
- `igny8_celery_worker` - `igny8_celery_worker`
- `igny8_celery_beat` - `igny8_celery_beat`
- **Env file:** `.env` (production settings) - `igny8_flower` (port 5555)
- **Env file:** `.env` (production settings) — NOT used inline; env vars set in compose
- **Domains:** igny8.com, api.igny8.com, marketing.igny8.com - **Domains:** igny8.com, api.igny8.com, marketing.igny8.com
- **Logs:** `/data/app/logs/production/` - **Logs:** `/data/app/logs/`
### 1.3 Staging Environment (To Be Built) ### 1.3 Staging Environment (To Be Built)
**Does not yet exist.** This document defines the complete staging setup. **Does not yet exist.** This document defines the complete staging setup.
@@ -61,11 +64,11 @@ A complete parallel environment sharing infrastructure with production:
│ Docker Containers (Staging) │ │ Docker Containers (Staging) │
├─────────────────────────────────────────────────────┤ ├─────────────────────────────────────────────────────┤
│ │ │ │
│ igny8_staging_backend:8012 → :8010 (Django) │ │ igny8_staging_backend:8012 → :8010 (Django/Gunicorn) │
│ igny8_staging_frontend:8024 → :5173 (Vue) │ │ igny8_staging_frontend:8024 → :5173 (React/Vite) │
│ igny8_staging_marketing_dev:8026 → :5174 (Nuxt) │ │ igny8_staging_marketing_dev:8026 → :5174 (Vite) │
│ igny8_staging_celery_worker │ │ igny8_staging_celery_worker │
│ igny8_staging_celery_beat │ │ igny8_staging_celery_beat │
│ │ │ │
└─────────────────────────────────────────────────────┘ └─────────────────────────────────────────────────────┘
↓ ↓
@@ -157,18 +160,20 @@ A complete parallel environment sharing infrastructure with production:
## 4. Implementation Steps ## 4. Implementation Steps
**Version Requirements:** All versions referenced below are from the **00B Version Matrix (source of truth for all versions)**. The staging environment uses identical versions to production on the NEW VPS: **Version Requirements:** All versions below are verified from the codebase. The staging environment uses identical versions to production:
- PostgreSQL 18 (postgres:18-alpine) - PostgreSQL (version set by infra stack)
- Redis 8 (redis:8-alpine) - Redis (version set by infra stack)
- Caddy 2.11 (caddy:2-alpine) - Caddy 2 (version set by infra stack)
- Ubuntu 24.04 LTS (base OS) - Ubuntu base OS (set by infra stack)
- Docker Engine 29.x - Docker Engine (installed on VPS)
- Python 3.14 (in backend container) - Python 3.11-slim (in backend Dockerfile)
- Node 24 LTS (in frontend and marketing containers) - Node 18-alpine (in frontend Dockerfile)
- Django 6.0 - Django >=5.2.7 (requirements.txt)
- Vite 8 - Vite ^6.1.0 (package.json)
- Gunicorn 25 - Gunicorn (requirements.txt)
- Celery 5.6 - Celery >=5.3.0 (requirements.txt)
**Note:** The actual `docker-compose.staging.yml` already exists in the repo and is the source of truth. The compose excerpt below is for reference only — always use the actual file.
### Step 1: Create Staging PostgreSQL Database ### Step 1: Create Staging PostgreSQL Database
@@ -196,15 +201,15 @@ GRANT ALL PRIVILEGES ON SCHEMA public TO igny8_user;
**Execution:** **Execution:**
```bash ```bash
# On host server # On host server
docker exec -i igny8_postgres psql -U postgres -d postgres << 'EOF' docker exec -i postgres psql -U postgres -d postgres << 'EOF'
CREATE DATABASE igny8_staging_db CREATE DATABASE igny8_staging_db
WITH OWNER igny8_user WITH OWNER igny8
ENCODING 'UTF8' ENCODING 'UTF8'
LOCALE 'en_US.UTF-8' LOCALE 'en_US.UTF-8'
TEMPLATE template0; TEMPLATE template0;
GRANT ALL PRIVILEGES ON DATABASE igny8_staging_db TO igny8_user; GRANT ALL PRIVILEGES ON DATABASE igny8_staging_db TO igny8;
GRANT ALL PRIVILEGES ON SCHEMA public TO igny8_user; GRANT ALL PRIVILEGES ON SCHEMA public TO igny8;
EOF EOF
echo "Staging database created" echo "Staging database created"
@@ -218,161 +223,141 @@ echo "Staging database created"
**Project Name:** `igny8-staging` **Project Name:** `igny8-staging`
```yaml ```yaml
version: '3.8' # Actual file: docker-compose.staging.yml (already exists in repo)
# Key differences from this reference: the actual file uses env_file: .env.staging
# and individual env vars (DB_HOST, DB_NAME) rather than DATABASE_URL format.
name: igny8-staging
services: services:
# Backend API Service # Backend API Service
igny8_staging_backend: igny8_staging_backend:
image: igny8-backend:staging image: igny8-backend:staging
container_name: igny8_staging_backend container_name: igny8_staging_backend
environment: restart: always
- DJANGO_ENV=staging working_dir: /app
- DEBUG=True
- ALLOWED_HOSTS=staging.igny8.com,staging-api.igny8.com,localhost,127.0.0.1
- SECRET_KEY=${STAGING_SECRET_KEY}
- DATABASE_URL=postgresql://igny8_user:${DB_PASSWORD}@igny8_postgres:5432/igny8_staging_db
- REDIS_URL=redis://igny8_redis:6379/1
- CELERY_BROKER_URL=redis://igny8_redis:6379/1
- CELERY_RESULT_BACKEND=redis://igny8_redis:6379/1
- CACHE_URL=redis://igny8_redis:6379/1
- CORS_ALLOWED_ORIGINS=https://staging.igny8.com,https://staging-marketing.igny8.com
- STRIPE_PUBLIC_KEY=${STAGING_STRIPE_PUBLIC_KEY}
- STRIPE_SECRET_KEY=${STAGING_STRIPE_SECRET_KEY}
- STRIPE_WEBHOOK_SECRET=${STAGING_STRIPE_WEBHOOK_SECRET}
- API_BASE_URL=https://staging-api.igny8.com
- FRONTEND_URL=https://staging.igny8.com
- MARKETING_URL=https://staging-marketing.igny8.com
- AWS_ACCESS_KEY_ID=${STAGING_AWS_ACCESS_KEY_ID}
- AWS_SECRET_ACCESS_KEY=${STAGING_AWS_SECRET_ACCESS_KEY}
- AWS_S3_BUCKET=${STAGING_AWS_S3_BUCKET}
- AWS_REGION=${AWS_REGION}
- SENTRY_DSN=${STAGING_SENTRY_DSN}
- LOG_LEVEL=INFO
ports: ports:
- "8012:8010" - "0.0.0.0:8012:8010"
environment:
DJANGO_ENV: staging
DB_HOST: postgres # External infra container name (NOT igny8_postgres)
DB_NAME: igny8_staging_db
DB_USER: igny8
DB_PASSWORD: igny8pass
REDIS_HOST: redis # External infra container name (NOT igny8_redis)
REDIS_PORT: "6379"
REDIS_DB: "1" # DB 1 for staging (production uses DB 0)
USE_SECURE_COOKIES: "True"
USE_SECURE_PROXY_HEADER: "True"
DEBUG: "False"
volumes: volumes:
- ./backend:/app/backend - /data/app/igny8/backend:/app:rw
- /data/app/logs/staging:/var/log/igny8 - /data/app/igny8:/data/app/igny8:rw
networks: - /var/run/docker.sock:/var/run/docker.sock:ro
- igny8_net - /data/logs/staging:/app/logs:rw
depends_on: env_file:
- igny8_postgres - .env.staging
- igny8_redis
restart: unless-stopped
healthcheck: healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8010/health/"] test: ["CMD-SHELL", "python -c \"import urllib.request; urllib.request.urlopen('http://localhost:8010/api/v1/system/status/').read()\" || exit 1"]
interval: 30s interval: 30s
timeout: 10s timeout: 10s
retries: 3 retries: 3
start_period: 40s start_period: 40s
labels: command: ["gunicorn", "igny8_core.wsgi:application", "--bind", "0.0.0.0:8010", "--workers", "2", "--timeout", "120"]
- "com.igny8.component=backend" networks: [igny8_net]
- "com.igny8.environment=staging"
# Frontend Service # Frontend Service (React + Vite)
igny8_staging_frontend: igny8_staging_frontend:
image: igny8-frontend-dev:staging image: igny8-frontend-dev:staging
container_name: igny8_staging_frontend container_name: igny8_staging_frontend
environment: restart: always
- NODE_ENV=staging
- VITE_API_URL=https://staging-api.igny8.com
- VITE_ENVIRONMENT=staging
ports: ports:
- "8024:5173" - "0.0.0.0:8024:5173"
environment:
VITE_BACKEND_URL: "https://staging-api.igny8.com/api"
VITE_ENV: "staging"
volumes: volumes:
- ./frontend:/app - /data/app/igny8/frontend:/app:rw
- /app/node_modules depends_on:
networks: igny8_staging_backend:
- igny8_net condition: service_healthy
restart: unless-stopped networks: [igny8_net]
labels:
- "com.igny8.component=frontend"
- "com.igny8.environment=staging"
# Marketing Site Service # Marketing Site Service (Vite, NOT Nuxt — built from frontend/Dockerfile.marketing.dev)
igny8_staging_marketing_dev: igny8_staging_marketing_dev:
image: igny8-marketing-dev:staging image: igny8-marketing-dev:staging
container_name: igny8_staging_marketing_dev container_name: igny8_staging_marketing_dev
environment: restart: always
- NODE_ENV=staging
- NUXT_PUBLIC_API_URL=https://staging-api.igny8.com
- NUXT_PUBLIC_ENVIRONMENT=staging
ports: ports:
- "8026:5174" - "0.0.0.0:8026:5174"
environment:
VITE_BACKEND_URL: "https://staging-api.igny8.com/api"
VITE_ENV: "staging"
volumes: volumes:
- ./marketing:/app - /data/app/igny8/frontend:/app:rw # Same frontend dir — marketing is a Vite build mode
- /app/.nuxt networks: [igny8_net]
- /app/node_modules
networks:
- igny8_net
restart: unless-stopped
labels:
- "com.igny8.component=marketing"
- "com.igny8.environment=staging"
# Celery Worker # Celery Worker
igny8_staging_celery_worker: igny8_staging_celery_worker:
image: igny8-backend:staging image: igny8-backend:staging
container_name: igny8_staging_celery_worker container_name: igny8_staging_celery_worker
command: celery -A backend.celery worker --loglevel=info --concurrency=2 restart: always
working_dir: /app
environment: environment:
- DJANGO_ENV=staging DJANGO_ENV: staging
- DEBUG=True DB_HOST: postgres
- SECRET_KEY=${STAGING_SECRET_KEY} DB_NAME: igny8_staging_db
- DATABASE_URL=postgresql://igny8_user:${DB_PASSWORD}@igny8_postgres:5432/igny8_staging_db DB_USER: igny8
- REDIS_URL=redis://igny8_redis:6379/1 DB_PASSWORD: igny8pass
- CELERY_BROKER_URL=redis://igny8_redis:6379/1 REDIS_HOST: redis
- CELERY_RESULT_BACKEND=redis://igny8_redis:6379/1 REDIS_PORT: "6379"
- AWS_ACCESS_KEY_ID=${STAGING_AWS_ACCESS_KEY_ID} REDIS_DB: "1"
- AWS_SECRET_ACCESS_KEY=${STAGING_AWS_SECRET_ACCESS_KEY} C_FORCE_ROOT: "true"
- AWS_S3_BUCKET=${STAGING_AWS_S3_BUCKET}
volumes: volumes:
- ./backend:/app/backend - /data/app/igny8/backend:/app:rw
- /data/app/logs/staging:/var/log/igny8 - /data/logs/staging:/app/logs:rw
networks: env_file:
- igny8_net - .env.staging
command: ["celery", "-A", "igny8_core", "worker", "--loglevel=info", "--concurrency=2"]
depends_on: depends_on:
- igny8_postgres igny8_staging_backend:
- igny8_redis condition: service_healthy
restart: unless-stopped networks: [igny8_net]
labels:
- "com.igny8.component=celery-worker"
- "com.igny8.environment=staging"
# Celery Beat (Scheduler) # Celery Beat (Scheduler)
igny8_staging_celery_beat: igny8_staging_celery_beat:
image: igny8-backend:staging image: igny8-backend:staging
container_name: igny8_staging_celery_beat container_name: igny8_staging_celery_beat
command: celery -A backend.celery beat --loglevel=info --scheduler django_celery_beat.schedulers:DatabaseScheduler restart: always
working_dir: /app
environment: environment:
- DJANGO_ENV=staging DJANGO_ENV: staging
- DEBUG=True DB_HOST: postgres
- SECRET_KEY=${STAGING_SECRET_KEY} DB_NAME: igny8_staging_db
- DATABASE_URL=postgresql://igny8_user:${DB_PASSWORD}@igny8_postgres:5432/igny8_staging_db DB_USER: igny8
- REDIS_URL=redis://igny8_redis:6379/1 DB_PASSWORD: igny8pass
- CELERY_BROKER_URL=redis://igny8_redis:6379/1 REDIS_HOST: redis
- CELERY_RESULT_BACKEND=redis://igny8_redis:6379/1 REDIS_PORT: "6379"
REDIS_DB: "1"
C_FORCE_ROOT: "true"
volumes: volumes:
- ./backend:/app/backend - /data/app/igny8/backend:/app:rw
- /data/app/logs/staging:/var/log/igny8 - /data/logs/staging:/app/logs:rw
networks: env_file:
- igny8_net - .env.staging
command: ["celery", "-A", "igny8_core", "beat", "--loglevel=info", "--scheduler", "django_celery_beat.schedulers:DatabaseScheduler"]
depends_on: depends_on:
- igny8_postgres igny8_staging_backend:
- igny8_redis condition: service_healthy
restart: unless-stopped networks: [igny8_net]
labels:
- "com.igny8.component=celery-beat"
- "com.igny8.environment=staging"
networks: networks:
igny8_net: igny8_net:
external: true external: true
volumes:
# Data volumes referenced from external production infrastructure
``` ```
> **Note:** The actual `docker-compose.staging.yml` in the repo is the definitive version. The above is aligned with it as of this writing.
--- ---
### Step 3: Create `.env.staging` ### Step 3: Create `.env.staging`
@@ -404,29 +389,27 @@ DB_PASSWORD=your-staging-db-password
# DATABASE # DATABASE
# ============================================================================== # ==============================================================================
DATABASE_ENGINE=postgresql DATABASE_ENGINE=postgresql
DATABASE_HOST=igny8_postgres DATABASE_HOST=postgres
DATABASE_PORT=5432 DATABASE_PORT=5432
DATABASE_NAME=igny8_staging_db DATABASE_NAME=igny8_staging_db
DATABASE_USER=igny8_user DATABASE_USER=igny8
DATABASE_PASSWORD=${DB_PASSWORD} DATABASE_PASSWORD=${DB_PASSWORD}
# Full URL for Django
DATABASE_URL=postgresql://igny8_user:${DB_PASSWORD}@igny8_postgres:5432/igny8_staging_db
# ============================================================================== # ==============================================================================
# CACHE & QUEUE (REDIS DB 1 - Separate from Production) # CACHE & QUEUE (REDIS DB 1 - Separate from Production)
# ============================================================================== # ==============================================================================
REDIS_HOST=igny8_redis REDIS_HOST=redis
REDIS_PORT=6379 REDIS_PORT=6379
REDIS_DB=1 REDIS_DB=1
REDIS_PASSWORD= REDIS_PASSWORD=
REDIS_URL=redis://igny8_redis:6379/1 REDIS_URL=redis://redis:6379/1
CACHE_URL=redis://igny8_redis:6379/1 CACHE_URL=redis://redis:6379/1
# ============================================================================== # ==============================================================================
# CELERY (Uses Redis DB 1) # CELERY (Uses Redis DB 1)
# ============================================================================== # ==============================================================================
CELERY_BROKER_URL=redis://igny8_redis:6379/1 CELERY_BROKER_URL=redis://redis:6379/1
CELERY_RESULT_BACKEND=redis://igny8_redis:6379/1 CELERY_RESULT_BACKEND=redis://redis:6379/1
CELERY_ACCEPT_CONTENT=json CELERY_ACCEPT_CONTENT=json
CELERY_TASK_SERIALIZER=json CELERY_TASK_SERIALIZER=json
@@ -458,20 +441,11 @@ AWS_S3_CUSTOM_DOMAIN=staging-assets.igny8.com
# ============================================================================== # ==============================================================================
# EXTERNAL SERVICES (STAGING / SANDBOX CREDENTIALS) # EXTERNAL SERVICES (STAGING / SANDBOX CREDENTIALS)
# ============================================================================== # ==============================================================================
# Email # Email (app uses Resend, not Mailgun)
MAILGUN_API_KEY=your-staging-mailgun-key # RESEND_API_KEY=your-staging-resend-key
MAILGUN_DOMAIN=staging-mail.igny8.com
# Analytics # Error Tracking (optional)
MIXPANEL_TOKEN=your-staging-mixpanel-token # SENTRY_DSN=https://your-staging-sentry-dsn
# Error Tracking
STAGING_SENTRY_DSN=https://your-staging-sentry-dsn
# SMS
TWILIO_ACCOUNT_SID=your-staging-twilio-sid
TWILIO_AUTH_TOKEN=your-staging-twilio-token
TWILIO_PHONE_NUMBER=+15551234567
# ============================================================================== # ==============================================================================
# SECURITY (STAGING) # SECURITY (STAGING)
@@ -490,16 +464,10 @@ DJANGO_SUPERUSER_EMAIL=admin@staging.igny8.com
DJANGO_SUPERUSER_PASSWORD=your-staging-admin-password DJANGO_SUPERUSER_PASSWORD=your-staging-admin-password
# ============================================================================== # ==============================================================================
# FRONTEND / VUE # FRONTEND (React + Vite)
# ============================================================================== # ==============================================================================
VITE_API_URL=https://staging-api.igny8.com VITE_BACKEND_URL=https://staging-api.igny8.com/api
VITE_ENVIRONMENT=staging VITE_ENV=staging
# ==============================================================================
# MARKETING / NUXT
# ==============================================================================
NUXT_PUBLIC_API_URL=https://staging-api.igny8.com
NUXT_PUBLIC_ENVIRONMENT=staging
``` ```
--- ---
@@ -602,28 +570,28 @@ docker exec igny8_caddy caddy reload --config /etc/caddy/Caddyfile
**Backend Image** **Backend Image**
```bash ```bash
cd /path/to/backend cd /data/app/igny8/backend
docker build -f Dockerfile -t igny8-backend:staging . docker build -f Dockerfile -t igny8-backend:staging .
# Verify # Verify
docker images | grep igny8-backend docker images | grep igny8-backend
``` ```
**Frontend Image** **Frontend Image (React/Vite)**
```bash ```bash
cd /path/to/frontend cd /data/app/igny8/frontend
docker build -f Dockerfile.dev -t igny8-frontend-dev:staging . docker build -f Dockerfile.dev -t igny8-frontend-dev:staging .
# Verify # Verify
docker images | grep igny8-frontend-dev docker images | grep igny8-frontend-dev
``` ```
**Marketing Image** **Marketing Image (built from frontend dir using Dockerfile.marketing.dev)**
```bash ```bash
cd /path/to/marketing cd /data/app/igny8/frontend
docker build -f Dockerfile.dev -t igny8-marketing-dev:staging . docker build -f Dockerfile.marketing.dev -t igny8-marketing-dev:staging .
# Verify # Verify
docker images | grep igny8-marketing-dev docker images | grep igny8-marketing-dev
@@ -675,7 +643,7 @@ PROJECT_NAME="igny8-staging"
COMPOSE_FILE="docker-compose.staging.yml" COMPOSE_FILE="docker-compose.staging.yml"
ENV_FILE=".env.staging" ENV_FILE=".env.staging"
LOG_DIR="/data/app/logs/staging" LOG_DIR="/data/app/logs/staging"
PROD_COMPOSE_FILE="docker-compose.yml" PROD_COMPOSE_FILE="docker-compose.app.yml"
PROD_ENV_FILE=".env" PROD_ENV_FILE=".env"
# Colors for output # Colors for output
@@ -749,21 +717,21 @@ verify_shared_services() {
log_info "Verifying shared infrastructure services..." log_info "Verifying shared infrastructure services..."
# Check PostgreSQL # Check PostgreSQL
if ! docker exec igny8_postgres pg_isready -U postgres &> /dev/null; then if ! docker exec postgres pg_isready -U postgres &> /dev/null; then
log_error "PostgreSQL service not running" log_error "PostgreSQL service not running"
exit 1 exit 1
fi fi
log_success "PostgreSQL verified" log_success "PostgreSQL verified"
# Check Redis # Check Redis
if ! docker exec igny8_redis redis-cli ping &> /dev/null; then if ! docker exec redis redis-cli ping &> /dev/null; then
log_error "Redis service not running" log_error "Redis service not running"
exit 1 exit 1
fi fi
log_success "Redis verified" log_success "Redis verified"
# Check Caddy # Check Caddy
if ! docker ps | grep -q igny8_caddy; then if ! docker ps | grep -q caddy; then
log_error "Caddy service not running" log_error "Caddy service not running"
exit 1 exit 1
fi fi
@@ -774,20 +742,20 @@ create_staging_database() {
log_info "Creating staging database..." log_info "Creating staging database..."
# Check if database already exists # Check if database already exists
if docker exec igny8_postgres psql -U postgres -lqt | cut -d \| -f 1 | grep -qw igny8_staging_db; then if docker exec postgres psql -U postgres -lqt | cut -d \| -f 1 | grep -qw igny8_staging_db; then
log_warn "Database 'igny8_staging_db' already exists, skipping creation" log_warn "Database 'igny8_staging_db' already exists, skipping creation"
return 0 return 0
fi fi
docker exec -i igny8_postgres psql -U postgres -d postgres << 'EOF' docker exec -i postgres psql -U postgres -d postgres << 'EOF'
CREATE DATABASE igny8_staging_db CREATE DATABASE igny8_staging_db
WITH OWNER igny8_user WITH OWNER igny8
ENCODING 'UTF8' ENCODING 'UTF8'
LOCALE 'en_US.UTF-8' LOCALE 'en_US.UTF-8'
TEMPLATE template0; TEMPLATE template0;
GRANT ALL PRIVILEGES ON DATABASE igny8_staging_db TO igny8_user; GRANT ALL PRIVILEGES ON DATABASE igny8_staging_db TO igny8;
GRANT ALL PRIVILEGES ON SCHEMA public TO igny8_user; GRANT ALL PRIVILEGES ON SCHEMA public TO igny8;
EOF EOF
log_success "Staging database created" log_success "Staging database created"
@@ -835,7 +803,8 @@ create_superuser() {
--file "$COMPOSE_FILE" \ --file "$COMPOSE_FILE" \
--env-file "$ENV_FILE" \ --env-file "$ENV_FILE" \
exec -T igny8_staging_backend python manage.py shell << 'EOF' exec -T igny8_staging_backend python manage.py shell << 'EOF'
from django.contrib.auth.models import User from django.contrib.auth import get_user_model
User = get_user_model()
print(User.objects.filter(is_superuser=True).exists()) print(User.objects.filter(is_superuser=True).exists())
EOF EOF
) )
@@ -862,7 +831,8 @@ EOF
--file "$COMPOSE_FILE" \ --file "$COMPOSE_FILE" \
--env-file "$ENV_FILE" \ --env-file "$ENV_FILE" \
exec -T igny8_staging_backend python manage.py shell << 'EOF' exec -T igny8_staging_backend python manage.py shell << 'EOF'
from django.contrib.auth.models import User from django.contrib.auth import get_user_model
User = get_user_model()
user = User.objects.get(username='admin') user = User.objects.get(username='admin')
user.set_password('$DJANGO_SUPERUSER_PASSWORD') user.set_password('$DJANGO_SUPERUSER_PASSWORD')
user.save() user.save()
@@ -878,7 +848,7 @@ health_check() {
RETRY_COUNT=0 RETRY_COUNT=0
while [ $RETRY_COUNT -lt $RETRIES ]; do while [ $RETRY_COUNT -lt $RETRIES ]; do
if curl -s -f "http://localhost:8012/health/" &> /dev/null; then if curl -s -f "http://localhost:8012/api/v1/system/status/" &> /dev/null; then
log_success "Backend health check passed" log_success "Backend health check passed"
return 0 return 0
fi fi
@@ -967,8 +937,8 @@ set -e # Exit on error
# Configuration # Configuration
PROD_DB="igny8_db" PROD_DB="igny8_db"
STAGING_DB="igny8_staging_db" STAGING_DB="igny8_staging_db"
DB_HOST="igny8_postgres" DB_HOST="postgres"
DB_USER="igny8_user" DB_USER="igny8"
BACKUP_DIR="/data/backups/staging" BACKUP_DIR="/data/backups/staging"
TIMESTAMP=$(date +%Y%m%d_%H%M%S) TIMESTAMP=$(date +%Y%m%d_%H%M%S)
@@ -1030,8 +1000,8 @@ check_prerequisites() {
exit 1 exit 1
fi fi
# Check PostgreSQL accessibility # Check PostgreSQL accessibility (container name is 'postgres', not 'igny8_postgres')
if ! docker exec igny8_postgres pg_isready -U "$DB_USER" -h "$DB_HOST" &> /dev/null; then if ! docker exec postgres pg_isready -U "$DB_USER" &> /dev/null; then
log_error "Cannot connect to PostgreSQL" log_error "Cannot connect to PostgreSQL"
exit 1 exit 1
fi fi
@@ -1050,7 +1020,7 @@ backup_staging_db() {
BACKUP_FILE="$BACKUP_DIR/igny8_staging_db_${TIMESTAMP}.sql.gz" BACKUP_FILE="$BACKUP_DIR/igny8_staging_db_${TIMESTAMP}.sql.gz"
docker exec igny8_postgres pg_dump \ docker exec postgres pg_dump \
-U "$DB_USER" \ -U "$DB_USER" \
--format=plain \ --format=plain \
"$STAGING_DB" | gzip > "$BACKUP_FILE" "$STAGING_DB" | gzip > "$BACKUP_FILE"
@@ -1061,7 +1031,7 @@ backup_staging_db() {
truncate_staging_tables() { truncate_staging_tables() {
log_info "Truncating staging database tables..." log_info "Truncating staging database tables..."
docker exec -i igny8_postgres psql \ docker exec -i postgres psql \
-U "$DB_USER" \ -U "$DB_USER" \
-d "$STAGING_DB" << 'EOF' -d "$STAGING_DB" << 'EOF'
-- Get list of all tables -- Get list of all tables
@@ -1088,7 +1058,7 @@ dump_production_data() {
DUMP_FILE="/tmp/igny8_prod_dump_${TIMESTAMP}.sql" DUMP_FILE="/tmp/igny8_prod_dump_${TIMESTAMP}.sql"
docker exec igny8_postgres pg_dump \ docker exec postgres pg_dump \
-U "$DB_USER" \ -U "$DB_USER" \
--format=plain \ --format=plain \
"$PROD_DB" > "$DUMP_FILE" "$PROD_DB" > "$DUMP_FILE"
@@ -1102,7 +1072,7 @@ restore_to_staging() {
log_info "Restoring data to staging database..." log_info "Restoring data to staging database..."
cat "$DUMP_FILE" | docker exec -i igny8_postgres psql \ cat "$DUMP_FILE" | docker exec -i postgres psql \
-U "$DB_USER" \ -U "$DB_USER" \
-d "$STAGING_DB" \ -d "$STAGING_DB" \
--quiet --quiet
@@ -1113,30 +1083,30 @@ restore_to_staging() {
handle_sensitive_data() { handle_sensitive_data() {
log_info "Anonymizing/resetting sensitive data in staging..." log_info "Anonymizing/resetting sensitive data in staging..."
docker exec -i ighty8_postgres psql \ docker exec -i postgres psql \
-U "$DB_USER" \ -U "$DB_USER" \
-d "$STAGING_DB" << 'EOF' -d "$STAGING_DB" << 'EOF'
-- Reset payment information -- Reset/anonymize sensitive data in staging using ACTUAL table names
UPDATE billing_paymentmethod SET token = NULL WHERE token IS NOT NULL; -- (All tables prefixed with igny8_ — see 00C for full table list)
-- Reset API tokens -- Reset user passwords for non-staff users (set to a known staging password hash)
UPDATE api_token SET token = 'staging-token-' || id WHERE 1=1; UPDATE igny8_users SET password = 'pbkdf2_sha256$600000$stagingsalt$hashedvalue' WHERE is_staff = false;
-- Reset user passwords (set to default) -- Clear payment tokens (integration keys are in DB, not env vars)
UPDATE auth_user SET password = 'pbkdf2_sha256$600000$abcdefg$hashed' WHERE is_staff = false; -- Integration settings are in igny8_integration_settings and igny8_integration_providers
-- Do NOT delete these — just note they need to be updated to sandbox keys post-sync
-- Reset email addresses for non-admin users (optional - for testing) -- Clear webhook event records (contain real payment data)
-- UPDATE auth_user SET email = CONCAT(username, '@staging-test.local') WHERE is_staff = false; DELETE FROM igny8_webhook_events;
-- Clear sensitive logs
DELETE FROM audit_log WHERE action_type IN ('payment', 'user_data_export');
-- Clear transient data -- Clear transient data
DELETE FROM celery_taskmeta;
DELETE FROM django_session; DELETE FROM django_session;
-- Reset any third-party API keys to staging versions -- Clear AI task logs (optional — may contain API call details)
UPDATE integration_apikey SET secret = 'sk_staging_' || id WHERE 1=1; -- DELETE FROM igny8_ai_task_logs;
-- NOTE: After sync, manually update igny8_integration_settings to use sandbox API keys
-- for openai, stripe, paypal, runware, resend providers
EOF EOF
log_success "Sensitive data handled" log_success "Sensitive data handled"
@@ -1145,7 +1115,7 @@ EOF
sync_redis_cache() { sync_redis_cache() {
log_info "Clearing staging Redis cache (DB 1)..." log_info "Clearing staging Redis cache (DB 1)..."
docker exec igny8_redis redis-cli -n 1 FLUSHDB docker exec redis redis-cli -n 1 FLUSHDB
log_success "Staging Redis cache cleared" log_success "Staging Redis cache cleared"
} }
@@ -1238,7 +1208,7 @@ docker-compose -f docker-compose.staging.yml -p igny8-staging ps
docker-compose -f docker-compose.staging.yml -p igny8-staging logs -f docker-compose -f docker-compose.staging.yml -p igny8-staging logs -f
# Test API endpoint # Test API endpoint
curl -v https://staging-api.igny8.com/health/ curl -v https://staging-api.igny8.com/api/v1/system/status/
# Test frontend # Test frontend
curl -v https://staging.igny8.com curl -v https://staging.igny8.com
@@ -1279,7 +1249,7 @@ curl -v https://staging.igny8.com
- [ ] Frontend loads at `https://staging.igny8.com` without SSL errors - [ ] Frontend loads at `https://staging.igny8.com` without SSL errors
- [ ] API accessible at `https://staging-api.igny8.com` with proper CORS headers - [ ] API accessible at `https://staging-api.igny8.com` with proper CORS headers
- [ ] Marketing site loads at `https://staging-marketing.igny8.com` - [ ] Marketing site loads at `https://staging-marketing.igny8.com`
- [ ] Health check endpoint returns 200 at `/health/` - [ ] Health check endpoint returns 200 at `/api/v1/system/status/`
### 5.4 Data Synchronization Acceptance ### 5.4 Data Synchronization Acceptance
@@ -1365,14 +1335,14 @@ Tasks:
# In project root # In project root
docker-compose ps # Verify production running docker-compose ps # Verify production running
docker network inspect igny8_net # Verify network docker network inspect igny8_net # Verify network
docker exec igny8_postgres pg_isready -U postgres # Verify PostgreSQL docker exec postgres pg_isready -U postgres # Verify PostgreSQL
docker exec igny8_redis redis-cli ping # Verify Redis docker exec redis redis-cli ping # Verify Redis
``` ```
**Step 2: Create Staging Database** **Step 2: Create Staging Database**
```bash ```bash
docker exec -i igny8_postgres psql -U postgres -d postgres << 'EOF' docker exec -i postgres psql -U postgres -d postgres << 'EOF'
CREATE DATABASE igny8_staging_db CREATE DATABASE igny8_staging_db
WITH OWNER igny8_user WITH OWNER igny8_user
ENCODING 'UTF8' ENCODING 'UTF8'
@@ -1388,7 +1358,7 @@ EOF
```bash ```bash
# Edit /data/caddy/Caddyfile and append staging routes # Edit /data/caddy/Caddyfile and append staging routes
# Reload: docker exec igny8_caddy caddy reload --config /etc/caddy/Caddyfile # Reload: docker exec caddy caddy reload --config /etc/caddy/Caddyfile
``` ```
**Step 4: Build Images** **Step 4: Build Images**
@@ -1414,7 +1384,7 @@ cd /path/to/marketing && docker build -t igny8-marketing-dev:staging .
docker-compose -f docker-compose.staging.yml -p igny8-staging ps docker-compose -f docker-compose.staging.yml -p igny8-staging ps
# Check health # Check health
curl https://staging-api.igny8.com/health/ curl https://staging-api.igny8.com/api/v1/system/status/
# Check logs # Check logs
docker-compose -f docker-compose.staging.yml -p igny8-staging logs -f igny8_staging_backend docker-compose -f docker-compose.staging.yml -p igny8-staging logs -f igny8_staging_backend
@@ -1453,12 +1423,12 @@ docker-compose -f docker-compose.staging.yml -p igny8-staging logs igny8_staging
**Database connection failing:** **Database connection failing:**
```bash ```bash
docker exec igny8_postgres psql -U igny8_user -d igny8_staging_db -c "SELECT 1" docker exec postgres psql -U igny8 -d igny8_staging_db -c "SELECT 1"
``` ```
**Redis connection failing:** **Redis connection failing:**
```bash ```bash
docker exec igny8_redis redis-cli -n 1 ping docker exec redis redis-cli -n 1 ping
``` ```
**DNS not resolving:** **DNS not resolving:**
@@ -1469,8 +1439,8 @@ dig +short staging.igny8.com
**Caddy route not working:** **Caddy route not working:**
```bash ```bash
docker exec igny8_caddy caddy list-config docker exec caddy caddy list-config
docker exec igny8_caddy caddy reload --config /etc/caddy/Caddyfile -v docker exec caddy caddy reload --config /etc/caddy/Caddyfile -v
``` ```
**Restart entire staging environment:** **Restart entire staging environment:**
@@ -1481,7 +1451,7 @@ docker-compose -f docker-compose.staging.yml -p igny8-staging down
**Reset staging database:** **Reset staging database:**
```bash ```bash
docker exec igny8_postgres dropdb -U igny8_user igny8_staging_db docker exec postgres dropdb -U igny8 igny8_staging_db
./deploy-staging.sh # Recreates and migrations ./deploy-staging.sh # Recreates and migrations
``` ```
@@ -1489,12 +1459,9 @@ docker exec igny8_postgres dropdb -U igny8_user igny8_staging_db
## 7. Related Documentation ## 7. Related Documentation
- **[00B Infrastructure Setup](00B-infrastructure-setup.md):** NEW VPS provisioning, Docker, PostgreSQL 18, Redis 8, Caddy 2.11 configuration - **[00B Infrastructure Setup](00B-infrastructure-setup.md):** NEW VPS provisioning, Docker, PostgreSQL, Redis, Caddy configuration. Contains aspirational version targets; current versions verified from codebase (Python 3.11, Django >=5.2.7, Node 18, etc.)
- **Version Matrix (in 00B):** SINGLE SOURCE OF TRUTH for all component versions (PostgreSQL 18, Redis 8, Caddy 2.11, Python 3.14, Node 24 LTS, Django 6.0, Vite 8, Gunicorn 25, Celery 5.6, etc.) - **[00C Production Migration](00C-production-migration.md):** 3-stage migration flow (Deploy & Test, DNS Flip, Cloudflare Onboarding). DNS Reference: Staging setup coordinates with 00C stages to determine active DNS provider.
- Staging environment on NEW VPS uses identical versions - **Codebase files:** `docker-compose.staging.yml` (actual staging compose), `docker-compose.app.yml` (production compose), `backend/requirements.txt` (Python deps), `frontend/package.json` (JS deps).
- **[00C Production Migration](00C-production-migration.md):** 3-stage migration flow (DNS Preparation, DNS Flip, Cloudflare Onboarding), production database setup and initial deployment
- **DNS Reference:** Staging setup coordinates with 00C Stage 1/2/3 to determine active DNS provider and domain naming (staging may use test variants during migration)
- **Prerequisite:** 00B must be complete to provision the NEW VPS where staging runs. 00C determines which DNS provider is active for staging domain records.
--- ---

57
v2/V2-Execution-Docs/00E-legacy-cleanup.md
View File

@@ -3,8 +3,9 @@
**Status:** Pre-Implementation **Status:** Pre-Implementation
**Phase:** Phase 0 - Foundation & Infrastructure **Phase:** Phase 0 - Foundation & Infrastructure
**Document ID:** 00E **Document ID:** 00E
**Version:** 1.0 **Version:** 1.1
**Created:** 2026-03-23 **Created:** 2026-03-23
**Source of Truth:** Codebase at `/data/app/igny8/`
--- ---
@@ -322,8 +323,8 @@ ssh user@new-vps-ip
docker ps -a | grep -v Exit docker ps -a | grep -v Exit
# Verify application health endpoints # Verify application health endpoints
curl -v https://app.igny8.local/health curl -v https://app.igny8.com/api/v1/system/status/
curl -v https://api.igny8.local/status curl -v https://api.igny8.com/api/v1/system/status/
# Check recent logs for errors # Check recent logs for errors
docker logs --tail 100 [service-name] | grep -i error docker logs --tail 100 [service-name] | grep -i error
@@ -366,22 +367,22 @@ OLD_VPS_IP="x.x.x.x"
NEW_VPS_IP="y.y.y.y" NEW_VPS_IP="y.y.y.y"
# Check all production DNS records for organization # Check all production DNS records for organization
nslookup igny8.local nslookup igny8.com
nslookup api.igny8.local nslookup api.igny8.com
nslookup app.igny8.local nslookup app.igny8.com
nslookup git.igny8.local # Should NOT resolve to old VPS nslookup git.igny8.com # Should NOT resolve to old VPS
# Use dig for more detailed DNS information # Use dig for more detailed DNS information
dig igny8.local +short dig igny8.com +short
dig @8.8.8.8 igny8.local +short # Check public DNS dig @8.8.8.8 igny8.com +short # Check public DNS
# Search DNS for any remaining old VPS references # Search DNS for any remaining old VPS references
getent hosts | grep $OLD_VPS_IP getent hosts | grep $OLD_VPS_IP
# Verify all subdomains point to new VPS # Verify all subdomains point to new VPS
for domain in api app git cdn mail; do for domain in api app git cdn mail; do
echo "Checking $domain.igny8.local..." echo "Checking $domain.igny8.com..."
dig $domain.igny8.local +short dig $domain.igny8.com +short
done done
# IMPORTANT: Identify test DNS records created during 00C validation that must be removed # IMPORTANT: Identify test DNS records created during 00C validation that must be removed
@@ -562,19 +563,19 @@ grep "Accepted" /var/log/auth.log | tail -20
**Commands:** **Commands:**
```bash ```bash
# Repeat DNS verification from Step 1.4 # Repeat DNS verification from Step 1.4
nslookup igny8.local nslookup igny8.com
dig api.igny8.local +short dig api.igny8.com +short
dig app.igny8.local +short dig app.igny8.com +short
# Check for any CNAME chains # Check for any CNAME chains
dig igny8.local CNAME dig igny8.com CNAME
# Verify mail records don't point to old VPS # Verify mail records don't point to old VPS
dig igny8.local MX dig igny8.com MX
dig igny8.local NS dig igny8.com NS
# Use external DNS checker # Use external DNS checker
curl "https://dns.google/resolve?name=igny8.local&type=A" | jq . curl "https://dns.google/resolve?name=igny8.com&type=A" | jq .
# Verify test DNS records still exist (to be removed in Step 3.4) # Verify test DNS records still exist (to be removed in Step 3.4)
nslookup test-app.igny8.com nslookup test-app.igny8.com
@@ -877,8 +878,8 @@ df -h
ssh user@new-vps-ip ssh user@new-vps-ip
# Run smoke tests for all critical services # Run smoke tests for all critical services
curl -v https://api.igny8.local/health curl -v https://api.igny8.com/api/v1/system/status/
curl -v https://app.igny8.local/ curl -v https://app.igny8.com/
# Run database operations # Run database operations
docker exec [app-container] /app/bin/test-db-connection docker exec [app-container] /app/bin/test-db-connection
@@ -1211,8 +1212,8 @@ echo "Checking new VPS health..."
ssh user@$VPS_IP "docker ps -a" | grep -E "Up|Exited" ssh user@$VPS_IP "docker ps -a" | grep -E "Up|Exited"
# Check endpoints # Check endpoints
curl -s https://api.igny8.local/health | jq . curl -s https://api.igny8.com/api/v1/system/status/ | jq .
curl -s https://app.igny8.local/ > /dev/null && echo "App endpoint OK" curl -s https://app.igny8.com/ > /dev/null && echo "App endpoint OK"
# Check resources # Check resources
ssh user@$VPS_IP "free -h | awk 'NR==2'" ssh user@$VPS_IP "free -h | awk 'NR==2'"
@@ -1238,7 +1239,7 @@ NEW_IP=$2
echo "Verifying DNS records..." echo "Verifying DNS records..."
domains=("igny8.local" "api.igny8.local" "app.igny8.local" "git.igny8.local") domains=("igny8.com" "api.igny8.com" "app.igny8.com" "git.igny8.com")
for domain in "${domains[@]}"; do for domain in "${domains[@]}"; do
current_ip=$(dig +short $domain @8.8.8.8 | head -1) current_ip=$(dig +short $domain @8.8.8.8 | head -1)
@@ -1486,11 +1487,11 @@ echo "Running smoke tests on new VPS..."
# Test APIs # Test APIs
echo "Testing API health..." echo "Testing API health..."
curl -s https://api.igny8.local/health | jq . || echo "FAILED" curl -s https://api.igny8.com/api/v1/system/status/ | jq . || echo "FAILED"
# Test app # Test app
echo "Testing web app..." echo "Testing web app..."
curl -s -o /dev/null -w "%{http_code}" https://app.igny8.local/ || echo "FAILED" curl -s -o /dev/null -w "%{http_code}" https://app.igny8.com/ || echo "FAILED"
# Test database # Test database
echo "Testing database..." echo "Testing database..."
@@ -1662,7 +1663,7 @@ git push origin main
**Diagnosis:** **Diagnosis:**
```bash ```bash
dig igny8.local +short dig igny8.com +short
# Returns: x.x.x.x (old VPS IP) # Returns: x.x.x.x (old VPS IP)
``` ```
@@ -1671,7 +1672,7 @@ dig igny8.local +short
2. Verify TTL has expired (may need to wait 24+ hours) 2. Verify TTL has expired (may need to wait 24+ hours)
3. Manually update DNS records if needed 3. Manually update DNS records if needed
4. Flush local DNS cache: `sudo systemctl restart systemd-resolved` 4. Flush local DNS cache: `sudo systemctl restart systemd-resolved`
5. Re-verify from external DNS: `dig @8.8.8.8 igny8.local` 5. Re-verify from external DNS: `dig @8.8.8.8 igny8.com`
--- ---
@@ -1718,7 +1719,7 @@ ssh user@$OLD_VPS_IP "docker top gitea"
**Diagnosis:** **Diagnosis:**
```bash ```bash
curl https://api.igny8.local/health curl https://api.igny8.com/api/v1/system/status/
# Returns: 500 Internal Server Error # Returns: 500 Internal Server Error
``` ```

61
v2/V2-Execution-Docs/00F-self-hosted-ai-infra.md
View File

@@ -1,10 +1,11 @@
# IGNY8 Phase 0: Self-Hosted AI Infrastructure (00F) # IGNY8 Phase 0: Self-Hosted AI Infrastructure (00F)
**Status:** Ready for Implementation **Status:** Ready for Implementation
**Version:** 1.1
**Priority:** High (cost savings critical for unit economics) **Priority:** High (cost savings critical for unit economics)
**Duration:** 5-7 days **Duration:** 5-7 days
**Dependencies:** 00B (VPS provisioning) must be complete first **Dependencies:** 00B (VPS provisioning) must be complete first
**Version Reference:** See [00B Version Matrix](./00B-vps-provisioning.md) for authoritative version information **Source of Truth:** Codebase at `/data/app/igny8/`
**Cost:** ~$200/month GPU rental + $0 software (open source) **Cost:** ~$200/month GPU rental + $0 software (open source)
--- ---
@@ -12,14 +13,12 @@
## 1. Current State ## 1. Current State
### Existing AI Integration ### Existing AI Integration
- **External providers:** OpenAI (GPT-4, GPT-3.5), Anthropic (Claude), Runware (image gen), Bria (image gen) - **External providers (verified from `IntegrationProvider` model):** OpenAI (GPT-4, GPT-3.5), Anthropic (Claude), Runware (image gen)
- **Storage:** API keys stored in `GlobalIntegrationSettings` / `IntegrationProvider` models in database - **Storage:** API keys stored in `IntegrationProvider` model (table: `igny8_integration_providers`) with per-account overrides in `IntegrationSettings` (table: `igny8_integration_settings`). Global defaults in `GlobalIntegrationSettings`.
- **Provider types in codebase:** `ai`, `payment`, `email`, `storage` (from `PROVIDER_TYPE_CHOICES`)
- **Existing provider_ids:** `openai`, `runware`, `stripe`, `paypal`, `resend`
- **Architecture:** Multi-provider AI engine with model selection capability - **Architecture:** Multi-provider AI engine with model selection capability
- **Current usage:** - **Current AI functions:** `auto_cluster`, `generate_ideas`, `generate_content`, `generate_images`, `generate_image_prompts`, `optimize_content`, `generate_site_structure`
- Content generation: articles, product descriptions, blog posts
- Image generation: product images, covers, social media graphics
- Keyword research and SEO optimization
- Content enhancement and rewriting
- **Async handling:** Celery workers process long-running AI tasks - **Async handling:** Celery workers process long-running AI tasks
- **Cost impact:** External APIs constitute 15-30% of monthly operational costs - **Cost impact:** External APIs constitute 15-30% of monthly operational costs
@@ -125,31 +124,35 @@
## 3. Data Models / APIs ## 3. Data Models / APIs
### Database Models (No Schema Changes Required) ### Database Models (Minimal Schema Changes)
Use existing `IntegrationProvider` and `GlobalIntegrationSettings` models: Use existing `IntegrationProvider` model — add a new row with `provider_type='ai'`:
```python ```python
# In GlobalIntegrationSettings # New IntegrationProvider row (NO new provider_type needed)
INTEGRATION_PROVIDER_SELF_HOSTED = "self_hosted_ai" # provider_type='ai' already exists in PROVIDER_TYPE_CHOICES
# Settings structure (stored as JSON) # Create via admin or migration:
{ IntegrationProvider.objects.create(
"provider": "self_hosted_ai", provider_id='self_hosted_ai',
"name": "Self-Hosted AI (LiteLLM)", display_name='Self-Hosted AI (LiteLLM)',
"base_url": "http://localhost:8000", provider_type='ai',
"api_key": "not_used", # LiteLLM doesn't require auth (internal) api_key='', # LiteLLM doesn't require auth (internal)
"enabled": True, api_endpoint='http://localhost:8000',
"priority": 10, # Try self-hosted first is_active=True,
"models": { is_sandbox=False,
"text_generation": "qwen3:32b", config={
"text_generation_fast": "qwen3:8b", "priority": 10, # Try self-hosted first
"image_generation": "flux.1-dev", "models": {
"image_generation_fast": "sdxl-lightning" "text_generation": "qwen3:32b",
}, "text_generation_fast": "qwen3:8b",
"timeout": 300, # 5 minute timeout for slow models "image_generation": "flux.1-dev",
"fallback_to": "openai" # Fallback provider if self-hosted fails "image_generation_fast": "sdxl-lightning"
} },
"timeout": 300, # 5 minute timeout for slow models
"fallback_to": "openai" # Fallback provider if self-hosted fails
}
)
``` ```
### LiteLLM API Endpoints ### LiteLLM API Endpoints

121
v2/V2-Execution-Docs/01A-sag-data-foundation.md
View File

@@ -1,10 +1,11 @@
# IGNY8 Phase 1: SAG Data Foundation (01A) # IGNY8 Phase 1: SAG Data Foundation (01A)
## Core Data Models & CRUD APIs ## Core Data Models & CRUD APIs
**Document Version:** 1.0 **Document Version:** 1.1
**Date:** 2026-03-23 **Date:** 2026-03-23
**Phase:** IGNY8 Phase 1 — SAG Data Foundation **Phase:** IGNY8 Phase 1 — SAG Data Foundation
**Status:** Build Ready **Status:** Build Ready
**Source of Truth:** Codebase at `/data/app/igny8/`
**Audience:** Claude Code, Backend Developers, Architects **Audience:** Claude Code, Backend Developers, Architects
--- ---
@@ -12,8 +13,9 @@
## 1. CURRENT STATE ## 1. CURRENT STATE
### Existing IGNY8 Architecture ### Existing IGNY8 Architecture
- **Framework:** Django 5.1 + Django REST Framework 3.15 (upgrading to Django 6.0 on new VPS) - **Framework:** Django >=5.2.7 + Django REST Framework (from requirements.txt)
- **Database:** PostgreSQL 16 (upgrading to PostgreSQL 18) - **Database:** PostgreSQL (version set by infra stack)
- **Primary Keys:** BigAutoField (integer PKs — NOT UUIDs). `DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'`
- **Multi-Tenancy:** AccountContextMiddleware enforces tenant isolation - **Multi-Tenancy:** AccountContextMiddleware enforces tenant isolation
- All new models inherit from `AccountBaseModel` or `SiteSectorBaseModel` - All new models inherit from `AccountBaseModel` or `SiteSectorBaseModel`
- Automatic tenant filtering in querysets - Automatic tenant filtering in querysets
@@ -26,17 +28,24 @@
} }
``` ```
- **ViewSet Pattern:** `AccountModelViewSet` (filters by account) and `SiteSectorModelViewSet` - **ViewSet Pattern:** `AccountModelViewSet` (filters by account) and `SiteSectorModelViewSet`
- **Async Processing:** Celery 5.4 for background tasks - **Async Processing:** Celery >=5.3.0 for background tasks
- **Existing Models:** - **Existing Models (PLURAL class names, all use BigAutoField PKs):**
- Account (tenant root) - Account, Site (in `auth/models.py`) — `AccountBaseModel`
- Site (per-account, multi-site support) - Sector (in `auth/models.py`) — `AccountBaseModel`
- Sector (site-level category) - Clusters (in `business/planning/models.py`) — `SiteSectorBaseModel`
- Keyword, Cluster, ContentIdea, Task, Content, Image, SiteIntegration - Keywords (in `business/planning/models.py`) — `SiteSectorBaseModel`
- ContentIdeas (in `business/planning/models.py`) — `SiteSectorBaseModel`
- Tasks (in `business/content/models.py`) — `SiteSectorBaseModel`
- Content (in `business/content/models.py`) — `SiteSectorBaseModel`
- Images (in `business/content/models.py`) — `SiteSectorBaseModel`
- SiteIntegration (in `business/integration/models.py`) — `SiteSectorBaseModel`
- IntegrationProvider (in `modules/system/models.py`) — standalone
### Frontend Stack ### Frontend Stack
- React 19 + TypeScript 5 - React ^19.0.0 + TypeScript ~5.7.2
- Tailwind CSS 3 - Tailwind CSS ^4.0.8
- Zustand 4 (state management) - Zustand ^5.0.8 (state management)
- Vite ^6.1.0
### What Doesn't Exist ### What Doesn't Exist
- Attribute-based cluster formation system - Attribute-based cluster formation system
@@ -128,10 +137,11 @@ sag/
class SAGBlueprint(AccountBaseModel): class SAGBlueprint(AccountBaseModel):
""" """
Core blueprint model: versioned taxonomy & cluster plan for a site. Core blueprint model: versioned taxonomy & cluster plan for a site.
Inherits: id (UUID), account_id (FK), created_at, updated_at Inherits: id (BigAutoField, integer PK), account_id (FK), created_at, updated_at
Note: Uses BigAutoField per project convention (DEFAULT_AUTO_FIELD), NOT UUID.
""" """
site = models.ForeignKey( site = models.ForeignKey(
'sites.Site', 'igny8_core_auth.Site', # Actual app_label for Site model
on_delete=models.CASCADE, on_delete=models.CASCADE,
related_name='sag_blueprints' related_name='sag_blueprints'
) )
@@ -232,7 +242,7 @@ class SAGBlueprint(AccountBaseModel):
class SAGAttribute(AccountBaseModel): class SAGAttribute(AccountBaseModel):
""" """
Attribute/dimension within a blueprint. Attribute/dimension within a blueprint.
Inherits: id (UUID), account_id (FK), created_at, updated_at Inherits: id (BigAutoField, integer PK), account_id (FK), created_at, updated_at
""" """
blueprint = models.ForeignKey( blueprint = models.ForeignKey(
'sag.SAGBlueprint', 'sag.SAGBlueprint',
@@ -315,7 +325,11 @@ class SAGAttribute(AccountBaseModel):
class SAGCluster(AccountBaseModel): class SAGCluster(AccountBaseModel):
""" """
Cluster: hub page + supporting content organized around attribute intersection. Cluster: hub page + supporting content organized around attribute intersection.
Inherits: id (UUID), account_id (FK), created_at, updated_at Inherits: id (BigAutoField, integer PK), account_id (FK), created_at, updated_at
IMPORTANT: This model coexists with the existing `Clusters` model (in business/planning/models.py).
Existing Clusters are pure topic-keyword groups. SAGClusters add attribute-based dimensionality.
They are linked via an optional FK on the existing Clusters model.
""" """
blueprint = models.ForeignKey( blueprint = models.ForeignKey(
'sag.SAGBlueprint', 'sag.SAGBlueprint',
@@ -323,7 +337,7 @@ class SAGCluster(AccountBaseModel):
related_name='clusters' related_name='clusters'
) )
site = models.ForeignKey( site = models.ForeignKey(
'sites.Site', 'igny8_core_auth.Site', # Actual app_label for Site model
on_delete=models.CASCADE, on_delete=models.CASCADE,
related_name='sag_clusters' related_name='sag_clusters'
) )
@@ -461,8 +475,8 @@ class SectorAttributeTemplate(models.Model):
""" """
Reusable template for attributes and keywords by industry + sector. Reusable template for attributes and keywords by industry + sector.
NOT tied to Account (admin-only, shared across tenants). NOT tied to Account (admin-only, shared across tenants).
Uses BigAutoField PK per project convention (do NOT use UUID).
""" """
id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
industry = models.CharField( industry = models.CharField(
max_length=200, max_length=200,
@@ -533,11 +547,12 @@ class SectorAttributeTemplate(models.Model):
All modifications are **backward-compatible** and **nullable**. Existing records are unaffected. All modifications are **backward-compatible** and **nullable**. Existing records are unaffected.
#### **Site** (in `sites/models.py`) #### **Site** (in `auth/models.py`, app_label: `igny8_core_auth`)
```python ```python
class Site(AccountBaseModel): class Site(SoftDeletableModel, AccountBaseModel):
# ... existing fields ... # ... existing fields ...
# NEW: SAG integration (nullable, backward-compatible)
sag_blueprint = models.ForeignKey( sag_blueprint = models.ForeignKey(
'sag.SAGBlueprint', 'sag.SAGBlueprint',
on_delete=models.SET_NULL, on_delete=models.SET_NULL,
@@ -548,11 +563,13 @@ class Site(AccountBaseModel):
) )
``` ```
#### **Cluster** (in `modules/planner/models.py`) #### **Clusters** (in `business/planning/models.py`, app_label: `planner`)
```python ```python
class Cluster(AccountBaseModel): class Clusters(SoftDeletableModel, SiteSectorBaseModel):
# ... existing fields ... # ... existing fields: name, description, keywords_count, volume,
# mapped_pages, status(new/mapped), disabled ...
# NEW: SAG integration (nullable, backward-compatible)
sag_cluster = models.ForeignKey( sag_cluster = models.ForeignKey(
'sag.SAGCluster', 'sag.SAGCluster',
on_delete=models.SET_NULL, on_delete=models.SET_NULL,
@@ -570,11 +587,16 @@ class Cluster(AccountBaseModel):
) )
``` ```
#### **Task** (in `modules/writer/models.py`) #### **Tasks** (in `business/content/models.py`, app_label: `writer`)
```python ```python
class Task(AccountBaseModel): class Tasks(SoftDeletableModel, SiteSectorBaseModel):
# ... existing fields ... # ... existing fields: title, description, content_type, content_structure,
# keywords, word_count, status(queued/completed) ...
# NOTE: Already has `cluster = FK('planner.Clusters')` and
# `idea = FK('planner.ContentIdeas')` — these are NOT being replaced.
# The new sag_cluster FK is an ADDITIONAL link to the SAG layer.
# NEW: SAG integration (nullable, backward-compatible)
sag_cluster = models.ForeignKey( sag_cluster = models.ForeignKey(
'sag.SAGCluster', 'sag.SAGCluster',
on_delete=models.SET_NULL, on_delete=models.SET_NULL,
@@ -592,11 +614,14 @@ class Task(AccountBaseModel):
) )
``` ```
#### **Content** (in `modules/writer/models.py`) #### **Content** (in `business/content/models.py`, app_label: `writer`)
```python ```python
class Content(AccountBaseModel): class Content(SoftDeletableModel, SiteSectorBaseModel):
# ... existing fields ... # ... existing fields ...
# NOTE: Already has task FK (to writer.Tasks which has cluster FK).
# The new sag_cluster FK is an ADDITIONAL direct link to SAG layer.
# NEW: SAG integration (nullable, backward-compatible)
sag_cluster = models.ForeignKey( sag_cluster = models.ForeignKey(
'sag.SAGCluster', 'sag.SAGCluster',
on_delete=models.SET_NULL, on_delete=models.SET_NULL,
@@ -607,11 +632,12 @@ class Content(AccountBaseModel):
) )
``` ```
#### **ContentIdea** (in `modules/planner/models.py`) #### **ContentIdeas** (in `business/planning/models.py`, app_label: `planner`)
```python ```python
class ContentIdea(AccountBaseModel): class ContentIdeas(SoftDeletableModel, SiteSectorBaseModel):
# ... existing fields ... # ... existing fields ...
# NEW: SAG integration (nullable, backward-compatible)
sag_cluster = models.ForeignKey( sag_cluster = models.ForeignKey(
'sag.SAGCluster', 'sag.SAGCluster',
on_delete=models.SET_NULL, on_delete=models.SET_NULL,
@@ -847,7 +873,7 @@ class SAGBlueprintViewSet(AccountModelViewSet):
@action(detail=False, methods=['get']) @action(detail=False, methods=['get'])
def active_by_site(self, request): def active_by_site(self, request):
"""GET /api/v1/sag/blueprints/active_by_site/?site_id=<uuid>""" """GET /api/v1/sag/blueprints/active_by_site/?site_id=<int>"""
site_id = request.query_params.get('site_id') site_id = request.query_params.get('site_id')
if not site_id: if not site_id:
return Response({ return Response({
@@ -1027,7 +1053,7 @@ blueprints_router.register(
urlpatterns = [ urlpatterns = [
path('', include(router.urls)), path('', include(router.urls)),
path('blueprints/<uuid:blueprint_id>/', include(blueprints_router.urls)), path('blueprints/<int:blueprint_id>/', include(blueprints_router.urls)),
] ]
``` ```
@@ -1309,7 +1335,7 @@ def compute_blueprint_health(blueprint):
logger.info(f"Computed health for blueprint {blueprint.id}: {blueprint.sag_health_score}") logger.info(f"Computed health for blueprint {blueprint.id}: {blueprint.sag_health_score}")
return { return {
'blueprint_id': str(blueprint.id), 'blueprint_id': blueprint.id,
'health_score': blueprint.sag_health_score, 'health_score': blueprint.sag_health_score,
'attribute_count': attribute_count, 'attribute_count': attribute_count,
'cluster_count': cluster_count, 'cluster_count': cluster_count,
@@ -1580,14 +1606,13 @@ def plan_supporting_content(cluster, hub_page_title, num_articles=5):
2. **Define Models** 2. **Define Models**
- Implement `SAGBlueprint`, `SAGAttribute`, `SAGCluster`, `SectorAttributeTemplate` - Implement `SAGBlueprint`, `SAGAttribute`, `SAGCluster`, `SectorAttributeTemplate`
- Add 5 new nullable fields to existing models (Site, Cluster, Task, Content, ContentIdea) - Add 5 new nullable fields to existing models (Site, Clusters, Tasks, Content, ContentIdeas)
- Ensure all models inherit from correct base class (AccountBaseModel or base Model) - Ensure all models inherit from correct base class (AccountBaseModel or base Model)
3. **Create Migrations** 3. **Create Migrations**
- Run `makemigrations sag` - Run `makemigrations sag`
- Manually verify for circular imports or dependencies - Manually verify for circular imports or dependencies
- Create migration for modifications to existing models - Create migration for modifications to existing models (Clusters, Tasks, Content, ContentIdeas in their respective apps; Site in igny8_core_auth)
- All existing fields must remain untouched
4. **Implement Serializers** 4. **Implement Serializers**
- SAGBlueprintDetailSerializer (nested attributes & clusters) - SAGBlueprintDetailSerializer (nested attributes & clusters)
@@ -1638,7 +1663,7 @@ def plan_supporting_content(cluster, hub_page_title, num_articles=5):
### Data Model ### Data Model
- [ ] All 4 models created and migrated successfully - [ ] All 4 models created and migrated successfully
- [ ] All 5 existing models have nullable SAG fields - [ ] All 5 existing models have nullable SAG fields (Site, Clusters, Tasks, Content, ContentIdeas)
- [ ] Unique constraints enforced (blueprint version, attribute slugs, cluster slugs, template industry/sector) - [ ] Unique constraints enforced (blueprint version, attribute slugs, cluster slugs, template industry/sector)
- [ ] Foreign key cascades correct (blueprint → attributes/clusters) - [ ] Foreign key cascades correct (blueprint → attributes/clusters)
- [ ] All model methods and properties work as documented - [ ] All model methods and properties work as documented
@@ -1653,7 +1678,7 @@ def plan_supporting_content(cluster, hub_page_title, num_articles=5):
- [ ] POST /api/v1/sag/blueprints/{id}/archive/ (active → archived) - [ ] POST /api/v1/sag/blueprints/{id}/archive/ (active → archived)
- [ ] POST /api/v1/sag/blueprints/{id}/regenerate/ (create v+1) - [ ] POST /api/v1/sag/blueprints/{id}/regenerate/ (create v+1)
- [ ] POST /api/v1/sag/blueprints/{id}/health_check/ (compute score) - [ ] POST /api/v1/sag/blueprints/{id}/health_check/ (compute score)
- [ ] GET /api/v1/sag/blueprints/active_by_site/?site_id=<uuid> - [ ] GET /api/v1/sag/blueprints/active_by_site/?site_id=<int>
- [ ] GET/POST /api/v1/sag/blueprints/{blueprint_id}/attributes/ - [ ] GET/POST /api/v1/sag/blueprints/{blueprint_id}/attributes/
- [ ] GET/POST /api/v1/sag/blueprints/{blueprint_id}/clusters/ - [ ] GET/POST /api/v1/sag/blueprints/{blueprint_id}/clusters/
- [ ] GET/POST /api/v1/sag/sector-templates/ (admin-only) - [ ] GET/POST /api/v1/sag/sector-templates/ (admin-only)
@@ -1711,7 +1736,7 @@ This document contains **everything Claude Code needs to build the sag/ app**.
6. **Copy admin.py exactly** as-is 6. **Copy admin.py exactly** as-is
7. **Create service files** with code from Section 3.7 7. **Create service files** with code from Section 3.7
8. **Create AI function stubs** from Section 3.8 8. **Create AI function stubs** from Section 3.8
9. **Create migration** for existing model changes (Site, Cluster, Task, Content, ContentIdea) 9. **Create migration** for existing model changes (Site in `igny8_core_auth`, Clusters/ContentIdeas in `planner`, Tasks/Content in `writer`)
10. **Run migrations** on development database 10. **Run migrations** on development database
11. **Test endpoints** with Postman or curl 11. **Test endpoints** with Postman or curl
12. **Write unit & integration tests** matching patterns in existing test suite 12. **Write unit & integration tests** matching patterns in existing test suite
@@ -1724,7 +1749,7 @@ python manage.py startapp sag igny8_core/
# Makemigrations # Makemigrations
python manage.py makemigrations sag python manage.py makemigrations sag
python manage.py makemigrations # For existing model changes python manage.py makemigrations igny8_core_auth planner writer # For existing model changes
# Migrate # Migrate
python manage.py migrate sag python manage.py migrate sag
@@ -1753,7 +1778,7 @@ curl -X POST http://localhost:8000/api/v1/sag/blueprints/ \
-H "Authorization: Token <YOUR_TOKEN>" \ -H "Authorization: Token <YOUR_TOKEN>" \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{ -d '{
"site": "<site-uuid>", "site": 42,
"status": "draft", "status": "draft",
"source": "manual", "source": "manual",
"taxonomy_plan": {} "taxonomy_plan": {}
@@ -1800,7 +1825,7 @@ POST /api/v1/sag/blueprints/
Authorization: Token <token> Authorization: Token <token>
{ {
"site": "550e8400-e29b-41d4-a716-446655440000", "site": 42,
"status": "draft", "status": "draft",
"source": "manual", "source": "manual",
"taxonomy_plan": { "taxonomy_plan": {
@@ -1819,9 +1844,9 @@ Authorization: Token <token>
{ {
"success": true, "success": true,
"data": { "data": {
"id": "660e8400-e29b-41d4-a716-446655440001", "id": 1,
"site": "550e8400-e29b-41d4-a716-446655440000", "site": 42,
"account": "770e8400-e29b-41d4-a716-446655440002", "account": 7,
"version": 1, "version": 1,
"status": "draft", "status": "draft",
"source": "manual", "source": "manual",
@@ -1856,7 +1881,7 @@ Authorization: Token <token>
**Request:** **Request:**
```json ```json
POST /api/v1/sag/blueprints/660e8400-e29b-41d4-a716-446655440001/confirm/ POST /api/v1/sag/blueprints/1/confirm/
Authorization: Token <token> Authorization: Token <token>
``` ```
@@ -1865,8 +1890,8 @@ Authorization: Token <token>
{ {
"success": true, "success": true,
"data": { "data": {
"id": "660e8400-e29b-41d4-a716-446655440001", "id": 1,
"site": "550e8400-e29b-41d4-a716-446655440000", "site": 42,
"version": 1, "version": 1,
"status": "active", "status": "active",
"confirmed_at": "2026-03-23T10:05:00Z", "confirmed_at": "2026-03-23T10:05:00Z",

152
v2/V2-Execution-Docs/01B-sector-attribute-templates.md
View File

@@ -1,7 +1,10 @@
# 01B - Sector Attribute Templates # 01B - Sector Attribute Templates
**IGNY8 Phase 1: Service Layer & AI Functions** **IGNY8 Phase 1: Service Layer & AI Functions**
**Version:** 1.0 > **Version:** 1.1 (codebase-verified)
> **Source of Truth:** Codebase at `/data/app/igny8/backend/`
> **Last Verified:** 2025-07-14
**Date:** 2026-03-23 **Date:** 2026-03-23
**Status:** Build-Ready **Status:** Build-Ready
**Owner:** SAG Team **Owner:** SAG Team
@@ -11,7 +14,7 @@
## 1. Current State ## 1. Current State
### Model Foundation ### Model Foundation
- `SectorAttributeTemplate` model defined in `01A` (sag/models.py) - `SectorAttributeTemplate` model defined in `01A` (`igny8_core/sag/models.py`, new sag/ app)
- Schema includes: - Schema includes:
- `industry` (string) - `industry` (string)
- `sector` (string) - `sector` (string)
@@ -43,7 +46,7 @@ From SAG Niche Definition Process:
### 2.1 Service Layer: template_service.py ### 2.1 Service Layer: template_service.py
**Location:** `sag/services/template_service.py` **Location:** `igny8_core/sag/services/template_service.py`
#### Core Functions #### Core Functions
@@ -65,7 +68,7 @@ def get_or_generate_template(
- If missing: trigger AI generation via `discover_sector_attributes()` AI function - If missing: trigger AI generation via `discover_sector_attributes()` AI function
- Save generated template with `source='ai_generated'` - Save generated template with `source='ai_generated'`
- Return completed template - Return completed template
- Cache in Redis for 7 days (key: `sag:template:{industry}:{sector}`) - Cache in Redis for 7 days (key: `planner:template:{industry}:{sector}`)
```python ```python
def merge_templates( def merge_templates(
@@ -128,17 +131,23 @@ def prune_template(template: SectorAttributeTemplate) -> SectorAttributeTemplate
### 2.2 AI Function: DiscoverSectorAttributes ### 2.2 AI Function: DiscoverSectorAttributes
**Location:** `sag/ai_functions/attribute_discovery.py` **Location:** `igny8_core/ai/functions/discover_sector_attributes.py`
**Register Key:** `discover_sector_attributes` **Register Key:** `discover_sector_attributes`
#### Function Signature #### Function Signature
```python ```python
@ai_function(key='discover_sector_attributes') class DiscoverSectorAttributesFunction(BaseAIFunction):
async def discover_sector_attributes( """Discover sector attributes using AI."""
industry: str,
sector: str, def get_name(self) -> str:
site_type: str # 'ecommerce' | 'local_services' | 'saas' | 'content' return 'discover_sector_attributes'
) -> dict:
async def execute(
self,
industry: str,
sector: str,
site_type: str # 'ecommerce' | 'local_services' | 'saas' | 'content'
) -> dict:
``` ```
#### Input #### Input
@@ -247,7 +256,7 @@ OUTPUT: Valid JSON matching the schema above. Ensure all constraints are met.
- **Cache:** Template generation results cache for 30 days - **Cache:** Template generation results cache for 30 days
- **Validation:** Run `validate_template()` on output before returning - **Validation:** Run `validate_template()` on output before returning
- **Fallback:** If validation fails, retry with stricter prompt, max 2 retries - **Fallback:** If validation fails, retry with stricter prompt, max 2 retries
- **Error Handling:** Log to `sag_ai_generation` logger with full prompt/response - **Error Handling:** Log to `planner_ai_generation` logger with full prompt/response
--- ---
@@ -383,7 +392,7 @@ Step 4: Return Merged Template
#### Seeding Implementation #### Seeding Implementation
**Fixture File:** `sag/fixtures/sector_templates_seed.json` **Fixture File:** `igny8_core/sag/fixtures/sector_templates_seed.json`
```json ```json
{ {
"industry": "Pet Supplies", "industry": "Pet Supplies",
@@ -448,10 +457,17 @@ Applied in `prune_template()`:
### 3.1 SectorAttributeTemplate Model ### 3.1 SectorAttributeTemplate Model
**Location:** `sag/models.py` (from 01A, extended here) **Location:** `igny8_core/sag/models.py` (from 01A sag/ app, extended here)
```python ```python
from django.db import models
class SectorAttributeTemplate(models.Model): class SectorAttributeTemplate(models.Model):
"""
Admin-only template: NOT tied to Account or Site.
Uses BigAutoField PK per project convention (do NOT use UUID).
"""
# Identity # Identity
industry = models.CharField(max_length=255, db_index=True) industry = models.CharField(max_length=255, db_index=True)
sector = models.CharField(max_length=255, db_index=True) sector = models.CharField(max_length=255, db_index=True)
@@ -496,7 +512,7 @@ class SectorAttributeTemplate(models.Model):
# Relationships # Relationships
created_by = models.ForeignKey( created_by = models.ForeignKey(
User, settings.AUTH_USER_MODEL,
on_delete=models.SET_NULL, on_delete=models.SET_NULL,
null=True, null=True,
blank=True, blank=True,
@@ -504,12 +520,19 @@ class SectorAttributeTemplate(models.Model):
) )
class Meta: class Meta:
app_label = 'planner'
db_table = 'igny8_sector_attribute_templates'
unique_together = [('industry', 'sector')] unique_together = [('industry', 'sector')]
indexes = [ indexes = [
models.Index(fields=['industry', 'sector']), models.Index(fields=['industry', 'sector']),
models.Index(fields=['source', 'is_active']), models.Index(fields=['source', 'is_active']),
] ]
ordering = ['-updated_at'] ordering = ['-updated_at']
verbose_name = 'Sector Attribute Template'
verbose_name_plural = 'Sector Attribute Templates'
objects = SoftDeleteManager()
all_objects = models.Manager()
def __str__(self): def __str__(self):
return f"{self.industry} / {self.sector}" return f"{self.industry} / {self.sector}"
@@ -519,7 +542,7 @@ class SectorAttributeTemplate(models.Model):
### 3.2 REST API Endpoints ### 3.2 REST API Endpoints
**Base URL:** `/api/v1/sag/` **Base URL:** `/api/v1/planner/`
**Authentication:** Requires authentication (session or token) **Authentication:** Requires authentication (session or token)
#### GET /sector-templates/{industry}/{sector}/ #### GET /sector-templates/{industry}/{sector}/
@@ -527,7 +550,7 @@ class SectorAttributeTemplate(models.Model):
Request: Request:
``` ```
GET /api/v1/sag/sector-templates/Pet%20Supplies/Dog%20Accessories/ GET /api/v1/planner/sector-templates/Pet%20Supplies/Dog%20Accessories/
``` ```
Response (200 OK): Response (200 OK):
@@ -603,7 +626,7 @@ Response (400 Bad Request):
Request: Request:
``` ```
GET /api/v1/sag/sector-templates/?industry=Pet%20Supplies&source=ai_generated&is_active=true GET /api/v1/planner/sector-templates/?industry=Pet%20Supplies&source=ai_generated&is_active=true
``` ```
Query Parameters: Query Parameters:
@@ -618,7 +641,7 @@ Response (200 OK):
```json ```json
{ {
"count": 450, "count": 450,
"next": "/api/v1/sag/sector-templates/?limit=100&offset=100", "next": "/api/v1/planner/sector-templates/?limit=100&offset=100",
"previous": null, "previous": null,
"results": [ "results": [
{ ... template 1 ... }, { ... template 1 ... },
@@ -694,7 +717,7 @@ Response (202 Accepted - async):
```json ```json
{ {
"status": "generating", "status": "generating",
"task_id": "uuid-1234-5678", "task_id": "celery-task-1234-5678",
"industry": "Pet Supplies", "industry": "Pet Supplies",
"sector": "Dog Accessories", "sector": "Dog Accessories",
"message": "Template generation in progress. Check back in 30 seconds." "message": "Template generation in progress. Check back in 30 seconds."
@@ -746,14 +769,14 @@ Response (200 OK):
### 3.3 Service Layer: TemplateService Class ### 3.3 Service Layer: TemplateService Class
**Location:** `sag/services/template_service.py` **Location:** `igny8_core/sag/services/template_service.py`
```python ```python
from typing import Optional, List, Tuple, Dict, Any from typing import Optional, List, Tuple, Dict, Any
from django.core.cache import cache from django.core.cache import cache
from django.db.models import Q from django.db.models import Q
from sag.models import SectorAttributeTemplate from igny8_core.sag.models import SectorAttributeTemplate
from sag.ai_functions.attribute_discovery import discover_sector_attributes from igny8_core.ai.functions.discover_sector_attributes import DiscoverSectorAttributesFunction
class TemplateService: class TemplateService:
"""Service for managing sector attribute templates.""" """Service for managing sector attribute templates."""
@@ -772,7 +795,7 @@ class TemplateService:
sector: str sector: str
) -> Optional[SectorAttributeTemplate]: ) -> Optional[SectorAttributeTemplate]:
"""Fetch template from database or cache.""" """Fetch template from database or cache."""
cache_key = f"sag:template:{TemplateService.normalize_key(industry, sector)}" cache_key = f"planner:template:{TemplateService.normalize_key(industry, sector)}"
# Try cache first # Try cache first
cached = cache.get(cache_key) cached = cache.get(cache_key)
@@ -1110,13 +1133,13 @@ class TemplateService:
**Priority:** Critical **Priority:** Critical
**Owner:** Backend team **Owner:** Backend team
1. **Create `sag/services/template_service.py`** 1. **Create `igny8_core/sag/services/template_service.py`**
- Implement all 6 core functions - Implement all 6 core functions
- Add unit tests for each function - Add unit tests for each function
- Test edge cases (missing templates, invalid data) - Test edge cases (missing templates, invalid data)
- Acceptance: All functions pass unit tests, caching works - Acceptance: All functions pass unit tests, caching works
2. **Create `sag/ai_functions/attribute_discovery.py`** 2. **Create `igny8_core/ai/functions/discover_sector_attributes.py`**
- Register AI function with key `discover_sector_attributes` - Register AI function with key `discover_sector_attributes`
- Implement prompt strategy - Implement prompt strategy
- Add input validation - Add input validation
@@ -1135,20 +1158,20 @@ class TemplateService:
**Priority:** Critical **Priority:** Critical
**Owner:** Backend team **Owner:** Backend team
1. **Create `sag/views/template_views.py`** 1. **Create `igny8_core/sag/views.py`**
- TemplateListCreateView (GET, POST) - TemplateListCreateView (GET, POST)
- TemplateDetailView (GET, PUT, PATCH) - TemplateDetailView (GET, PUT, PATCH)
- TemplateGenerateView (POST) - TemplateGenerateView (POST)
- TemplateMergeView (POST) - TemplateMergeView (POST)
- All endpoints require authentication - All endpoints require authentication
2. **Create `sag/serializers/template_serializers.py`** 2. **Create `igny8_core/sag/serializers.py`**
- SectorAttributeTemplateSerializer - SectorAttributeTemplateSerializer
- Custom validation in serializer - Custom validation in serializer
- Nested serializers for attribute_framework, keyword_templates - Nested serializers for attribute_framework, keyword_templates
3. **Register URLs in `sag/urls.py`** 3. **Register URLs in `igny8_core/sag/urls.py`**
- Route all endpoints under `/api/v1/sag/sector-templates/` - Route all endpoints under `/api/v1/planner/sector-templates/`
- Use trailing slashes - Use trailing slashes
- Include proper HTTP method routing - Include proper HTTP method routing
@@ -1165,7 +1188,7 @@ class TemplateService:
**Priority:** High **Priority:** High
**Owner:** Data team **Owner:** Data team
1. **Create `sag/fixtures/sector_templates_seed.json`** 1. **Create `igny8_core/sag/fixtures/sector_templates_seed.json`**
- Template definitions for top 20 industries - Template definitions for top 20 industries
- Minimal valid data (5-8 attributes each) - Minimal valid data (5-8 attributes each)
- Should include: Pet Supplies, E-commerce Software, Digital Marketing, Healthcare, Real Estate - Should include: Pet Supplies, E-commerce Software, Digital Marketing, Healthcare, Real Estate
@@ -1280,8 +1303,8 @@ class TemplateService:
| Criterion | Target | Status | | Criterion | Target | Status |
|-----------|--------|--------| |-----------|--------|--------|
| Code coverage (sag/services/) | >85% | PENDING | | Code coverage (igny8_core/sag/services/) | >85% | PENDING |
| Code coverage (sag/ai_functions/) | >80% | PENDING | | Code coverage (igny8_core/ai/functions/) | >80% | PENDING |
| API tests coverage | 100% (all endpoints) | PENDING | | API tests coverage | 100% (all endpoints) | PENDING |
| All templates pass validate_template() | 100% | PENDING | | All templates pass validate_template() | 100% | PENDING |
| Documentation completeness | All endpoints documented | PENDING | | Documentation completeness | All endpoints documented | PENDING |
@@ -1295,55 +1318,54 @@ class TemplateService:
```bash ```bash
# Create service file # Create service file
touch sag/services/template_service.py touch igny8_core/sag/services/template_service.py
# Copy code from section 3.3 above # Copy code from section 3.3 above
# Create AI function file # Create AI function file
touch sag/ai_functions/attribute_discovery.py touch igny8_core/ai/functions/discover_sector_attributes.py
# Implement discover_sector_attributes() with prompt from section 2.2 # Implement DiscoverSectorAttributesFunction class with prompt from section 2.2
# Create tests # Create tests
touch sag/tests/test_template_service.py touch igny8_core/sag/tests/test_template_service.py
touch sag/tests/test_attribute_discovery.py touch igny8_core/sag/tests/test_attribute_discovery.py
# Run tests # Run tests
python manage.py test sag.tests.test_template_service --verbosity=2 python manage.py test igny8_core.modules.planner.tests.test_template_service --verbosity=2
python manage.py test sag.tests.test_attribute_discovery --verbosity=2 python manage.py test igny8_core.modules.planner.tests.test_attribute_discovery --verbosity=2
``` ```
### For Building the API Layer ### For Building the API Layer
```bash ```bash
# Create views and serializers # Create views and serializers
touch sag/views/template_views.py touch igny8_core/sag/views.py
touch sag/serializers/template_serializers.py touch igny8_core/sag/serializers.py
# Register URLs # Register URLs
# Edit sag/urls.py: # Edit igny8_core/sag/urls.py:
# from sag.views.template_views import * # from igny8_core.modules.planner.views.template_views import *
# urlpatterns += [ # urlpatterns += [
# path('sector-templates/', TemplateListCreateView.as_view(), ...), # path('sector-templates/', TemplateListCreateView.as_view(), ...),
# path('sector-templates/<int:pk>/', TemplateDetailView.as_view(), ...), # path('sector-templates/<int:id>/', TemplateDetailView.as_view(), ...),
# path('sector-templates/generate/', TemplateGenerateView.as_view(), ...), # path('sector-templates/generate/', TemplateGenerateView.as_view(), ...),
# path('sector-templates/merge/', TemplateMergeView.as_view(), ...), # path('sector-templates/merge/', TemplateMergeView.as_view(), ...),
# ] # ]
# Create API tests # Create API tests
touch sag/tests/test_template_api.py touch igny8_core/sag/tests/test_template_api.py
# Run API tests # Run API tests
python manage.py test sag.tests.test_template_api --verbosity=2 python manage.py test igny8_core.modules.planner.tests.test_template_api --verbosity=2
``` ```
### For Seeding Data ### For Seeding Data
```bash ```bash
# Create fixture file # Create fixture file
touch sag/fixtures/sector_templates_seed.json touch igny8_core/sag/fixtures/sector_templates_seed.json
# Create management command # Create management command
mkdir -p sag/management/commands touch igny8_core/management/commands/seed_sector_templates.py
touch sag/management/commands/seed_sector_templates.py
# Run seeding # Run seeding
python manage.py seed_sector_templates --industry "Pet Supplies" python manage.py seed_sector_templates --industry "Pet Supplies"
@@ -1357,14 +1379,14 @@ python manage.py validate_sector_templates
```bash ```bash
# Create integration test # Create integration test
touch sag/tests/test_integration_templates.py touch igny8_core/sag/tests/test_integration_templates.py
# Test with 01C (Cluster formation) # Test with 01C (Cluster formation)
# Test with 01D (Setup wizard) # Test with 01D (Setup wizard)
# Test with 01F (Existing site analysis) # Test with 01F (Existing site analysis)
# Run full integration test # Run full integration test
python manage.py test sag.tests.test_integration_templates --verbosity=2 python manage.py test igny8_core.modules.planner.tests.test_integration_templates --verbosity=2
``` ```
--- ---
@@ -1372,7 +1394,7 @@ python manage.py test sag.tests.test_integration_templates --verbosity=2
## Cross-Reference Index ## Cross-Reference Index
### Related Documents ### Related Documents
- **01A:** SectorAttributeTemplate model definition (`sag/models.py`) - **01A:** SectorAttributeTemplate model definition (`igny8_core/sag/models.py`)
- **01C:** Cluster Formation (uses keyword_templates) - **01C:** Cluster Formation (uses keyword_templates)
- **01D:** Setup Wizard (loads templates in Step 3a) - **01D:** Setup Wizard (loads templates in Step 3a)
- **01F:** Existing Site Analysis (validates against templates) - **01F:** Existing Site Analysis (validates against templates)
@@ -1381,16 +1403,16 @@ python manage.py test sag.tests.test_integration_templates --verbosity=2
### Key Files to Create ### Key Files to Create
``` ```
sag/services/template_service.py (450 lines) igny8_core/sag/services/template_service.py (450 lines)
sag/ai_functions/attribute_discovery.py (200 lines) igny8_core/ai/functions/discover_sector_attributes.py (200 lines)
sag/views/template_views.py (300 lines) igny8_core/sag/views.py (300 lines)
sag/serializers/template_serializers.py (150 lines) igny8_core/sag/serializers.py (150 lines)
sag/fixtures/sector_templates_seed.json (5000+ lines) igny8_core/sag/fixtures/sector_templates_seed.json (5000+ lines)
sag/management/commands/seed_sector_templates.py (100 lines) igny8_core/management/commands/seed_sector_templates.py (100 lines)
sag/tests/test_template_service.py (400 lines) igny8_core/sag/tests/test_template_service.py (400 lines)
sag/tests/test_attribute_discovery.py (300 lines) igny8_core/sag/tests/test_attribute_discovery.py (300 lines)
sag/tests/test_template_api.py (500 lines) igny8_core/sag/tests/test_template_api.py (500 lines)
sag/tests/test_integration_templates.py (300 lines) igny8_core/sag/tests/test_integration_templates.py (300 lines)
``` ```
### Total Estimated Effort ### Total Estimated Effort
@@ -1420,6 +1442,6 @@ All code is production-ready and integrates with related documents (01A, 01C, 01
--- ---
**Document Version:** 1.0 **Document Version:** 1.1
**Last Updated:** 2026-03-23 **Last Updated:** 2025-07-14
**Next Review:** Upon Phase 1 completion **Next Review:** Upon Phase 1 completion

49
v2/V2-Execution-Docs/01C-cluster-formation-keyword-engine.md
View File

@@ -1,6 +1,10 @@
# IGNY8 Phase 1: Cluster Formation & Keyword Engine (Doc 01C) # IGNY8 Phase 1: Cluster Formation & Keyword Engine (Doc 01C)
**Document Version:** 1.0 > **Version:** 1.1 (codebase-verified)
> **Source of Truth:** Codebase at `/data/app/igny8/backend/`
> **Last Verified:** 2025-07-14
**Document Version:** 1.1
**Date:** 2026-03-23 **Date:** 2026-03-23
**Phase:** Phase 1 - Foundation & Intelligence **Phase:** Phase 1 - Foundation & Intelligence
**Status:** Build Ready **Status:** Build Ready
@@ -48,7 +52,7 @@
{"name": "Health Condition", "values": ["Allergies", "Arthritis", "Obesity"]} {"name": "Health Condition", "values": ["Allergies", "Arthritis", "Obesity"]}
], ],
"sector_context": { "sector_context": {
"sector_id": str, "sector_id": int, # FK to igny8_core_auth.Sector (BigAutoField PK)
"site_type": "ecommerce|saas|blog|local_service", "site_type": "ecommerce|saas|blog|local_service",
"sector_name": str "sector_name": str
}, },
@@ -257,7 +261,7 @@ For each intersection, the AI must answer:
} }
], ],
"sector_context": { "sector_context": {
"sector_id": str, "sector_id": int, # FK to igny8_core_auth.Sector (BigAutoField PK)
"site_type": "ecommerce|saas|blog|local_service", "site_type": "ecommerce|saas|blog|local_service",
"site_intent": "sell|inform|book|download" "site_intent": "sell|inform|book|download"
}, },
@@ -503,7 +507,8 @@ keyword_templates = {
#### Input Contract #### Input Contract
```python ```python
assemble_blueprint( assemble_blueprint(
site: Website, # from 01A site: Site, # igny8_core_auth.Site (integer PK)
sector: Sector, # igny8_core_auth.Sector (integer PK)
attributes: List[Tuple[name, values]], # user-populated attributes: List[Tuple[name, values]], # user-populated
clusters: List[Dict], # from cluster_formation() clusters: List[Dict], # from cluster_formation()
keywords: Dict[cluster_id, List[Dict]] # from generate_keywords() keywords: Dict[cluster_id, List[Dict]] # from generate_keywords()
@@ -518,7 +523,7 @@ assemble_blueprint(
site=site, site=site,
status='draft', status='draft',
phase='phase_1_foundation', phase='phase_1_foundation',
sector_id=site.sector_id, sector=sector,
created_by=current_user, created_by=current_user,
metadata={ metadata={
'version': '1.0', 'version': '1.0',
@@ -844,7 +849,8 @@ END FUNCTION
#### SAGBlueprint (existing from 01A, extended) #### SAGBlueprint (existing from 01A, extended)
```python ```python
class SAGBlueprint(models.Model): # Inherits account, created_at, updated_at from AccountBaseModel
class SAGBlueprint(AccountBaseModel):
STATUS_CHOICES = ( STATUS_CHOICES = (
('draft', 'Draft'), ('draft', 'Draft'),
('cluster_formation_complete', 'Cluster Formation Complete'), ('cluster_formation_complete', 'Cluster Formation Complete'),
@@ -854,10 +860,10 @@ class SAGBlueprint(models.Model):
('published', 'Published'), ('published', 'Published'),
) )
site = models.ForeignKey(Website, on_delete=models.CASCADE) site = models.ForeignKey('igny8_core_auth.Site', on_delete=models.CASCADE)
status = models.CharField(max_length=50, choices=STATUS_CHOICES, default='draft') status = models.CharField(max_length=50, choices=STATUS_CHOICES, default='draft')
phase = models.CharField(max_length=50, default='phase_1_foundation') phase = models.CharField(max_length=50, default='phase_1_foundation')
sector_id = models.CharField(max_length=100) sector = models.ForeignKey('igny8_core_auth.Sector', on_delete=models.CASCADE)
# Denormalized JSON for fast access # Denormalized JSON for fast access
attributes_json = models.JSONField(default=dict, blank=True) attributes_json = models.JSONField(default=dict, blank=True)
@@ -865,9 +871,8 @@ class SAGBlueprint(models.Model):
taxonomy_plan = models.JSONField(default=dict, blank=True) taxonomy_plan = models.JSONField(default=dict, blank=True)
execution_priority = models.JSONField(default=dict, blank=True) execution_priority = models.JSONField(default=dict, blank=True)
created_by = models.ForeignKey(User, on_delete=models.SET_NULL, null=True) created_by = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.SET_NULL, null=True)
created_at = models.DateTimeField(auto_now_add=True) # created_at, updated_at inherited from AccountBaseModel
updated_at = models.DateTimeField(auto_now=True)
class Meta: class Meta:
db_table = 'sag_blueprint' db_table = 'sag_blueprint'
@@ -876,13 +881,14 @@ class SAGBlueprint(models.Model):
#### SAGAttribute (existing from 01A, no changes required) #### SAGAttribute (existing from 01A, no changes required)
```python ```python
class SAGAttribute(models.Model): # Inherits account, created_at, updated_at from AccountBaseModel
class SAGAttribute(AccountBaseModel):
blueprint = models.ForeignKey(SAGBlueprint, on_delete=models.CASCADE) blueprint = models.ForeignKey(SAGBlueprint, on_delete=models.CASCADE)
name = models.CharField(max_length=255) name = models.CharField(max_length=255)
values = models.JSONField() # array of strings values = models.JSONField() # array of strings
is_primary = models.BooleanField(default=False) is_primary = models.BooleanField(default=False)
source = models.CharField(max_length=50) # 'user_input', 'template', 'api' source = models.CharField(max_length=50) # 'user_input', 'template', 'api'
created_at = models.DateTimeField(auto_now_add=True) # created_at, updated_at inherited from AccountBaseModel
class Meta: class Meta:
db_table = 'sag_attribute' db_table = 'sag_attribute'
@@ -891,7 +897,8 @@ class SAGAttribute(models.Model):
#### SAGCluster (existing from 01A, extended) #### SAGCluster (existing from 01A, extended)
```python ```python
class SAGCluster(models.Model): # Inherits account, created_at, updated_at from AccountBaseModel
class SAGCluster(AccountBaseModel):
TYPE_CHOICES = ( TYPE_CHOICES = (
('product_category', 'Product/Service Category'), ('product_category', 'Product/Service Category'),
('condition_problem', 'Condition/Problem'), ('condition_problem', 'Condition/Problem'),
@@ -935,8 +942,7 @@ class SAGCluster(models.Model):
keyword_count = models.IntegerField(default=0) keyword_count = models.IntegerField(default=0)
status = models.CharField(max_length=50, choices=STATUS_CHOICES, default='draft') status = models.CharField(max_length=50, choices=STATUS_CHOICES, default='draft')
created_at = models.DateTimeField(auto_now_add=True) # created_at, updated_at inherited from AccountBaseModel
updated_at = models.DateTimeField(auto_now=True)
class Meta: class Meta:
db_table = 'sag_cluster' db_table = 'sag_cluster'
@@ -946,7 +952,8 @@ class SAGCluster(models.Model):
#### SAGKeyword (new) #### SAGKeyword (new)
```python ```python
class SAGKeyword(models.Model): # Inherits account, created_at, updated_at from AccountBaseModel
class SAGKeyword(AccountBaseModel):
INTENT_CHOICES = ( INTENT_CHOICES = (
('informational', 'Informational'), ('informational', 'Informational'),
('transactional', 'Transactional'), ('transactional', 'Transactional'),
@@ -987,9 +994,7 @@ class SAGKeyword(models.Model):
cpc = models.FloatField(null=True, blank=True) # if available from API cpc = models.FloatField(null=True, blank=True) # if available from API
competition = models.CharField(max_length=50, blank=True) # 'low', 'medium', 'high' competition = models.CharField(max_length=50, blank=True) # 'low', 'medium', 'high'
# created_at, updated_at inherited from AccountBaseModel
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
class Meta: class Meta:
db_table = 'sag_keyword' db_table = 'sag_keyword'
@@ -1542,7 +1547,7 @@ populated_attributes = [
] ]
sector_context = { sector_context = {
"sector_id": "pet_health", "sector_id": 1, # integer PK (BigAutoField)
"site_type": "ecommerce", "site_type": "ecommerce",
"sector_name": "Pet Health Products" "sector_name": "Pet Health Products"
} }
@@ -1563,7 +1568,7 @@ populated_attributes = [
] ]
sector_context = { sector_context = {
"sector_id": "vet_clinic", "sector_id": 2, # integer PK (BigAutoField)
"site_type": "local_service", "site_type": "local_service",
"sector_name": "Veterinary Clinic" "sector_name": "Veterinary Clinic"
} }

53
v2/V2-Execution-Docs/01D-setup-wizard-case2-new-site.md
View File

@@ -1,8 +1,12 @@
# IGNY8 Phase 1: Setup Wizard — Case 2 (New Site) # IGNY8 Phase 1: Setup Wizard — Case 2 (New Site)
## Document 01D: Build Specification ## Document 01D: Build Specification
> **Version:** 1.1 (codebase-verified)
> **Source of Truth:** Codebase at `/data/app/igny8/backend/`
> **Last Verified:** 2025-07-14
**Status**: Draft **Status**: Draft
**Version**: 1.0 **Version**: 1.1
**Date**: 2026-03-23 **Date**: 2026-03-23
**Phase**: Phase 1 — Foundation **Phase**: Phase 1 — Foundation
**Scope**: New site workflow with enhanced Site Structure step **Scope**: New site workflow with enhanced Site Structure step
@@ -144,8 +148,9 @@ Step 4 → Step 5 → Step 6
```python ```python
# Fields to emphasize for this wizard: # Fields to emphasize for this wizard:
class SAGBlueprint(models.Model): # SAGBlueprint inherits AccountBaseModel (provides account, created_by, etc.)
site_id = models.ForeignKey(Site) class SAGBlueprint(AccountBaseModel):
site = models.ForeignKey('igny8_core_auth.Site', on_delete=models.CASCADE)
status = models.CharField( status = models.CharField(
choices=['draft', 'active', 'archived'], choices=['draft', 'active', 'archived'],
default='draft' default='draft'
@@ -174,8 +179,9 @@ class SAGBlueprint(models.Model):
**Location**: Reference 01A (Attribute Definition) **Location**: Reference 01A (Attribute Definition)
```python ```python
class SAGAttribute(models.Model): # SAGAttribute inherits AccountBaseModel (provides account, created_by, etc.)
blueprint = models.ForeignKey(SAGBlueprint) class SAGAttribute(AccountBaseModel):
blueprint = models.ForeignKey('sag.SAGBlueprint', on_delete=models.CASCADE)
name = models.CharField() # e.g., "Target Area" name = models.CharField() # e.g., "Target Area"
description = models.TextField() description = models.TextField()
@@ -624,7 +630,7 @@ Be conservative: only map if connection is clear. Do not invent values not suppo
### Phase 2: Frontend Components (React) ### Phase 2: Frontend Components (React)
#### Step 2.1: Implement WizardStep3Container #### Step 2.1: Implement WizardStep3Container
- [ ] Create `frontend/src/components/wizard/WizardStep3Container.jsx` - [ ] Create `frontend/src/components/wizard/WizardStep3Container.tsx`
- [ ] Manage state for all sub-steps (3a–3f): - [ ] Manage state for all sub-steps (3a–3f):
- `currentSubstep` (enum: 'generate', 'review', 'business', 'populate', 'preview', 'confirm') - `currentSubstep` (enum: 'generate', 'review', 'business', 'populate', 'preview', 'confirm')
- `attributes` (from API) - `attributes` (from API)
@@ -644,7 +650,7 @@ Be conservative: only map if connection is clear. Do not invent values not suppo
--- ---
#### Step 2.2: Implement AttributeReviewPanel (Step 3b) #### Step 2.2: Implement AttributeReviewPanel (Step 3b)
- [ ] Create `frontend/src/components/wizard/AttributeReviewPanel.jsx` - [ ] Create `frontend/src/components/wizard/AttributeReviewPanel.tsx`
- [ ] Render attributes grouped by level: - [ ] Render attributes grouped by level:
- **Primary Attributes** section - **Primary Attributes** section
- **Secondary Attributes** section - **Secondary Attributes** section
@@ -672,7 +678,7 @@ Be conservative: only map if connection is clear. Do not invent values not suppo
--- ---
#### Step 2.3: Implement BusinessDetailsForm (Step 3c) #### Step 2.3: Implement BusinessDetailsForm (Step 3c)
- [ ] Create `frontend/src/components/wizard/BusinessDetailsForm.jsx` - [ ] Create `frontend/src/components/wizard/BusinessDetailsForm.tsx`
- [ ] Fields: - [ ] Fields:
- [ ] **Products** — textarea, accepts comma-separated or line-break list - [ ] **Products** — textarea, accepts comma-separated or line-break list
- [ ] **Services** — textarea, same format - [ ] **Services** — textarea, same format
@@ -697,7 +703,7 @@ Be conservative: only map if connection is clear. Do not invent values not suppo
--- ---
#### Step 2.4: Implement BlueprintPreviewPanel (Step 3e) #### Step 2.4: Implement BlueprintPreviewPanel (Step 3e)
- [ ] Create `frontend/src/components/wizard/BlueprintPreviewPanel.jsx` - [ ] Create `frontend/src/components/wizard/BlueprintPreviewPanel.tsx`
- [ ] Render tree view of clusters: - [ ] Render tree view of clusters:
- [ ] Cluster name (e.g., "Neck Massage Devices") - [ ] Cluster name (e.g., "Neck Massage Devices")
- [ ] Type badge (e.g., "Topic Hub") - [ ] Type badge (e.g., "Topic Hub")
@@ -757,7 +763,7 @@ Be conservative: only map if connection is clear. Do not invent values not suppo
- [ ] Detailed Mode: 3a → 3b → 3c → 3d → 3e → 3f → Step 4 - [ ] Detailed Mode: 3a → 3b → 3c → 3d → 3e → 3f → Step 4
- [ ] Step 4 → Step 5 (always) - [ ] Step 4 → Step 5 (always)
- [ ] Step 5 → Step 6 (always) - [ ] Step 5 → Step 6 (always)
- [ ] Implement state persistence (Redux or context): - [ ] Implement state persistence (Zustand store):
- [ ] Save wizard state to localStorage or session - [ ] Save wizard state to localStorage or session
- [ ] Allow user to resume if page refreshes - [ ] Allow user to resume if page refreshes
- [ ] Unit test: navigation logic for both modes - [ ] Unit test: navigation logic for both modes
@@ -905,7 +911,7 @@ Be conservative: only map if connection is clear. Do not invent values not suppo
### Functional Criteria ### Functional Criteria
#### 5.1: Step 3a — Generate Attributes #### 5.1: Step 3a — Generate Attributes
- [x] **AC-3a-1**: GET /api/v1/sag/wizard/generate-attributes/ returns attribute framework - [x] **AC-3a-1**: POST /api/v1/sag/wizard/generate-attributes/ returns attribute framework
- [ ] Response includes 4–8 attributes (depending on industry/sectors) - [ ] Response includes 4–8 attributes (depending on industry/sectors)
- [ ] Each attribute has name, level, suggested_values, description - [ ] Each attribute has name, level, suggested_values, description
- [ ] Attributes are organized by level (primary → secondary → tertiary) - [ ] Attributes are organized by level (primary → secondary → tertiary)
@@ -1074,7 +1080,7 @@ Be conservative: only map if connection is clear. Do not invent values not suppo
- [ ] User can navigate freely between steps (prev/next) - [ ] User can navigate freely between steps (prev/next)
- [x] **AC-5-3**: Mode selection persists - [x] **AC-5-3**: Mode selection persists
- [ ] Mode stored in session/Redux state - [ ] Mode stored in session/Zustand state
- [ ] Navigation logic respects mode throughout wizard - [ ] Navigation logic respects mode throughout wizard
--- ---
@@ -1188,7 +1194,7 @@ This section provides step-by-step instructions for Claude Code (or equivalent A
2. **Set Up Environment** 2. **Set Up Environment**
- Clone repository - Clone repository
- Install dependencies (backend: Django/DRF, frontend: React + Redux, WordPress: plugin SDK) - Install dependencies (backend: Django >=5.2.7/DRF, frontend: React 19 + Zustand + TypeScript ~5.7.2 + Vite ^6.1.0, WordPress: plugin SDK)
- Create feature branch: `feature/wizard-step-3-site-structure` - Create feature branch: `feature/wizard-step-3-site-structure`
- Ensure tests pass on main branch - Ensure tests pass on main branch
@@ -1271,7 +1277,7 @@ Location: backend/sag/api/views/wizard.py
``` ```
Location: frontend/src/components/wizard/ Location: frontend/src/components/wizard/
A) WizardStep3Container.jsx (2 hours) A) WizardStep3Container.tsx (2 hours)
- Create state object: - Create state object:
{ {
mode: 'quick' | 'detailed', mode: 'quick' | 'detailed',
@@ -1284,23 +1290,23 @@ A) WizardStep3Container.jsx (2 hours)
- Implement navigation logic (next, prev, skip) - Implement navigation logic (next, prev, skip)
- Implement conditional rendering of sub-steps based on mode - Implement conditional rendering of sub-steps based on mode
- Handle loading/error states - Handle loading/error states
- Connect to Redux (or context) for wizard state - Connect to Zustand store for wizard state
B) AttributeReviewPanel.jsx (1.5 hours) B) AttributeReviewPanel.tsx (1.5 hours)
- Render three sections: Primary, Secondary, Tertiary - Render three sections: Primary, Secondary, Tertiary
- For each attribute: toggle + values + edit/delete + reorder - For each attribute: toggle + values + edit/delete + reorder
- Implement inline edit modal for values - Implement inline edit modal for values
- Implement "+ Add Custom Attribute" form - Implement "+ Add Custom Attribute" form
- Show completeness status (ready/thin/empty) - Show completeness status (ready/thin/empty)
C) BusinessDetailsForm.jsx (1 hour) C) BusinessDetailsForm.tsx (1 hour)
- Five input fields: products, services, brands, locations, conditions - Five input fields: products, services, brands, locations, conditions
- Implement text parsing (comma-separated, line-break) - Implement text parsing (comma-separated, line-break)
- Show "x items detected" feedback - Show "x items detected" feedback
- Implement validation (at least one field, max 50 items) - Implement validation (at least one field, max 50 items)
- Pass data to parent state on change - Pass data to parent state on change
D) BlueprintPreviewPanel.jsx (1.5 hours) D) BlueprintPreviewPanel.tsx (1.5 hours)
- Render tree view of clusters - Render tree view of clusters
- Each cluster: name, type badge, keyword count, content plan count - Each cluster: name, type badge, keyword count, content plan count
- Expand/collapse per cluster - Expand/collapse per cluster
@@ -1321,7 +1327,7 @@ Tests:
#### Task 6: Integrate Wizard Navigation (2 hours) #### Task 6: Integrate Wizard Navigation (2 hours)
``` ```
Location: frontend/src/routes/wizard.js (or similar routing) Location: frontend/src/routes/wizard.tsx (or similar routing)
- Update router to include Step 3 routes - Update router to include Step 3 routes
- Implement navigation logic: - Implement navigation logic:
- Step 1 → Step 2 (always) - Step 1 → Step 2 (always)
@@ -1329,15 +1335,15 @@ Location: frontend/src/routes/wizard.js (or similar routing)
- Step 3a → Step 3b (Detailed) or Step 3e (Quick) - Step 3a → Step 3b (Detailed) or Step 3e (Quick)
- Step 3b → Step 3c, Step 3c → Step 3d, Step 3d → Step 3e - Step 3b → Step 3c, Step 3c → Step 3d, Step 3d → Step 3e
- Step 3e → Step 3f, Step 3f → Step 4 - Step 3e → Step 3f, Step 3f → Step 4
- Implement state persistence (Redux or localStorage) - Implement state persistence (Zustand store with localStorage persist)
- Test Quick Mode flow and Detailed Mode flow (E2E) - Test Quick Mode flow and Detailed Mode flow (E2E)
``` ```
#### Task 7: Update Step 1 (Welcome) (1 hour) #### Task 7: Update Step 1 (Welcome) (1 hour)
``` ```
Location: frontend/src/components/wizard/WizardStep1.jsx (or similar) Location: frontend/src/components/wizard/WizardStep1.tsx (or similar)
- Add mode selection UI (quick vs. detailed) - Add mode selection UI (quick vs. detailed)
- Store mode in wizard state (Redux/context) - Store mode in wizard state (Zustand store)
- Pass mode to WizardStep3Container - Pass mode to WizardStep3Container
- Test mode selection - Test mode selection
``` ```
@@ -1383,7 +1389,7 @@ Location: wordpress-plugin/igny8-blueprint-sync.php (or similar)
- Test data persistence and transitions - Test data persistence and transitions
- Frontend: E2E test both wizard flows - Frontend: E2E test both wizard flows
- Location: frontend/tests/e2e/wizard.test.js (Selenium/Cypress) - Location: frontend/tests/e2e/wizard.test.ts (Vitest/Playwright)
- Test Quick Mode: 10 min, full journey - Test Quick Mode: 10 min, full journey
- Test Detailed Mode: 20 min, full journey - Test Detailed Mode: 20 min, full journey
- Test error scenarios (invalid input, API failure) - Test error scenarios (invalid input, API failure)
@@ -1492,6 +1498,7 @@ Location: wordpress-plugin/igny8-blueprint-sync.php (or similar)
| Version | Date | Author | Change | | Version | Date | Author | Change |
|---------|------|--------|--------| |---------|------|--------|--------|
| 1.0 | 2026-03-23 | System | Initial draft | | 1.0 | 2026-03-23 | System | Initial draft |
| 1.1 | 2025-07-14 | Codebase Audit | Fixed: model inheritance (AccountBaseModel), FK app_labels, .jsx→.tsx, Redux→Zustand, GET→POST AC-3a-1, version refs |
--- ---

291
v2/V2-Execution-Docs/01E-blueprint-aware-pipeline.md
View File

@@ -1,4 +1,9 @@
# 01E: Blueprint-Aware Content Pipeline # 01E: Blueprint-Aware Content Pipeline
> **Version:** 1.1 (codebase-verified)
> **Source of Truth:** Codebase at `/data/app/igny8/backend/`
> **Last Verified:** 2025-07-14
**IGNY8 Phase 1: Content Automation with SAG Blueprint Enhancement** **IGNY8 Phase 1: Content Automation with SAG Blueprint Enhancement**
--- ---
@@ -150,15 +155,15 @@ ELSE:
2. `blueprint_context` structure: 2. `blueprint_context` structure:
```json ```json
{ {
"cluster_id": "uuid", "cluster_id": "integer",
"cluster_name": "string", "cluster_name": "string",
"cluster_type": "string (topical|product|service)", "cluster_type": "string (topical|product|service)",
"cluster_sector": "string", "cluster_sector": "string",
"hub_title": "string (cluster's main hub page title)", "hub_title": "string (cluster's main hub page title)",
"hub_url": "string (blueprint.site.domain/cluster_slug)", "hub_url": "string (blueprint.site.domain/cluster_slug)",
"cluster_attributes": ["list of attribute terms"], "cluster_attributes": ["list of attribute terms"],
"related_clusters": ["list of related cluster ids"], "related_clusters": ["list of related cluster integer ids"],
"cluster_products": ["list of product ids if product cluster"], "cluster_products": ["list of product integer ids if product cluster"],
"content_structure": "string (guide_tutorial|comparison|review|how_to|question|listicle)", "content_structure": "string (guide_tutorial|comparison|review|how_to|question|listicle)",
"content_type": "string (cluster_hub|blog_post|product_page|term_page|service_page)", "content_type": "string (cluster_hub|blog_post|product_page|term_page|service_page)",
"execution_phase": "integer (1-4)", "execution_phase": "integer (1-4)",
@@ -287,10 +292,13 @@ execution_priority = {
### Related Models (from 01A, 01C, 01D) ### Related Models (from 01A, 01C, 01D)
```python ```python
# sag/models.py — SAG Blueprint Structure # igny8_core/sag/models.py — SAG Blueprint Structure
# DEFAULT_AUTO_FIELD = BigAutoField (integer PKs)
class SAGBlueprint(models.Model): from igny8_core.auth.models import AccountBaseModel
site = ForeignKey(Site)
class SAGBlueprint(AccountBaseModel):
site = ForeignKey('igny8_core_auth.Site', on_delete=models.CASCADE)
name = CharField(max_length=255) name = CharField(max_length=255)
status = CharField(choices=['draft', 'active', 'archived']) status = CharField(choices=['draft', 'active', 'archived'])
created_at = DateTimeField(auto_now_add=True) created_at = DateTimeField(auto_now_add=True)
@@ -303,8 +311,8 @@ class SAGBlueprint(models.Model):
# Taxonomy mapping to WordPress custom taxonomies # Taxonomy mapping to WordPress custom taxonomies
wp_taxonomy_mapping = JSONField() # cluster_id → tax values wp_taxonomy_mapping = JSONField() # cluster_id → tax values
class SAGCluster(models.Model): class SAGCluster(AccountBaseModel):
blueprint = ForeignKey(SAGBlueprint) blueprint = ForeignKey('sag.SAGBlueprint', on_delete=models.CASCADE)
name = CharField(max_length=255) name = CharField(max_length=255)
cluster_type = CharField(choices=['topical', 'product', 'service']) cluster_type = CharField(choices=['topical', 'product', 'service'])
sector = CharField(max_length=255) sector = CharField(max_length=255)
@@ -314,69 +322,135 @@ class SAGCluster(models.Model):
updated_at = DateTimeField(auto_now=True) updated_at = DateTimeField(auto_now=True)
``` ```
### Pipeline Models (existing) ### Pipeline Models (existing — names are PLURAL per codebase convention)
```python ```python
# content/models.py — Content Pipeline # igny8_core/business/planning/models.py — Planning Pipeline (app_label: planner)
# DEFAULT_AUTO_FIELD = BigAutoField (integer PKs, NOT UUIDs)
class Keyword(models.Model): class Keywords(SoftDeletableModel, SiteSectorBaseModel):
site = ForeignKey(Site) """Site-specific keyword instances referencing global SeedKeywords."""
term = CharField(max_length=255) seed_keyword = ForeignKey(SeedKeyword, on_delete=models.CASCADE)
source = CharField(choices=['csv_import', 'seed_list', 'user', 'sag_blueprint']) volume_override = IntegerField(null=True, blank=True)
sag_cluster_id = UUIDField(null=True, blank=True) # NEW: links to blueprint cluster difficulty_override = IntegerField(null=True, blank=True)
attribute_values = JSONField(default=list, blank=True)
cluster = ForeignKey('Clusters', on_delete=models.SET_NULL, null=True, blank=True)
status = CharField(max_length=50, choices=[('new','New'),('mapped','Mapped')], default='new')
disabled = BooleanField(default=False)
# NEW: optional SAG cluster link
sag_cluster_id = IntegerField(null=True, blank=True) # Links to sag.SAGCluster PK
created_at = DateTimeField(auto_now_add=True) created_at = DateTimeField(auto_now_add=True)
class Meta:
app_label = 'planner'
class Cluster(models.Model): class Clusters(SoftDeletableModel, SiteSectorBaseModel):
site = ForeignKey(Site) """Keyword clusters — pure topic clusters."""
name = CharField(max_length=255) name = CharField(max_length=255, db_index=True)
keywords = JSONField(default=list) description = TextField(blank=True, null=True)
created_by = CharField(choices=['auto_cluster', 'sag_blueprint']) keywords_count = IntegerField(default=0)
volume = IntegerField(default=0)
mapped_pages = IntegerField(default=0)
status = CharField(max_length=50, choices=[('new','New'),('mapped','Mapped')], default='new')
disabled = BooleanField(default=False)
created_at = DateTimeField(auto_now_add=True)
updated_at = DateTimeField(auto_now=True)
class Meta:
app_label = 'planner'
class Idea(models.Model): class ContentIdeas(SoftDeletableModel, SiteSectorBaseModel):
site = ForeignKey(Site) """Content ideas generated from keyword clusters."""
title = CharField(max_length=255) idea_title = CharField(max_length=255, db_index=True)
keyword = ForeignKey(Keyword) description = TextField(blank=True, null=True)
cluster = ForeignKey(Cluster, null=True) primary_focus_keywords = CharField(max_length=500, blank=True)
sector = CharField(max_length=255) # NEW target_keywords = CharField(max_length=500, blank=True)
structure = CharField(choices=['guide_tutorial', 'comparison', 'review', 'how_to', 'question', 'listicle']) # NEW keyword_objects = ManyToManyField('Keywords', blank=True, related_name='content_ideas')
content_type = CharField(choices=['cluster_hub', 'blog_post', 'product_page', 'term_page', 'service_page', 'landing_page', 'business_page']) # NEW keyword_cluster = ForeignKey('Clusters', on_delete=models.SET_NULL, null=True, blank=True)
sag_cluster_id = UUIDField(null=True, blank=True) # NEW status = CharField(max_length=50, choices=[('new','New'),('queued','Queued'),('completed','Completed')], default='new')
idea_source = CharField(choices=['auto_generate', 'sag_blueprint']) # NEW disabled = BooleanField(default=False)
estimated_word_count = IntegerField(default=1000)
content_type = CharField(max_length=50, choices=[('post','Post'),('page','Page'),('product','Product'),('taxonomy','Taxonomy')], default='post')
content_structure = CharField(max_length=50, choices=[
('article','Article'),('guide','Guide'),('comparison','Comparison'),
('review','Review'),('listicle','Listicle'),('landing_page','Landing Page'),
('business_page','Business Page'),('service_page','Service Page'),
('general','General'),('cluster_hub','Cluster Hub'),('product_page','Product Page'),
('category_archive','Category Archive'),('tag_archive','Tag Archive'),
('attribute_archive','Attribute Archive'),
], default='article')
# NEW: SAG fields
sag_cluster_id = IntegerField(null=True, blank=True) # Links to sag.SAGCluster PK
idea_source = CharField(choices=['auto_generate', 'sag_blueprint'], null=True, blank=True) # NEW
execution_phase = IntegerField(null=True) # NEW: 1-4 from blueprint execution_phase = IntegerField(null=True) # NEW: 1-4 from blueprint
created_at = DateTimeField(auto_now_add=True) created_at = DateTimeField(auto_now_add=True)
class Meta:
app_label = 'planner'
class Task(models.Model): # igny8_core/business/content/models.py — Content Pipeline (app_label: writer)
site = ForeignKey(Site)
title = CharField(max_length=255) class Tasks(SoftDeletableModel, SiteSectorBaseModel):
idea = ForeignKey(Idea) """Tasks model for content generation queue."""
status = CharField(choices=['pending', 'assigned', 'in_progress', 'review', 'completed']) title = CharField(max_length=255, db_index=True)
assigned_to = ForeignKey(User, null=True) description = TextField(blank=True, null=True)
sag_cluster_id = UUIDField(null=True, blank=True) # NEW cluster = ForeignKey('planner.Clusters', on_delete=models.SET_NULL, null=True, blank=False)
idea = ForeignKey('planner.ContentIdeas', on_delete=models.SET_NULL, null=True, blank=True)
content_type = CharField(max_length=100, choices=[('post','Post'),('page','Page'),('product','Product'),('taxonomy','Taxonomy')], default='post')
content_structure = CharField(max_length=100, choices=[...same as ContentIdeas...], default='article')
taxonomy_term = ForeignKey('ContentTaxonomy', on_delete=models.SET_NULL, null=True, blank=True)
keywords = TextField(blank=True, null=True, help_text='Comma-separated keywords')
word_count = IntegerField(default=1000)
status = CharField(max_length=50, choices=[('queued','Queued'),('completed','Completed')], default='queued')
# NEW: SAG fields
sag_cluster_id = IntegerField(null=True, blank=True) # Links to sag.SAGCluster PK
blueprint_context = JSONField(null=True, blank=True) # NEW: execution context blueprint_context = JSONField(null=True, blank=True) # NEW: execution context
created_at = DateTimeField(auto_now_add=True) created_at = DateTimeField(auto_now_add=True)
updated_at = DateTimeField(auto_now=True)
class Meta:
app_label = 'writer'
class Content(models.Model): class Content(SoftDeletableModel, SiteSectorBaseModel):
site = ForeignKey(Site) """Content model for AI-generated or WordPress-imported content."""
title = CharField(max_length=255) title = CharField(max_length=255, db_index=True)
body = TextField() content_html = TextField(help_text='Final HTML content') # NOTE: field is content_html, NOT body
task = ForeignKey(Task, null=True) word_count = IntegerField(default=0)
content_type = CharField(choices=['cluster_hub', 'blog_post', 'product_page', 'term_page', 'service_page', 'landing_page', 'business_page']) # NEW meta_title = CharField(max_length=255, blank=True, null=True)
content_structure = CharField(choices=['guide_tutorial', 'comparison', 'review', 'how_to', 'question', 'listicle']) # NEW meta_description = TextField(blank=True, null=True)
sag_cluster_id = UUIDField(null=True, blank=True) # NEW primary_keyword = CharField(max_length=255, blank=True, null=True)
taxonomies = JSONField(default=dict, null=True, blank=True) # NEW: custom WP taxonomies secondary_keywords = JSONField(default=list, blank=True)
status = CharField(choices=['draft', 'review', 'published']) cluster = ForeignKey('planner.Clusters', on_delete=models.SET_NULL, null=True, blank=False)
content_type = CharField(max_length=50, choices=[('post','Post'),('page','Page'),('product','Product'),('taxonomy','Taxonomy')], default='post')
content_structure = CharField(max_length=50, choices=[...same as Tasks...], default='article')
taxonomy_terms = ManyToManyField('ContentTaxonomy', through='ContentTaxonomyRelation', blank=True)
external_id = CharField(max_length=255, blank=True, null=True)
external_url = URLField(blank=True, null=True)
source = CharField(max_length=50, choices=[('igny8','IGNY8 Generated'),('wordpress','WordPress Imported')], default='igny8')
status = CharField(max_length=50, choices=[('draft','Draft'),('review','Review'),('approved','Approved'),('published','Published')], default='draft')
# NEW: SAG fields
sag_cluster_id = IntegerField(null=True, blank=True) # Links to sag.SAGCluster PK
created_at = DateTimeField(auto_now_add=True) created_at = DateTimeField(auto_now_add=True)
updated_at = DateTimeField(auto_now=True)
class Meta:
app_label = 'writer'
class Image(models.Model): class Images(SoftDeletableModel, SiteSectorBaseModel):
content = ForeignKey(Content) """Images model — note: class is Images (plural)."""
url = URLField() content = ForeignKey(Content, on_delete=models.CASCADE, null=True, blank=True)
alt_text = CharField(max_length=255) task = ForeignKey(Tasks, on_delete=models.CASCADE, null=True, blank=True)
style_type = CharField(choices=['hero', 'supporting', 'ecommerce', 'category', 'service', 'conversion']) # NEW image_type = CharField(max_length=50, choices=[('featured','Featured'),('desktop','Desktop'),('mobile','Mobile'),('in_article','In-Article')], default='featured')
sag_cluster_id = UUIDField(null=True, blank=True) # NEW image_url = CharField(max_length=500, blank=True, null=True) # NOTE: field is image_url, NOT url
image_path = CharField(max_length=500, blank=True, null=True)
prompt = TextField(blank=True, null=True) # Generation prompt
caption = TextField(blank=True, null=True) # NOTE: field is caption, NOT alt_text
status = CharField(max_length=50, default='pending')
position = IntegerField(default=0)
# NEW: SAG fields
sag_cluster_id = IntegerField(null=True, blank=True) # Links to sag.SAGCluster PK
style_type = CharField(max_length=50, choices=[('hero','Hero'),('supporting','Supporting'),('ecommerce','Ecommerce'),('category','Category'),('service','Service'),('conversion','Conversion')], null=True, blank=True) # NEW
created_at = DateTimeField(auto_now_add=True) created_at = DateTimeField(auto_now_add=True)
class Meta:
app_label = 'writer'
class Job(models.Model): class Job(models.Model):
"""Pipeline execution tracking""" """Pipeline execution tracking (NEW model — does not yet exist in codebase)."""
site = ForeignKey(Site) site = ForeignKey('igny8_core_auth.Site', on_delete=models.CASCADE)
status = CharField(choices=['pending', 'running', 'completed', 'failed']) status = CharField(choices=['pending', 'running', 'completed', 'failed'])
stage = IntegerField(choices=[(0, 'Blueprint Check'), (1, 'Keywords'), (2, 'Cluster'), (3, 'Ideas'), (4, 'Tasks'), (5, 'Content'), (6, 'Taxonomy'), (7, 'Images')]) stage = IntegerField(choices=[(0, 'Blueprint Check'), (1, 'Keywords'), (2, 'Cluster'), (3, 'Ideas'), (4, 'Tasks'), (5, 'Content'), (6, 'Taxonomy'), (7, 'Images')])
blueprint_mode = CharField(choices=['legacy', 'blueprint_aware']) # NEW blueprint_mode = CharField(choices=['legacy', 'blueprint_aware']) # NEW
@@ -389,24 +463,27 @@ class Job(models.Model):
#### Stage 0: Blueprint Check #### Stage 0: Blueprint Check
```python ```python
# celery_app/tasks.py # igny8_core/tasks.py (Celery app: celery -A igny8_core)
@app.task(bind=True, max_retries=3) @app.task(bind=True, max_retries=3)
def check_blueprint(self, site_id): def check_blueprint(self, site_id):
""" """
Stage 0: Determine execution mode and load blueprint context. Stage 0: Determine execution mode and load blueprint context.
Args:
site_id: integer PK (BigAutoField)
Returns: Returns:
{ {
'status': 'success', 'status': 'success',
'pipeline_mode': 'blueprint_aware' | 'legacy', 'pipeline_mode': 'blueprint_aware' | 'legacy',
'blueprint_id': 'uuid' (if active), 'blueprint_id': integer (if active),
'execution_phases': list, 'execution_phases': list,
'next_stage': 1 'next_stage': 1
} }
""" """
try: try:
site = Site.objects.get(id=site_id) site = Site.objects.get(id=site_id) # integer PK lookup
job = Job.objects.create(site=site, stage=0, status='running') job = Job.objects.create(site=site, stage=0, status='running')
blueprint = SAGBlueprint.objects.filter( blueprint = SAGBlueprint.objects.filter(
@@ -418,7 +495,7 @@ def check_blueprint(self, site_id):
result = { result = {
'status': 'success', 'status': 'success',
'pipeline_mode': 'blueprint_aware', 'pipeline_mode': 'blueprint_aware',
'blueprint_id': str(blueprint.id), 'blueprint_id': blueprint.id,
'execution_phases': blueprint.execution_priority, 'execution_phases': blueprint.execution_priority,
} }
job.blueprint_mode = 'blueprint_aware' job.blueprint_mode = 'blueprint_aware'
@@ -464,7 +541,7 @@ def process_keywords(self, site_id, blueprint_context):
blueprint_mode=blueprint_context['pipeline_mode'] blueprint_mode=blueprint_context['pipeline_mode']
) )
keywords = Keyword.objects.filter(site=site, sag_cluster_id__isnull=True) keywords = Keywords.objects.filter(site=site, sag_cluster_id__isnull=True)
if blueprint_context['pipeline_mode'] == 'blueprint_aware': if blueprint_context['pipeline_mode'] == 'blueprint_aware':
blueprint = SAGBlueprint.objects.get(id=blueprint_context['blueprint_id']) blueprint = SAGBlueprint.objects.get(id=blueprint_context['blueprint_id'])
@@ -479,11 +556,11 @@ def process_keywords(self, site_id, blueprint_context):
if cluster: if cluster:
keyword.sag_cluster_id = cluster.id keyword.sag_cluster_id = cluster.id
keyword.save() keyword.save()
cluster.keywords.append(keyword.term) cluster.keywords.append(keyword.keyword)
cluster.save() cluster.save()
matched_count += 1 matched_count += 1
else: else:
unmatched_keywords.append(keyword.term) unmatched_keywords.append(keyword.keyword)
job.log = f"Matched {matched_count} keywords. Unmatched: {unmatched_keywords}" job.log = f"Matched {matched_count} keywords. Unmatched: {unmatched_keywords}"
else: else:
@@ -615,15 +692,15 @@ def create_tasks(self, site_id, blueprint_context):
blueprint_mode=blueprint_context['pipeline_mode'] blueprint_mode=blueprint_context['pipeline_mode']
) )
ideas = Idea.objects.filter(site=site, task__isnull=True) ideas = ContentIdeas.objects.filter(site=site, task__isnull=True)
task_count = 0 task_count = 0
for idea in ideas: for idea in ideas:
task = Task.objects.create( task = Tasks.objects.create(
site=site, site=site,
title=idea.title, title=idea.idea_title,
idea=idea, idea=idea,
status='pending' status='queued' # Tasks.STATUS_CHOICES: queued/completed
) )
if blueprint_context['pipeline_mode'] == 'blueprint_aware' and idea.sag_cluster_id: if blueprint_context['pipeline_mode'] == 'blueprint_aware' and idea.sag_cluster_id:
@@ -632,14 +709,14 @@ def create_tasks(self, site_id, blueprint_context):
task.sag_cluster_id = idea.sag_cluster_id task.sag_cluster_id = idea.sag_cluster_id
task.blueprint_context = { task.blueprint_context = {
'cluster_id': str(cluster.id), 'cluster_id': cluster.id,
'cluster_name': cluster.name, 'cluster_name': cluster.name,
'cluster_type': cluster.cluster_type, 'cluster_type': cluster.cluster_type,
'cluster_sector': cluster.sector, 'cluster_sector': cluster.sector,
'hub_title': blueprint.content_plan.get(str(cluster.id), {}).get('hub_title'), 'hub_title': blueprint.content_plan.get(str(cluster.id), {}).get('hub_title'),
'hub_url': f"{site.domain}/hubs/{cluster.name.lower().replace(' ', '-')}", 'hub_url': f"{site.domain}/hubs/{cluster.name.lower().replace(' ', '-')}",
'cluster_attributes': cluster.attributes, 'cluster_attributes': cluster.attributes,
'content_structure': idea.structure, 'content_structure': idea.content_structure,
'content_type': idea.content_type, 'content_type': idea.content_type,
'execution_phase': idea.execution_phase, 'execution_phase': idea.execution_phase,
} }
@@ -683,7 +760,7 @@ def generate_content(self, site_id, blueprint_context):
blueprint_mode=blueprint_context['pipeline_mode'] blueprint_mode=blueprint_context['pipeline_mode']
) )
tasks = Task.objects.filter(site=site, status='completed', content__isnull=True) tasks = Tasks.objects.filter(site=site, status='completed', content__isnull=True)
content_count = 0 content_count = 0
for task in tasks: for task in tasks:
@@ -795,7 +872,7 @@ def assign_taxonomy(self, site_id, blueprint_context):
cluster = SAGCluster.objects.get(id=content.sag_cluster_id) cluster = SAGCluster.objects.get(id=content.sag_cluster_id)
# Load taxonomy mapping from blueprint # Load taxonomy mapping from blueprint
tax_mapping = blueprint.wp_taxonomy_mapping.get(str(cluster.id), {}) tax_mapping = blueprint.wp_taxonomy_mapping.get(cluster.id, {})
# Assign taxonomies # Assign taxonomies
content.taxonomies = tax_mapping content.taxonomies = tax_mapping
@@ -863,7 +940,7 @@ def generate_images(self, site_id, blueprint_context):
# Generate featured image # Generate featured image
featured_image = GenerateImage(content.title, style) featured_image = GenerateImage(content.title, style)
image = Image.objects.create( image = Images.objects.create(
content=content, content=content,
url=featured_image['url'], url=featured_image['url'],
alt_text=featured_image['alt_text'], alt_text=featured_image['alt_text'],
@@ -1019,7 +1096,7 @@ redis-server
# Create sample site and blueprint # Create sample site and blueprint
python manage.py shell << EOF python manage.py shell << EOF
from django.contrib.auth.models import User from django.contrib.auth.models import User
from sites.models import Site from igny8_core.auth.models import Site
from sag.models import SAGBlueprint, SAGCluster from sag.models import SAGBlueprint, SAGCluster
site = Site.objects.create(name="Test Site", domain="test.local") site = Site.objects.create(name="Test Site", domain="test.local")
@@ -1052,27 +1129,25 @@ EOF
#### Execute Pipeline Stages #### Execute Pipeline Stages
```bash ```bash
# Start Celery worker (in separate terminal) # Start Celery worker (in separate terminal)
celery -A igny8.celery_app worker --loglevel=info celery -A igny8_core worker --loglevel=info
# Run Stage 0: Blueprint Check # Run Stage 0: Blueprint Check
python manage.py shell << EOF python manage.py shell << EOF
from celery_app.tasks import check_blueprint from igny8_core.tasks import check_blueprint
result = check_blueprint.delay(site_id="<site-uuid>") result = check_blueprint.delay(site_id="<site-id>")
print(result.get()) print(result.get())
EOF EOF
# Run full pipeline # Run full pipeline
python manage.py shell << EOF python manage.py shell << EOF
from celery_app.tasks import check_blueprint from igny8_core.tasks import check_blueprint
from uuid import UUID site_id = 1 # integer PK (BigAutoField)
site_id = UUID("<site-uuid>")
check_blueprint.delay(site_id) check_blueprint.delay(site_id)
# Each stage automatically chains to the next # Each stage automatically chains to the next
EOF EOF
# Monitor pipeline execution # Monitor pipeline execution
celery -A igny8.celery_app events celery -A igny8_core events
# or view logs: tail -f celery.log # or view logs: tail -f celery.log
``` ```
@@ -1080,20 +1155,20 @@ celery -A igny8.celery_app events
#### Unit Tests #### Unit Tests
```bash ```bash
pytest content/tests/test_pipeline.py -v pytest igny8_core/business/content/tests/test_pipeline.py -v
pytest sag/tests/test_blueprint.py -v pytest igny8_core/sag/tests/test_blueprint.py -v
pytest celery_app/tests/test_tasks.py -v pytest igny8_core/tests/test_tasks.py -v
``` ```
#### Integration Test #### Integration Test
```bash ```bash
pytest content/tests/test_pipeline_integration.py::test_full_blueprint_pipeline -v pytest igny8_core/business/content/tests/test_pipeline_integration.py::test_full_blueprint_pipeline -v
# Test legacy mode # Test legacy mode
pytest content/tests/test_pipeline_integration.py::test_full_legacy_pipeline -v pytest igny8_core/business/content/tests/test_pipeline_integration.py::test_full_legacy_pipeline -v
# Test mixed mode (some sites with blueprint, some without) # Test mixed mode (some sites with blueprint, some without)
pytest content/tests/test_pipeline_integration.py::test_mixed_mode_execution -v pytest igny8_core/business/content/tests/test_pipeline_integration.py::test_mixed_mode_execution -v
``` ```
#### Manual Test Scenario #### Manual Test Scenario
@@ -1103,37 +1178,37 @@ python manage.py shell < scripts/setup_test_data.py
# 2. Import sample keywords # 2. Import sample keywords
python manage.py shell << EOF python manage.py shell << EOF
from content.models import Keyword from igny8_core.business.content.models import Keyword
from sites.models import Site from igny8_core.auth.models import Site
site = Site.objects.get(name="Test Site") site = Site.objects.get(name="Test Site")
keywords = ["python tutorial", "django rest", "web scraping"] keywords = ["python tutorial", "django rest", "web scraping"]
for kw in keywords: for kw in keywords:
Keyword.objects.create(site=site, term=kw, source='csv_import') Keywords.objects.create(site=site, term=kw, source='csv_import')
EOF EOF
# 3. Run pipeline # 3. Run pipeline
celery -A igny8.celery_app worker --loglevel=debug & celery -A igny8_core worker --loglevel=debug &
python manage.py shell << EOF python manage.py shell << EOF
from celery_app.tasks import check_blueprint from igny8_core.tasks import check_blueprint
from sites.models import Site from igny8_core.auth.models import Site
site = Site.objects.get(name="Test Site") site = Site.objects.get(name="Test Site")
check_blueprint.delay(site.id) check_blueprint.delay(site.id)
EOF EOF
# 4. Inspect results # 4. Inspect results
python manage.py shell << EOF python manage.py shell << EOF
from content.models import Keyword, Idea, Task, Content, Image from igny8_core.business.content.models import Keyword, Idea, Task, Content, Image
from sites.models import Site from igny8_core.auth.models import Site
site = Site.objects.get(name="Test Site") site = Site.objects.get(name="Test Site")
print("Keywords:", Keyword.objects.filter(site=site).count()) print("Keywords:", Keywords.objects.filter(site=site).count())
print("Ideas:", Idea.objects.filter(site=site).count()) print("Ideas:", ContentIdeas.objects.filter(site=site).count())
print("Tasks:", Task.objects.filter(site=site).count()) print("Tasks:", Tasks.objects.filter(site=site).count())
print("Content:", Content.objects.filter(site=site).count()) print("Content:", Content.objects.filter(site=site).count())
print("Images:", Image.objects.filter(site=site).count()) print("Images:", Images.objects.filter(site=site).count())
# Check blueprint context # Check blueprint context
task = Task.objects.filter(site=site, blueprint_context__isnull=False).first() task = Tasks.objects.filter(site=site, blueprint_context__isnull=False).first()
if task: if task:
print("Blueprint context:", task.blueprint_context) print("Blueprint context:", task.blueprint_context)
EOF EOF
@@ -1146,7 +1221,7 @@ EOF
# Check if blueprint exists and is active # Check if blueprint exists and is active
python manage.py shell << EOF python manage.py shell << EOF
from sag.models import SAGBlueprint from sag.models import SAGBlueprint
from sites.models import Site from igny8_core.auth.models import Site
site = Site.objects.get(id="<site-id>") site = Site.objects.get(id="<site-id>")
blueprint = SAGBlueprint.objects.filter(site=site, status='active').first() blueprint = SAGBlueprint.objects.filter(site=site, status='active').first()
print(f"Blueprint: {blueprint}") print(f"Blueprint: {blueprint}")
@@ -1160,9 +1235,9 @@ EOF
```bash ```bash
# Check keyword-cluster mapping # Check keyword-cluster mapping
python manage.py shell << EOF python manage.py shell << EOF
from content.models import Keyword from igny8_core.business.content.models import Keyword
from sag.models import SAGCluster from sag.models import SAGCluster
keywords = Keyword.objects.filter(sag_cluster_id__isnull=True) keywords = Keywords.objects.filter(sag_cluster_id__isnull=True)
print(f"Unmatched keywords: {[kw.term for kw in keywords]}") print(f"Unmatched keywords: {[kw.term for kw in keywords]}")
# Check available clusters # Check available clusters
@@ -1176,16 +1251,16 @@ EOF
```bash ```bash
# Check task status # Check task status
python manage.py shell << EOF python manage.py shell << EOF
from content.models import Task from igny8_core.business.content.models import Task
tasks = Task.objects.all() tasks = Tasks.objects.all()
for task in tasks: for task in tasks:
print(f"Task {task.id}: status={task.status}, blueprint_context={bool(task.blueprint_context)}") print(f"Task {task.id}: status={task.status}, blueprint_context={bool(task.blueprint_context)}")
EOF EOF
# Check Celery task logs # Check Celery task logs
celery -A igny8.celery_app inspect active celery -A igny8_core inspect active
celery -A igny8.celery_app inspect reserved celery -A igny8_core inspect reserved
celery -A igny8.celery_app purge # WARNING: clears queue celery -A igny8_core purge # WARNING: clears queue
``` ```
### Extending with Custom Prompt Templates ### Extending with Custom Prompt Templates
@@ -1225,7 +1300,7 @@ PROMPT_TEMPLATES = {
```bash ```bash
# View pipeline execution history # View pipeline execution history
python manage.py shell << EOF python manage.py shell << EOF
from content.models import Job from igny8_core.business.content.models import Job
jobs = Job.objects.filter(stage=5).order_by('-created_at')[:10] jobs = Job.objects.filter(stage=5).order_by('-created_at')[:10]
for job in jobs: for job in jobs:
duration = (job.completed_at - job.created_at).total_seconds() if job.completed_at else None duration = (job.completed_at - job.created_at).total_seconds() if job.completed_at else None

1265
v2/V2-Execution-Docs/01E-blueprint-aware-pipeline.md.bak Normal file
View File

File diff suppressed because it is too large Load Diff

105
v2/V2-Execution-Docs/01F-existing-site-analysis-case1.md
View File

@@ -1,5 +1,9 @@
# 01F: IGNY8 Phase 1 — Existing Site Analysis (Case 1) # 01F: IGNY8 Phase 1 — Existing Site Analysis (Case 1)
> **Version:** 1.1 (codebase-verified)
> **Source of Truth:** Codebase at `/data/app/igny8/backend/`
> **Last Verified:** 2025-07-14
**Document Type:** Build Specification **Document Type:** Build Specification
**Phase:** Phase 1: Existing Site Analysis **Phase:** Phase 1: Existing Site Analysis
**Use Case:** Case 1 (Users with existing sites) **Use Case:** Case 1 (Users with existing sites)
@@ -176,8 +180,8 @@ def extract_site_attributes(
```json ```json
{ {
"analysis_id": "uuid", "analysis_id": 42,
"site_id": "uuid", "site_id": 7,
"timestamp": "2026-03-23T14:30:00Z", "timestamp": "2026-03-23T14:30:00Z",
"analysis_confidence": 0.82, "analysis_confidence": 0.82,
"attributes": [ "attributes": [
@@ -298,7 +302,7 @@ from typing import List, Dict, Optional
@dataclass @dataclass
class Product: class Product:
id: str id: int
title: str title: str
description: str description: str
sku: str sku: str
@@ -310,10 +314,10 @@ class Product:
@dataclass @dataclass
class Category: class Category:
id: str id: int
name: str name: str
slug: str slug: str
parent_id: Optional[str] parent_id: Optional[int]
description: str description: str
product_count: int product_count: int
@@ -326,16 +330,16 @@ class Taxonomy:
@dataclass @dataclass
class Term: class Term:
id: str id: int
name: str name: str
slug: str slug: str
parent_id: Optional[str] parent_id: Optional[int]
description: str description: str
count: int count: int
@dataclass @dataclass
class Page: class Page:
id: str id: int
title: str title: str
url: str url: str
content_summary: str content_summary: str
@@ -343,7 +347,7 @@ class Page:
@dataclass @dataclass
class Post: class Post:
id: str id: int
title: str title: str
url: str url: str
content_summary: str content_summary: str
@@ -353,15 +357,15 @@ class Post:
@dataclass @dataclass
class MenuItem: class MenuItem:
id: str id: int
title: str title: str
url: str url: str
target: str target: str
parent_id: Optional[str] parent_id: Optional[int]
@dataclass @dataclass
class SiteMetadata: class SiteMetadata:
site_id: str site_id: int
domain: str domain: str
wordpress_version: str wordpress_version: str
woocommerce_version: str woocommerce_version: str
@@ -425,8 +429,8 @@ class AnalysisNotes:
@dataclass @dataclass
class AttributeExtractionResult: class AttributeExtractionResult:
analysis_id: str analysis_id: int
site_id: str site_id: int
timestamp: str timestamp: str
analysis_confidence: float analysis_confidence: float
attributes: List[DiscoveredAttribute] attributes: List[DiscoveredAttribute]
@@ -483,9 +487,9 @@ class AttributeExtractionResult:
```json ```json
{ {
"analysis_id": "uuid", "analysis_id": 42,
"site_id": "uuid", "site_id": 7,
"blueprint_id": "uuid", "blueprint_id": 15,
"timestamp": "2026-03-23T14:30:00Z", "timestamp": "2026-03-23T14:30:00Z",
"summary": { "summary": {
"products_current": 50, "products_current": 50,
@@ -599,15 +603,15 @@ class AttributeExtractionResult:
```json ```json
{ {
"batch_id": "uuid", "batch_id": 23,
"site_id": "uuid", "site_id": 7,
"blueprint_id": "uuid", "blueprint_id": 15,
"timestamp": "2026-03-23T14:30:00Z", "timestamp": "2026-03-23T14:30:00Z",
"total_products": 50, "total_products": 50,
"total_suggestions": 87, "total_suggestions": 87,
"suggestions": [ "suggestions": [
{ {
"product_id": "woo_123", "product_id": 123,
"product_title": "Nekteck Foot Massager with Heat", "product_title": "Nekteck Foot Massager with Heat",
"proposed_tags": [ "proposed_tags": [
{ {
@@ -659,7 +663,7 @@ All endpoints are authenticated via `Authorization: Bearer {IGNY8_API_TOKEN}` he
{ {
"include_draft_products": false, "include_draft_products": false,
"product_limit": 500, "product_limit": 500,
"sector_template_id": "optional_uuid", "sector_template_id": null,
"webhook_url": "optional_https_url_for_completion_notification" "webhook_url": "optional_https_url_for_completion_notification"
} }
``` ```
@@ -668,7 +672,7 @@ All endpoints are authenticated via `Authorization: Bearer {IGNY8_API_TOKEN}` he
```json ```json
{ {
"task_id": "celery_task_uuid", "task_id": "celery_task_uuid",
"site_id": "site_uuid", "site_id": 7,
"status": "queued", "status": "queued",
"estimated_duration_seconds": 120, "estimated_duration_seconds": 120,
"check_status_url": "/api/v1/sag/sites/{site_id}/analysis-status/?task_id={task_id}" "check_status_url": "/api/v1/sag/sites/{site_id}/analysis-status/?task_id={task_id}"
@@ -694,7 +698,7 @@ All endpoints are authenticated via `Authorization: Bearer {IGNY8_API_TOKEN}` he
```json ```json
{ {
"task_id": "celery_task_uuid", "task_id": "celery_task_uuid",
"site_id": "site_uuid", "site_id": 7,
"status": "processing", "status": "processing",
"progress_percent": 45, "progress_percent": 45,
"current_step": "Analyzing product attributes", "current_step": "Analyzing product attributes",
@@ -718,8 +722,8 @@ All endpoints are authenticated via `Authorization: Bearer {IGNY8_API_TOKEN}` he
**Response:** 200 OK **Response:** 200 OK
```json ```json
{ {
"analysis_id": "uuid", "analysis_id": 42,
"site_id": "site_uuid", "site_id": 7,
"timestamp": "2026-03-23T14:30:00Z", "timestamp": "2026-03-23T14:30:00Z",
"site_data_summary": { "site_data_summary": {
"total_products": 50, "total_products": 50,
@@ -756,7 +760,7 @@ All endpoints are authenticated via `Authorization: Bearer {IGNY8_API_TOKEN}` he
**Request:** **Request:**
```json ```json
{ {
"analysis_id": "uuid", "analysis_id": 42,
"approved_attributes": [ "approved_attributes": [
{ {
"name": "Target Area", "name": "Target Area",
@@ -764,16 +768,16 @@ All endpoints are authenticated via `Authorization: Bearer {IGNY8_API_TOKEN}` he
"exclude_values": [] "exclude_values": []
} }
], ],
"confirmed_by_user_id": "user_uuid" "confirmed_by_user_id": 3
} }
``` ```
**Response:** 201 Created **Response:** 201 Created
```json ```json
{ {
"blueprint_id": "uuid", "blueprint_id": 15,
"site_id": "site_uuid", "site_id": 7,
"analysis_id": "uuid", "analysis_id": 42,
"status": "created", "status": "created",
"attributes_count": 8, "attributes_count": 8,
"attribute_values_count": 45, "attribute_values_count": 45,
@@ -800,12 +804,12 @@ All endpoints are authenticated via `Authorization: Bearer {IGNY8_API_TOKEN}` he
**Response:** 200 OK **Response:** 200 OK
```json ```json
{ {
"batch_id": "uuid", "batch_id": 23,
"blueprint_id": "blueprint_uuid", "blueprint_id": 15,
"total_suggestions": 87, "total_suggestions": 87,
"suggestions": [ "suggestions": [
{ {
"product_id": "woo_123", "product_id": 123,
"product_title": "Nekteck Foot Massager", "product_title": "Nekteck Foot Massager",
"proposed_tags": [ "proposed_tags": [
{ {
@@ -829,10 +833,10 @@ All endpoints are authenticated via `Authorization: Bearer {IGNY8_API_TOKEN}` he
**Request:** **Request:**
```json ```json
{ {
"blueprint_id": "uuid", "blueprint_id": 15,
"approved_suggestions": [ "approved_suggestions": [
{ {
"product_id": "woo_123", "product_id": 123,
"approved_tags": [ "approved_tags": [
{ {
"attribute": "Target Area", "attribute": "Target Area",
@@ -849,8 +853,8 @@ All endpoints are authenticated via `Authorization: Bearer {IGNY8_API_TOKEN}` he
```json ```json
{ {
"task_id": "celery_task_uuid", "task_id": "celery_task_uuid",
"site_id": "site_uuid", "site_id": 7,
"blueprint_id": "blueprint_uuid", "blueprint_id": 15,
"status": "processing", "status": "processing",
"products_to_tag": 47, "products_to_tag": 47,
"tags_to_apply": 87, "tags_to_apply": 87,
@@ -871,7 +875,7 @@ All endpoints are authenticated via `Authorization: Bearer {IGNY8_API_TOKEN}` he
```json ```json
{ {
"task_id": "celery_task_uuid", "task_id": "celery_task_uuid",
"site_id": "site_uuid", "site_id": 7,
"status": "processing", "status": "processing",
"progress_percent": 62, "progress_percent": 62,
"products_tagged": 29, "products_tagged": 29,
@@ -902,7 +906,7 @@ All endpoints are authenticated via `Authorization: Bearer {IGNY8_API_TOKEN}` he
```json ```json
{ {
"metadata": { "metadata": {
"site_id": "uuid", "site_id": 7,
"domain": "example-store.com", "domain": "example-store.com",
"wordpress_version": "6.4.2", "wordpress_version": "6.4.2",
"woocommerce_version": "8.5.0", "woocommerce_version": "8.5.0",
@@ -915,7 +919,7 @@ All endpoints are authenticated via `Authorization: Bearer {IGNY8_API_TOKEN}` he
}, },
"products": [ "products": [
{ {
"id": "woo_123", "id": 123,
"title": "Nekteck Foot Massager with Heat", "title": "Nekteck Foot Massager with Heat",
"description": "Premium foot massage device...", "description": "Premium foot massage device...",
"sku": "NEKTECK-FM-001", "sku": "NEKTECK-FM-001",
@@ -932,7 +936,7 @@ All endpoints are authenticated via `Authorization: Bearer {IGNY8_API_TOKEN}` he
], ],
"categories": [ "categories": [
{ {
"id": "cat_1", "id": 1,
"name": "Foot Massagers", "name": "Foot Massagers",
"slug": "foot-massagers", "slug": "foot-massagers",
"parent_id": null, "parent_id": null,
@@ -947,7 +951,7 @@ All endpoints are authenticated via `Authorization: Bearer {IGNY8_API_TOKEN}` he
"is_hierarchical": false, "is_hierarchical": false,
"terms": [ "terms": [
{ {
"id": "brand_1", "id": 1,
"name": "Nekteck", "name": "Nekteck",
"slug": "nekteck", "slug": "nekteck",
"parent_id": null, "parent_id": null,
@@ -959,7 +963,7 @@ All endpoints are authenticated via `Authorization: Bearer {IGNY8_API_TOKEN}` he
], ],
"pages": [ "pages": [
{ {
"id": "page_1", "id": 1,
"title": "Shop", "title": "Shop",
"url": "/shop", "url": "/shop",
"content_summary": "Browse our selection of massage devices", "content_summary": "Browse our selection of massage devices",
@@ -968,7 +972,7 @@ All endpoints are authenticated via `Authorization: Bearer {IGNY8_API_TOKEN}` he
], ],
"posts": [ "posts": [
{ {
"id": "post_1", "id": 1,
"title": "Benefits of Foot Massage", "title": "Benefits of Foot Massage",
"url": "/blog/foot-massage-benefits", "url": "/blog/foot-massage-benefits",
"content_summary": "Learn why foot massage is beneficial...", "content_summary": "Learn why foot massage is beneficial...",
@@ -979,11 +983,11 @@ All endpoints are authenticated via `Authorization: Bearer {IGNY8_API_TOKEN}` he
], ],
"menus": [ "menus": [
{ {
"id": "menu_1", "id": 1,
"title": "Main Menu", "title": "Main Menu",
"items": [ "items": [
{ {
"id": "item_1", "id": 1,
"title": "Shop", "title": "Shop",
"url": "/shop", "url": "/shop",
"target": "_self", "target": "_self",
@@ -1184,7 +1188,10 @@ All endpoints are authenticated via `Authorization: Bearer {IGNY8_API_TOKEN}` he
--- ---
### Phase 6: Frontend Components (Week 3-4) ### Phase 6: Frontend Components — React + TypeScript (Week 3-4)
> **Tech Stack:** React ^19.0.0, TypeScript ~5.7.2, Vite ^6.1.0, Zustand ^5.0.8, Tailwind ^4.0.8
> All components are `.tsx` files in the `frontend/src/` directory.
**Tasks:** **Tasks:**
1. Implement SiteAnalysisPanel 1. Implement SiteAnalysisPanel
@@ -1455,7 +1462,7 @@ Step 6: Complete & Next Steps
**Common Issues:** **Common Issues:**
**Issue:** Analysis hangs or times out **Issue:** Analysis hangs or times out
- Check: Celery worker status (`celery -A sag inspect active`) - Check: Celery worker status (`celery -A igny8_core inspect active`)
- Check: Redis/message queue status - Check: Redis/message queue status
- Check: LLM API rate limits - Check: LLM API rate limits
- Solution: Reduce product limit, retry analysis - Solution: Reduce product limit, retry analysis

28
v2/V2-Execution-Docs/01G-sag-health-monitoring.md
View File

@@ -1,5 +1,9 @@
# IGNY8 Phase 1: SAG Health Monitoring (Doc 01G) # IGNY8 Phase 1: SAG Health Monitoring (Doc 01G)
> **Version:** 1.1 (codebase-verified)
> **Source of Truth:** Codebase at `/data/app/igny8/backend/`
> **Last Verified:** 2025-07-14
**Document ID:** 01G **Document ID:** 01G
**Module:** SAG Health Monitoring **Module:** SAG Health Monitoring
**Phase:** Phase 1 - Core Implementation **Phase:** Phase 1 - Core Implementation
@@ -599,7 +603,7 @@ def check_blueprint_evolution_triggers(site_id: int):
5. Trigger notification to user 5. Trigger notification to user
#### Step 2.2: Configure Celery Beat Schedule #### Step 2.2: Configure Celery Beat Schedule
**File:** `config/celery.py` or `config/celery_beat_schedule.py` **File:** `igny8_core/celery.py`
```python ```python
CELERY_BEAT_SCHEDULE = { CELERY_BEAT_SCHEDULE = {
@@ -802,7 +806,7 @@ Test scenarios:
### Phase 4: Dashboard Widget & Frontend (Week 4) ### Phase 4: Dashboard Widget & Frontend (Week 4)
#### Step 4.1: Create Dashboard Widget Component #### Step 4.1: Create Dashboard Widget Component
**File:** `frontend/components/SAGHealthWidget.jsx` **File:** `frontend/src/components/SAGHealthWidget.tsx`
Display: Display:
``` ```
@@ -848,7 +852,7 @@ Display:
- Add 4-week trend chart - Add 4-week trend chart
#### Step 4.2: Create Health History Chart #### Step 4.2: Create Health History Chart
**File:** `frontend/components/SAGHealthChart.jsx` **File:** `frontend/src/components/SAGHealthChart.tsx`
Line chart showing: Line chart showing:
- X-axis: Last 4 weeks (Monday to Monday) - X-axis: Last 4 weeks (Monday to Monday)
@@ -858,7 +862,7 @@ Line chart showing:
- Hover: Show detailed scores for week - Hover: Show detailed scores for week
#### Step 4.3: Create Recommendations Page #### Step 4.3: Create Recommendations Page
**File:** `frontend/pages/SAGRecommendations.jsx` **File:** `frontend/src/pages/SAGRecommendations.tsx`
Page showing: Page showing:
- All recommendations (20+) - All recommendations (20+)
@@ -869,7 +873,7 @@ Page showing:
- Status tracking (completed/in-progress/pending) - Status tracking (completed/in-progress/pending)
#### Step 4.4: Create Blueprint Version History Page #### Step 4.4: Create Blueprint Version History Page
**File:** `frontend/pages/BlueprintVersionHistory.jsx` **File:** `frontend/src/pages/BlueprintVersionHistory.tsx`
Display: Display:
- Timeline of all versions - Timeline of all versions
@@ -881,7 +885,7 @@ Display:
- Activate/rollback buttons for archived versions - Activate/rollback buttons for archived versions
#### Step 4.5: Create Evolution Trigger Review Page #### Step 4.5: Create Evolution Trigger Review Page
**File:** `frontend/pages/EvolutionTriggerReview.jsx` **File:** `frontend/src/pages/EvolutionTriggerReview.tsx`
Display detected triggers: Display detected triggers:
- New product categories (with suggestion) - New product categories (with suggestion)
@@ -891,7 +895,7 @@ Display detected triggers:
- Preview new blueprint structure before creation - Preview new blueprint structure before creation
#### Step 4.6: Frontend Tests #### Step 4.6: Frontend Tests
**File:** `tests/frontend/SAGHealthWidget.test.js` **File:** `frontend/src/__tests__/SAGHealthWidget.test.tsx`
Test: Test:
- Widget renders with health score - Widget renders with health score
@@ -1118,7 +1122,7 @@ Follow Phase 1 → Phase 5 sequentially. Do not skip phases.
#### 6.5 Code Style & Standards #### 6.5 Code Style & Standards
- Follow PEP 8 for Python - Follow PEP 8 for Python
- Follow ESLint rules for JavaScript/React - Follow ESLint rules for TypeScript/React
- Comment complex calculations (especially health score components) - Comment complex calculations (especially health score components)
- Use meaningful variable names (not `x`, `y`, `temp`) - Use meaningful variable names (not `x`, `y`, `temp`)
- Docstrings on all public methods - Docstrings on all public methods
@@ -1134,7 +1138,7 @@ Follow Phase 1 → Phase 5 sequentially. Do not skip phases.
**Celery task not running:** **Celery task not running:**
- Verify Celery Beat schedule is configured - Verify Celery Beat schedule is configured
- Check task is registered (`celery -A project inspect active_queues`) - Check task is registered (`celery -A igny8_core inspect active_queues`)
- Check for errors in Celery worker logs - Check for errors in Celery worker logs
- Test task manually via shell: `run_blueprint_health_check.delay(site_id=1)` - Test task manually via shell: `run_blueprint_health_check.delay(site_id=1)`
@@ -1147,7 +1151,7 @@ Follow Phase 1 → Phase 5 sequentially. Do not skip phases.
**Frontend chart not rendering:** **Frontend chart not rendering:**
- Inspect network tab for API response - Inspect network tab for API response
- Verify response format matches serializer schema - Verify response format matches serializer schema
- Check console for JavaScript errors - Check console for TypeScript/runtime errors
- Test with mock data first - Test with mock data first
#### 6.7 Version Control Workflow #### 6.7 Version Control Workflow
@@ -1211,8 +1215,8 @@ Create/update these files:
**Staging deployment:** **Staging deployment:**
1. Deploy code to staging server 1. Deploy code to staging server
2. Run migrations: `python manage.py migrate sag` 2. Run migrations: `python manage.py migrate sag`
3. Start Celery worker: `celery -A project worker -l info` 3. Start Celery worker: `celery -A igny8_core worker -l info`
4. Start Celery Beat: `celery -A project beat -l info` 4. Start Celery Beat: `celery -A igny8_core beat -l info`
5. Run smoke tests against staging API 5. Run smoke tests against staging API
6. Manual QA: test all UI flows 6. Manual QA: test all UI flows
7. Monitor logs for errors 7. Monitor logs for errors