Files
igny8/part2-dev/IGNY8-HOLISTIC-ARCHITECTURE-PLAN.md
alorig 8040d983de asd
2025-11-18 03:32:36 +05:00

1026 lines
44 KiB
Markdown

# IGNY8 HOLISTIC ARCHITECTURE PLAN
**Complete System Architecture, Implementation Strategy, and Structure**
**Last Updated:** 2025-01-XX
**Purpose:** Comprehensive architecture plan combining existing system, Phase 2 requirements, automation, multi-site management, and best practices.
---
## TABLE OF CONTENTS
1. [Architecture Overview](#1-architecture-overview)
2. [Container Architecture](#2-container-architecture)
3. [Backend Layer Structure](#3-backend-layer-structure)
4. [Frontend Structure](#4-frontend-structure)
5. [Automation System](#5-automation-system)
6. [Multi-Site Management](#6-multi-site-management)
7. [Volume & Storage Strategy](#7-volume--storage-strategy)
8. [Repository Structure](#8-repository-structure)
9. [Implementation Phases](#9-implementation-phases)
10. [Limits & Boundaries](#10-limits--boundaries)
---
## 1. ARCHITECTURE OVERVIEW
### 1.1 Core Principles
| Principle | Description | Implementation |
|-----------|-------------|----------------|
| **Domain-Driven Design** | Organize by business domains, not technical layers | `business/` folder with content, planning, linking, optimization, publishing domains |
| **Service Layer Pattern** | Business logic in services, not ViewSets | All modules delegate to domain services |
| **Single Responsibility** | Each layer has one clear purpose | Core → Domain → Module → Infrastructure |
| **No Duplication** | Reuse services across modules | ContentGenerationService used by Writer + Site Builder |
| **Multi-Tenancy First** | All features respect account/site/sector boundaries | All models inherit AccountBaseModel or SiteSectorBaseModel |
| **Automation Ready** | All functions can be automated via Celery | All AI functions exposed as Celery tasks with scheduling support |
### 1.2 System Boundaries
| System | Purpose | Technology | Container |
|--------|---------|------------|-----------|
| **IGNY8 Core App** | Main backend API (all domains) | Django 5.2+ | `igny8_backend` |
| **IGNY8 Main App** | Dashboard UI | React 19 + Vite | `igny8_frontend` |
| **IGNY8 Site Builder** | Site creation wizard | React 19 + Vite | `igny8_site_builder` (NEW) |
| **IGNY8 Sites** | Public site renderer | React 19 + Vite/Next.js | `igny8_sites` (NEW) |
| **IGNY8 Marketing** | Marketing website | React 19 + Vite | `igny8_marketing_dev` (can merge with Sites) |
| **Celery Workers** | Async task processing | Celery + Redis | `igny8_celery_worker` |
| **Celery Beat** | Scheduled automation | Celery Beat | `igny8_celery_beat` |
---
## 2. CONTAINER ARCHITECTURE
### 2.1 Container Structure
| Container | Port | Purpose | Volumes | Dependencies |
|-----------|------|---------|---------|--------------|
| `igny8_backend` | 8011:8010 | Django API server | `/data/app/igny8/backend:/app`, `/data/app/logs:/app/logs`, `/data/app/storage:/app/storage` | postgres, redis |
| `igny8_frontend` | 8021:5173 | Main app UI (dev) | `/data/app/igny8/frontend:/app` | igny8_backend |
| `igny8_site_builder` | 8022:5175 | Site Builder UI (NEW) | `/data/app/igny8/site-builder:/app` | igny8_backend |
| `igny8_sites` | 8024:5176 | Sites renderer (NEW) | `/data/app/igny8/sites:/app`, `/data/app/sites-data:/sites` | igny8_backend |
| `igny8_marketing_dev` | 8023:5174 | Marketing site | `/data/app/igny8/frontend:/app` | igny8_backend |
| `igny8_celery_worker` | - | Async tasks | `/data/app/igny8/backend:/app`, `/data/app/logs:/app/logs` | postgres, redis |
| `igny8_celery_beat` | - | Scheduled tasks | `/data/app/igny8/backend:/app`, `/data/app/logs:/app/logs` | postgres, redis |
### 2.2 Volume Strategy
| Volume Path | Purpose | Mount Type | Backup Strategy |
|-------------|---------|------------|-----------------|
| `/data/app/igny8/backend` | Backend code (dev) | Bind mount (rw) | Git repo |
| `/data/app/igny8/frontend` | Frontend code (dev) | Bind mount (rw) | Git repo |
| `/data/app/igny8/site-builder` | Site Builder code (NEW) | Bind mount (rw) | Git repo |
| `/data/app/igny8/sites` | Sites renderer code (NEW) | Bind mount (rw) | Git repo |
| `/data/app/storage` | Generated files, images | Named volume | Daily backup |
| `/data/app/sites-data` | Rendered site files (NEW) | Named volume | Daily backup |
| `/data/app/logs` | Application logs | Named volume | Rotate weekly |
| `/data/app/cache` | Redis cache (optional) | Named volume | No backup |
**Volume Improvements:**
- Use named volumes for data (storage, sites-data, logs) - better performance, easier backup
- Use bind mounts only for code (development)
- Production: Use named volumes for all, code deployed via images
---
## 3. BACKEND LAYER STRUCTURE
### 3.1 Complete Backend Structure
```
backend/igny8_core/
├── core/ # CORE LAYER (Foundation)
│ ├── auth/ # Multi-tenancy, users, accounts
│ │ ├── models.py # Account, User, Plan, Site, Sector
│ │ ├── serializers.py
│ │ ├── views.py
│ │ └── urls.py
│ │
│ ├── base/ # BASE CLASSES (Shared)
│ │ ├── models.py # AccountBaseModel, SiteSectorBaseModel
│ │ ├── viewsets.py # AccountModelViewSet, SiteSectorModelViewSet
│ │ └── serializers.py # Base serializers
│ │
│ └── api/ # API INFRASTRUCTURE
│ ├── base.py # Base ViewSets
│ ├── response.py # success_response, error_response
│ ├── exceptions.py # Unified exception handler
│ ├── pagination.py # CustomPageNumberPagination
│ ├── permissions.py # IsAuthenticatedAndActive, HasTenantAccess
│ └── throttles.py # DebugScopedRateThrottle
├── business/ # BUSINESS LAYER (Business Logic)
│ ├── content/ # Content domain
│ │ ├── models.py # Content, Tasks, Images (unified, extended)
│ │ ├── services/
│ │ │ ├── content_generation_service.py # Unified generation
│ │ │ ├── content_pipeline_service.py # Pipeline orchestration
│ │ │ └── content_versioning_service.py # Version management
│ │ └── repositories.py # ContentRepository (if needed)
│ │
│ ├── planning/ # Planning domain
│ │ ├── models.py # Keywords, Clusters, Ideas
│ │ └── services/
│ │ ├── clustering_service.py
│ │ └── ideas_service.py
│ │
│ ├── linking/ # Linking domain (NEW)
│ │ ├── models.py # InternalLinks, LinkGraph
│ │ └── services/
│ │ └── linker_service.py
│ │
│ ├── optimization/ # Optimization domain (NEW)
│ │ ├── models.py # OptimizationTask, OptimizationScores
│ │ └── services/
│ │ └── optimizer_service.py
│ │
│ ├── publishing/ # Publishing domain (NEW)
│ │ ├── models.py # PublishingRecord, DeploymentRecord
│ │ └── services/
│ │ ├── publisher_service.py
│ │ ├── adapters/
│ │ │ ├── base_adapter.py
│ │ │ ├── wordpress_adapter.py
│ │ │ └── sites_renderer_adapter.py
│ │ └── deployment_service.py
│ │
│ ├── site_building/ # Site Building domain (NEW)
│ │ ├── models.py # SiteBlueprint, PageBlueprint
│ │ └── services/
│ │ ├── structure_generation_service.py
│ │ └── site_deployment_service.py
│ │
│ └── billing/ # Billing domain
│ ├── models.py # Credits, Transactions
│ └── services/
│ └── credit_service.py
├── modules/ # MODULE LAYER (API/UI Interface)
│ ├── planner/ # Planner API endpoints
│ │ ├── views.py # ViewSets (thin, delegates to services)
│ │ ├── serializers.py # Request/Response serializers
│ │ └── urls.py
│ │
│ ├── writer/ # Writer API endpoints (legacy name)
│ │ ├── views.py
│ │ ├── serializers.py
│ │ └── urls.py
│ │
│ ├── content/ # Unified content API (NEW)
│ │ ├── views.py
│ │ ├── serializers.py
│ │ └── urls.py
│ │
│ ├── site_builder/ # Site Builder API (NEW)
│ │ ├── views.py
│ │ ├── serializers.py
│ │ └── urls.py
│ │
│ ├── linker/ # Linker API (NEW)
│ │ ├── views.py
│ │ └── urls.py
│ │
│ ├── optimizer/ # Optimizer API (NEW)
│ │ ├── views.py
│ │ └── urls.py
│ │
│ ├── publisher/ # Publisher API (NEW)
│ │ ├── views.py
│ │ └── urls.py
│ │
│ ├── integration/ # Site Integration API (NEW)
│ │ ├── views.py
│ │ └── urls.py
│ │
│ ├── automation/ # Automation API (NEW)
│ │ ├── views.py
│ │ ├── serializers.py
│ │ └── urls.py
│ │
│ ├── system/ # System API
│ │ ├── views.py
│ │ └── urls.py
│ │
│ └── billing/ # Billing API
│ ├── views.py
│ └── urls.py
├── infrastructure/ # INFRASTRUCTURE LAYER
│ ├── ai/ # AI Framework (existing)
│ │ ├── engine.py
│ │ ├── functions/
│ │ │ ├── auto_cluster.py
│ │ │ ├── generate_ideas.py
│ │ │ ├── generate_content.py
│ │ │ ├── generate_image_prompts.py
│ │ │ ├── generate_images.py
│ │ │ ├── generate_site_structure.py # NEW for Site Builder
│ │ │ └── optimize_content.py # NEW for Optimizer
│ │ ├── registry.py
│ │ └── tasks.py # run_ai_task
│ │
│ ├── external/ # External integrations
│ │ ├── wordpress.py # WordPress client
│ │ ├── sites_renderer.py # Sites renderer client (NEW)
│ │ └── shopify.py # Shopify client (Future)
│ │
│ ├── storage/ # Storage abstractions
│ │ └── file_storage.py
│ │
│ └── messaging/ # Async messaging
│ ├── celery_tasks.py # Celery task definitions
│ └── automation_tasks.py # Automation tasks (NEW)
└── shared/ # SHARED LAYER
├── schemas/ # Shared schemas
│ ├── blocks.py # Block component schema
│ └── content_types.py # Content type definitions
├── utils/ # Shared utilities
│ ├── content_normalizer.py
│ └── queue_manager.py
└── middleware/ # Middleware
├── account.py # AccountContextMiddleware
└── request_id.py # RequestIDMiddleware
```
### 3.2 Model Extensions
| Model | Current Location | New Location | Extensions Needed |
|-------|------------------|-------------|-------------------|
| `Content` | `modules/writer/models.py` | `business/content/models.py` | Add: `entity_type`, `json_blocks`, `structure_data`, `linker_version`, `optimizer_version`, `internal_links`, `optimization_scores`, `published_to`, `external_ids`, `source`, `sync_status`, `external_id`, `external_url`, `sync_metadata` |
| `Tasks` | `modules/writer/models.py` | `business/content/models.py` | Add: `entity_type` choices (product, service, taxonomy, etc.) |
| `Keywords` | `modules/planner/models.py` | `business/planning/models.py` | No changes |
| `Clusters` | `modules/planner/models.py` | `business/planning/models.py` | No changes |
| `ContentIdeas` | `modules/planner/models.py` | `business/planning/models.py` | Add: `entity_type` support |
| `InternalLinks` | - | `business/linking/models.py` | NEW: `source_id`, `target_id`, `anchor`, `position`, `link_type` |
| `OptimizationTask` | - | `business/optimization/models.py` | NEW: `content_id`, `type`, `target_keyword`, `scores_before`, `scores_after`, `html_before`, `html_after` |
| `SiteBlueprint` | - | `business/site_building/models.py` | NEW: `tenant`, `site`, `config_json`, `structure_json`, `status`, `hosting_type` |
| `PageBlueprint` | - | `business/site_building/models.py` | NEW: `site_blueprint`, `slug`, `type`, `blocks_json`, `status` |
| `SiteIntegration` | - | `business/integration/models.py` | NEW: `site`, `platform`, `platform_type`, `config_json`, `credentials`, `is_active`, `sync_enabled` |
| `PublishingRecord` | - | `business/publishing/models.py` | NEW: `content_id`, `destination`, `destination_type`, `status`, `external_id`, `published_at`, `sync_status` |
| `DeploymentRecord` | - | `business/publishing/models.py` | NEW: `site_blueprint`, `version`, `status`, `build_url`, `deployed_at`, `deployment_type` |
| `AutomationRule` | - | `business/automation/models.py` | NEW: `name`, `trigger`, `conditions`, `actions`, `schedule`, `is_active` |
| `ScheduledTask` | - | `business/automation/models.py` | NEW: `automation_rule`, `scheduled_at`, `status`, `executed_at` |
---
## 4. FRONTEND STRUCTURE
### 4.1 Main App Structure (igny8_frontend)
```
frontend/src/
├── pages/
│ ├── Planner/ # Existing
│ ├── Writer/ # Existing
│ ├── Thinker/ # Existing
│ ├── Billing/ # Existing
│ ├── Settings/ # Existing
│ ├── Automation/ # EXISTING (placeholder) - IMPLEMENT
│ │ ├── Dashboard.tsx # Automation overview (includes schedules functionality)
│ │ ├── Rules.tsx # Automation rules management
│ │ ├── Workflows.tsx # Workflow templates
│ │ └── History.tsx # Automation execution history
│ ├── Linker/ # NEW
│ │ ├── Dashboard.tsx
│ │ ├── Candidates.tsx
│ │ └── Graph.tsx
│ ├── Optimizer/ # NEW
│ │ ├── Dashboard.tsx
│ │ ├── Analyze.tsx
│ │ └── Compare.tsx
│ └── SiteBuilder/ # NEW (if integrated, else separate container)
├── modules/ # NEW: Feature modules
│ ├── automation/
│ │ ├── components/
│ │ ├── hooks/
│ │ └── stores/
│ ├── linker/
│ ├── optimizer/
│ └── publisher/
└── services/
├── automation.api.ts # NEW
├── linker.api.ts # NEW
├── optimizer.api.ts # NEW
└── publisher.api.ts # NEW
```
### 4.2 Site Builder Structure (NEW Container)
```
site-builder/src/
├── pages/
│ ├── wizard/ # Wizard steps
│ │ ├── Step1TypeSelection.tsx
│ │ ├── Step2BusinessBrief.tsx
│ │ ├── Step3Objectives.tsx
│ │ └── Step4Style.tsx
│ ├── preview/ # Preview renderer
│ │ └── PreviewCanvas.tsx
│ └── dashboard/ # Site management
│ └── SiteList.tsx
├── components/
│ ├── blocks/ # Block components
│ ├── forms/ # Form components
│ └── preview-canvas/ # Preview renderer
├── state/
│ ├── builderStore.ts # Zustand store
│ └── siteDefinitionStore.ts
└── api/
├── builder.api.ts
└── sites.api.ts
```
### 4.3 Sites Renderer Structure (NEW Container)
```
sites/src/
├── renderer/
│ ├── index.html
│ ├── main.tsx
│ ├── router.tsx
│ └── components/
│ ├── Layout/
│ ├── Blocks/ # Block components (shared schema)
│ └── UI/
├── loaders/
│ └── loadSiteDefinition.ts
└── config/
├── themes/
└── block-schema/ # Shared with WP theme
```
---
## 5. AUTOMATION SYSTEM
### 5.1 Automation Architecture
| Component | Purpose | Implementation |
|-----------|---------|----------------|
| **AutomationRule Model** | Store automation rules | `business/automation/models.py` |
| **AutomationService** | Execute automation rules | `business/automation/services/automation_service.py` |
| **Celery Beat Tasks** | Scheduled automation | `infrastructure/messaging/automation_tasks.py` |
| **Automation API** | CRUD for rules | `modules/automation/views.py` |
| **Automation UI** | Manage rules | `frontend/src/pages/Automation/` |
### 5.2 Automation Rule Structure
```python
class AutomationRule(SiteSectorBaseModel):
name = models.CharField(max_length=255)
trigger = models.CharField(choices=[
('schedule', 'Scheduled'),
('keyword_added', 'Keyword Added'),
('cluster_created', 'Cluster Created'),
('idea_created', 'Idea Created'),
('content_generated', 'Content Generated'),
])
conditions = models.JSONField() # {field: value, operator: 'eq', 'gt', etc.}
actions = models.JSONField() # [{type: 'generate_ideas', params: {...}}]
schedule = models.JSONField() # {cron: '0 9 * * *', timezone: 'UTC'}
is_active = models.BooleanField(default=True)
max_executions_per_day = models.IntegerField(default=10)
credit_limit_per_execution = models.IntegerField(default=100)
```
### 5.3 Automation Workflows
| Workflow | Trigger | Actions | Schedule |
|----------|---------|---------|----------|
| **Auto Cluster Keywords** | New keywords added | Run clustering AI | Daily at 2 AM |
| **Auto Generate Ideas** | Cluster created | Generate ideas for cluster | Immediate or scheduled |
| **Auto Create Tasks** | Ideas created | Create tasks from ideas | Daily at 9 AM |
| **Auto Generate Content** | Tasks queued | Generate content for tasks | Hourly (respects daily limit) |
| **Auto Generate Images** | Content generated | Generate images for content | Immediate |
| **Auto Link Content** | Content generated | Run linker on content | After content generation |
| **Auto Optimize Content** | Content linked | Run optimizer on content | After linking |
| **Auto Publish** | Content optimized | Publish to WordPress/Sites | Daily at 10 AM |
### 5.4 Automation Implementation
| Task | File | Implementation |
|------|------|----------------|
| **AutomationRule Model** | `business/automation/models.py` | Create model with trigger, conditions, actions, schedule |
| **AutomationService** | `business/automation/services/automation_service.py` | `execute_rule()`, `check_conditions()`, `execute_actions()` |
| **Celery Beat Tasks** | `infrastructure/messaging/automation_tasks.py` | `@periodic_task` decorators for scheduled rules |
| **Automation API** | `modules/automation/views.py` | CRUD ViewSet for AutomationRule |
| **Automation UI** | `frontend/src/pages/Automation/` | Dashboard, Rules management, History |
---
## 6. MULTI-SITE MANAGEMENT
### 6.1 Site Isolation Strategy
| Aspect | Implementation | Purpose |
|--------|----------------|---------|
| **Database Filtering** | All models inherit AccountBaseModel/SiteSectorBaseModel | Automatic site isolation |
| **API Filtering** | ViewSets use base classes | Automatic filtering by site_id |
| **Storage Isolation** | `/data/app/storage/{site_id}/` | Separate storage per site |
| **Site-Specific Limits** | Plan limits enforced per site | Prevent one site from overwhelming system |
| **Queue Isolation** | Celery queues per site (optional) | Prevent site blocking |
| **Rate Limiting** | Per-site rate limits | Prevent abuse |
### 6.2 Site Resource Management
| Resource | Limit Type | Enforcement | Location |
|-----------|------------|-------------|----------|
| **Daily Content Tasks** | Per site | `Plan.daily_content_tasks` | `business/content/services/content_generation_service.py` |
| **Daily AI Requests** | Per site | `Plan.daily_ai_requests` | `infrastructure/ai/engine.py` |
| **Monthly Word Count** | Per site | `Plan.monthly_word_count_limit` | `business/content/services/content_generation_service.py` |
| **Daily Image Generation** | Per site | `Plan.daily_image_generation_limit` | `infrastructure/ai/functions/generate_images.py` |
| **Storage Quota** | Per site | Configurable (default: 10GB) | `infrastructure/storage/file_storage.py` |
| **Concurrent Tasks** | Per site | Configurable (default: 5) | Celery queue configuration |
### 6.3 Site Performance Optimization
| Optimization | Implementation | Benefit |
|---------------|----------------|---------|
| **Database Indexing** | Indexes on `site_id`, `sector_id`, `account_id` | Faster queries |
| **Query Optimization** | `select_related()`, `prefetch_related()` | Reduce DB queries |
| **Caching** | Redis cache per site | Faster responses |
| **Background Processing** | Celery for heavy operations | Non-blocking |
| **Resource Throttling** | Per-site rate limits | Prevent overload |
---
## 7. VOLUME & STORAGE STRATEGY
### 7.1 Volume Structure
```
/data/app/
├── igny8/ # Code (bind mount, git repo)
│ ├── backend/
│ ├── frontend/
│ ├── site-builder/ # NEW
│ └── sites/ # NEW
├── storage/ # Named volume (data)
│ ├── {site_id}/
│ │ ├── images/
│ │ ├── content/
│ │ └── exports/
│ └── shared/
│ └── templates/
├── sites-data/ # Named volume (NEW - rendered sites)
│ ├── marketing/
│ └── clients/
│ ├── {site_id}/
│ │ └── v{version}/
│ │ ├── site.json
│ │ ├── pages/
│ │ └── assets/
├── logs/ # Named volume
│ ├── backend/
│ ├── celery/
│ └── nginx/
└── cache/ # Named volume (optional)
└── redis/
```
### 7.2 Volume Mounting Strategy
| Volume | Type | Purpose | Backup |
|--------|------|---------|--------|
| **Code directories** | Bind mount (dev) / Image (prod) | Source code | Git repo |
| **storage/** | Named volume | Generated files, images | Daily backup |
| **sites-data/** | Named volume | Rendered site files | Daily backup |
| **logs/** | Named volume | Application logs | Weekly rotation |
| **cache/** | Named volume | Redis cache | No backup |
### 7.3 Storage Service Implementation
```python
# infrastructure/storage/file_storage.py
class FileStorageService:
def get_site_storage_path(self, site_id: int) -> str:
return f"/data/app/storage/{site_id}/"
def get_site_quota(self, site_id: int) -> int:
# Get from Plan or Site model
return site.plan.storage_quota_gb * 1024 * 1024 * 1024
def check_quota(self, site_id: int, file_size: int) -> bool:
current_size = self.get_site_storage_size(site_id)
quota = self.get_site_quota(site_id)
return (current_size + file_size) <= quota
```
### 7.4 Sites Folder Access for Site Builder
**User-Level File Management**:
- Each user has access to their account's sites folder: `/data/app/sites-data/clients/{site_id}/`
- Site Builder users can manage files for sites they have access to (via `SiteUserAccess`)
- File operations (upload, delete, organize) scoped to user's accessible sites
- Storage quota enforced per site (from Plan)
- File permissions: Owner/Admin (full access), Editor (upload/edit), Viewer (read-only)
**Site Builder File Management**:
```python
# business/site_building/services/file_management_service.py
class SiteBuilderFileService:
def get_user_accessible_sites(self, user) -> List[Site]:
"""Get sites user can access for file management"""
return user.get_accessible_sites()
def get_site_files_path(self, site_id: int) -> str:
"""Get site's files directory"""
return f"/data/app/sites-data/clients/{site_id}/v{version}/assets/"
def check_file_access(self, user, site_id: int) -> bool:
"""Check if user can access site's files"""
accessible_sites = self.get_user_accessible_sites(user)
return any(site.id == site_id for site in accessible_sites)
def upload_file(self, user, site_id: int, file) -> str:
"""Upload file to site's assets folder"""
if not self.check_file_access(user, site_id):
raise PermissionDenied("No access to this site")
# Upload logic
```
---
## 8. REPOSITORY STRUCTURE
### 8.1 Main Repository Structure
```
igny8/ # Main Git repository
├── backend/ # Django backend
├── frontend/ # React frontend (main app + marketing)
├── site-builder/ # Site Builder (NEW - separate or subfolder)
├── sites/ # Sites renderer (NEW - separate or subfolder)
├── docs/ # Documentation
├── docker-compose.app.yml # App stack
├── .gitignore # Excludes: node_modules, __pycache__, .env, logs, storage, sites-data
└── README.md
```
### 8.2 Branching Strategy
| Branch | Purpose | Merge To | Protection |
|--------|---------|----------|------------|
| `main` | Production-ready code | - | Protected, requires PR |
| `develop` | Integration branch | `main` | Protected, requires PR |
| `feature/*` | Feature branches | `develop` | Not protected |
| `phase-2/*` | Phase 2 features | `develop` | Not protected |
| `hotfix/*` | Critical fixes | `main`, `develop` | Not protected |
| `release/*` | Release candidates | `main` | Protected |
### 8.3 Repository Exclusions (.gitignore)
```
# Code dependencies
node_modules/
**/node_modules/
__pycache__/
**/__pycache__/
*.pyc
*.pyo
# Build outputs
dist/
build/
.vite/
*.tsbuildinfo
# Environment
.env
.env.local
.env.*.local
# Data (NEVER commit)
/data/app/storage/
/data/app/sites-data/
/data/app/logs/
/data/app/cache/
# Generated files
*.log
*.tmp
*.temp
# IDE
.vscode/
.idea/
*.swp
*.swo
# Docker (if any)
docker-data/
.docker/
```
### 8.4 Separate Repositories (Optional)
| Repository | Contents | Purpose |
|------------|----------|---------|
| `igny8-core` | Backend + Main frontend | Core platform |
| `igny8-site-builder` | Site Builder code | Separate deployment |
| `igny8-sites` | Sites renderer | Separate deployment |
| `igny8-wp-plugin` | WordPress plugin | External (separate) |
| `igny8-wp-theme` | WordPress theme | External (separate) |
**Recommendation:** Keep all in one repo (`igny8`) with clear folder structure. Separate repos only if deployment cycles differ significantly.
---
## 9. IMPLEMENTATION PHASES
### 9.1 Phase 0: Foundation (Current + Immediate)
| Task | Files | Status | Priority |
|------|-------|--------|-----------|
| **Extend Content Model** | `business/content/models.py` | TODO | HIGH |
| **Create Service Layer** | `business/*/services/` | TODO | HIGH |
| **Refactor ViewSets** | `modules/*/views.py` | TODO | HIGH |
| **Implement Automation Models** | `business/automation/models.py` | TODO | HIGH |
| **Implement Automation Service** | `business/automation/services/` | TODO | HIGH |
| **Implement Automation API** | `modules/automation/` | TODO | HIGH |
| **Implement Automation UI** | `frontend/src/pages/Automation/` | TODO | HIGH |
| **Note**: Schedules functionality will be integrated into Automation UI, not as a separate page | - | - | - |
### 9.2 Phase 1: Site Builder
| Task | Files | Dependencies | Priority |
|------|-------|--------------|----------|
| **Create Site Builder Container** | `docker-compose.app.yml` | Phase 0 | HIGH |
| **Site Builder Models** | `business/site_building/models.py` | Phase 0 | HIGH |
| **Structure Generation Service** | `business/site_building/services/` | Phase 0 | HIGH |
| **Structure Generation AI Function** | `infrastructure/ai/functions/generate_site_structure.py` | Phase 0 | HIGH |
| **Site Builder API** | `modules/site_builder/` | Phase 0 | HIGH |
| **Site Builder Frontend** | `site-builder/src/` | Phase 0 | HIGH |
### 9.3 Phase 2: Linker & Optimizer
| Task | Files | Dependencies | Priority |
|------|-------|--------------|----------|
| **Linker Models** | `business/linking/models.py` | Phase 0 | MEDIUM |
| **Linker Service** | `business/linking/services/` | Phase 0 | MEDIUM |
| **Linker API** | `modules/linker/` | Phase 0 | MEDIUM |
| **Linker UI** | `frontend/src/pages/Linker/` | Phase 0 | MEDIUM |
| **Optimizer Models** | `business/optimization/models.py` | Phase 0 | MEDIUM |
| **Optimizer Service** | `business/optimization/services/` | Phase 0 | MEDIUM |
| **Optimizer AI Function** | `infrastructure/ai/functions/optimize_content.py` | Phase 0 | MEDIUM |
| **Optimizer API** | `modules/optimizer/` | Phase 0 | MEDIUM |
| **Optimizer UI** | `frontend/src/pages/Optimizer/` | Phase 0 | MEDIUM |
### 9.4 Phase 3: Sites Renderer
| Task | Files | Dependencies | Priority |
|------|-------|--------------|----------|
| **Create Sites Container** | `docker-compose.app.yml` | Phase 1 | MEDIUM |
| **Sites Renderer Frontend** | `sites/src/` | Phase 1 | MEDIUM |
| **Publisher Service** | `business/publishing/services/` | Phase 0 | MEDIUM |
| **Sites Renderer Adapter** | `business/publishing/services/adapters/` | Phase 1 | MEDIUM |
| **Publisher API** | `modules/publisher/` | Phase 0 | MEDIUM |
| **Deployment Service** | `business/publishing/services/deployment_service.py` | Phase 1 | MEDIUM |
### 9.5 Phase 4: Universal Content Types
| Task | Files | Dependencies | Priority |
|------|-------|--------------|----------|
| **Extend Content Model** | `business/content/models.py` | Phase 0 | LOW |
| **Product Content Prompts** | `infrastructure/ai/prompts.py` | Phase 0 | LOW |
| **Service Page Prompts** | `infrastructure/ai/prompts.py` | Phase 0 | LOW |
| **Taxonomy Prompts** | `infrastructure/ai/prompts.py` | Phase 0 | LOW |
| **Content Type Support in Writer** | `business/content/services/` | Phase 0 | LOW |
| **Content Type Support in Linker** | `business/linking/services/` | Phase 2 | LOW |
| **Content Type Support in Optimizer** | `business/optimization/services/` | Phase 2 | LOW |
---
## 10. CREDIT-BASED USAGE SYSTEM
### 10.1 Core Principle
**Credit-Only Model**: All features are unlimited. Only credits restrict usage.
- ✅ All features unlocked for paid users
- ✅ No plan-based limits (keywords, clusters, tasks, etc.)
- ✅ Only credits control usage
- ✅ Operational limits only (50 keywords/request, 6 images/request)
### 10.2 Plan Model (Simplified)
| Field | Type | Purpose |
|-------|------|---------|
| `monthly_credits` | Integer | Credits added per month |
| `support_level` | CharField | Support tier (basic, priority, enterprise) |
| `billing_cycle` | CharField | 'monthly' or 'annual' |
| `price` | Decimal | Subscription price |
| `features` | JSONField | Feature flags (for future use) |
**REMOVED FIELDS** (no longer needed):
- `max_keywords`, `max_clusters`, `max_content_ideas`
- `daily_content_tasks`, `monthly_word_count_limit`
- `daily_image_generation_limit`, `monthly_image_count`
- All other usage limits
### 10.3 Credit Cost Structure
| Operation | Credit Cost | Type | Notes |
|-----------|-------------|------|-------|
| **Clustering** | 10 credits | Fixed | Per clustering request |
| **Idea Generation** | 15 credits | Fixed | Per cluster → ideas request |
| **Content Generation** | 1 credit per 100 words | Variable | Based on word count |
| **Image Generation** | 5 credits | Fixed | Per image (user sees batch) |
| **Image Prompt Extraction** | 2 credits | Fixed | Per content piece |
| **Linking** | 8 credits | Fixed | Per content piece (NEW) |
| **Optimization** | 1 credit per 200 words | Variable | Per content piece (NEW) |
| **Site Structure Generation** | 50 credits | Fixed | Per site blueprint (NEW) |
| **Site Page Generation** | 20 credits | Fixed | Per page (NEW) |
### 10.4 Credit Enforcement
| Location | Check | Implementation |
|----------|-------|----------------|
| **AI Engine** | Before AI call | `infrastructure/ai/engine.py` - Check credits, deduct before request |
| **Content Generation** | Before generation | `business/content/services/content_generation_service.py` |
| **Image Generation** | Before generation | `infrastructure/ai/functions/generate_images.py` |
| **Linking** | Before linking | `business/linking/services/linker_service.py` (NEW) |
| **Optimization** | Before optimization | `business/optimization/services/optimizer_service.py` (NEW) |
| **Site Building** | Before structure gen | `business/site_building/services/structure_generation_service.py` (NEW) |
### 10.5 Credit Logging
| Log Type | Fields | Purpose |
|----------|--------|---------|
| **CreditUsageLog** | `account`, `operation_type`, `credits_used`, `related_object_type`, `related_object_id`, `metadata` | Track all credit consumption |
| **CreditTransaction** | `account`, `transaction_type`, `amount`, `description` | Track credit additions/deductions |
**Usage Tracking**:
- Total credits used (account level)
- Credits per operation type (clustering, content, images, etc.)
- Credits per site (site level breakdown)
- Credits per user (user level breakdown)
- Historical usage trends
### 10.6 Operational Limits (Technical Constraints)
| Limit | Value | Type | Reason |
|-------|-------|-------|--------|
| **Keywords per request** | 50 | Technical | API payload size, processing time |
| **Images per request** | 6 | Technical | Queue management (user sees as batch) |
| **Images per AI call** | 1 | Technical | Image API limitation (internal) |
These are **NOT** business limits - they're technical constraints for request processing.
### 10.7 Credit System Features
| Feature | Implementation | Location |
|---------|----------------|----------|
| **Credit Check** | Before every AI operation | `business/billing/services/credit_service.py` |
| **Credit Deduction** | After successful operation | `business/billing/services/credit_service.py` |
| **Credit Top-up** | On-demand purchase | `modules/billing/views.py` |
| **Monthly Replenishment** | Celery Beat task | `infrastructure/messaging/automation_tasks.py` |
| **Low Credit Warning** | When < 10% remaining | Frontend + Email notification |
| **Usage Dashboard** | Show credits by operation type | `frontend/src/pages/Billing/Usage.tsx` |
### 10.8 Admin Boundaries
| Boundary | Implementation | Location |
|----------|----------------|----------|
| **System Account** | `Account.is_system_account()` | `core/auth/models.py` |
| **Superuser Access** | `User.is_superuser` | `core/auth/models.py` |
| **Plan Management** | Admin-only ViewSet | `modules/system/views.py` |
| **User Management** | Role-based permissions | `core/api/permissions.py` |
| **Credit Management** | Admin-only actions | `modules/billing/views.py` |
| **Automation Management** | Role-based (Owner/Admin) | `modules/automation/views.py` |
### 10.9 Multi-Tenancy Boundaries
| Boundary | Implementation | Location |
|----------|----------------|----------|
| **Account Isolation** | `AccountBaseModel` | `core/base/models.py` |
| **Site Isolation** | `SiteSectorBaseModel` | `core/base/models.py` |
| **Sector Isolation** | `SiteSectorBaseModel` | `core/base/models.py` |
| **API Filtering** | Base ViewSets | `core/api/base.py` |
| **Storage Isolation** | Site-specific paths | `infrastructure/storage/file_storage.py` |
| **Queue Isolation** | Site-specific queues (optional) | Celery config |
---
## 11. SITE INTEGRATION & MULTI-DESTINATION PUBLISHING
### 11.1 Site Integration Model
| Field | Type | Purpose |
|-------|------|---------|
| `site` | ForeignKey(Site) | Links to Site model |
| `platform` | CharField | Platform name ('wordpress', 'shopify', 'custom') |
| `platform_type` | CharField | Type: 'cms', 'ecommerce', 'custom_api' |
| `config_json` | JSONField | Platform-specific configuration |
| `credentials` | EncryptedField | Encrypted API keys, tokens, passwords |
| `is_active` | Boolean | Enable/disable integration |
| `sync_enabled` | Boolean | Enable two-way sync |
| `last_sync_at` | DateTime | Last successful sync |
| `sync_status` | CharField | 'success', 'failed', 'pending' |
**Purpose**:
- Sites can be NEW (built via Site Builder) or EXISTING (already integrated)
- One site can have multiple integrations (WordPress + Shopify)
- Integration credentials stored securely
- Two-way sync support
### 11.2 Site Types & Hosting
| Site Type | Description | Integration | Hosting |
|-----------|-------------|-------------|---------|
| **NEW Site (IGNY8 Built)** | Created via Site Builder | None initially | IGNY8 Sites renderer |
| **EXISTING WordPress** | Already has WordPress site | WordPress integration | WordPress hosting |
| **EXISTING Shopify** | Already has Shopify store | Shopify integration | Shopify hosting |
| **EXISTING Custom** | Custom platform/API | Custom integration | External hosting |
| **Hybrid** | Built in IGNY8, also integrated | Multiple integrations | IGNY8 + External |
### 11.3 Site Builder Structure
```
Site Builder Flow:
1. User creates SiteBlueprint (via wizard)
2. SiteBlueprint.hosting_type = 'igny8_sites' | 'wordpress' | 'shopify' | 'multi'
3. If 'wordpress' or 'shopify':
- User must configure SiteIntegration first
- Site Builder generates content compatible with platform
4. If 'igny8_sites':
- Site deployed to Sites renderer
5. If 'multi':
- Site can publish to multiple destinations
```
### 11.4 Multi-Destination Publishing
| Content Type | Can Publish To | Implementation |
|--------------|----------------|----------------|
| **Blog Posts** | WordPress, IGNY8 Sites, Shopify Blog | PublisherService routes to adapters |
| **Pages** | WordPress, IGNY8 Sites, Shopify Pages | PublisherService routes to adapters |
| **Products** | Shopify, WooCommerce (WordPress), IGNY8 Sites | Product adapter per platform |
| **Categories** | WordPress, Shopify Collections, IGNY8 Sites | Taxonomy adapter per platform |
| **Site Blueprint** | IGNY8 Sites, WordPress (via plugin), Shopify (future) | DeploymentService routes to adapters |
**Publishing Flow:**
```python
# Single content can publish to multiple destinations
publisher_service.publish(
content=content,
destinations=['wordpress', 'igny8_sites', 'shopify'] # Multiple destinations
)
```
### 11.5 Integration Adapters
| Adapter | Platform | Purpose | Implementation |
|---------|----------|---------|-----------------|
| **WordPressAdapter** | WordPress | Publish content, sync status | Uses WordPress REST API |
| **SitesRendererAdapter** | IGNY8 Sites | Deploy site blueprints | Uploads to Sites container |
| **ShopifyAdapter** | Shopify | Publish products, pages, blog | Uses Shopify Admin API |
| **BaseAdapter** | Abstract | Common interface | All adapters inherit |
### 11.6 Site Integration Service
```python
# business/integration/services/integration_service.py
class IntegrationService:
def create_integration(self, site, platform, config, credentials):
"""Create new site integration"""
def test_integration(self, integration):
"""Test integration connection"""
def sync_site_data(self, integration, sync_type='full'):
"""Sync site data from external platform"""
# Full: All posts/products
# Incremental: Only changes since last sync
def get_available_platforms(self):
"""Get list of supported platforms"""
return ['wordpress', 'shopify', 'custom_api']
```
### 11.7 Site Model Extensions
| Field | Type | Purpose |
|-------|------|---------|
| `site_type` | CharField | 'new', 'existing', 'hybrid' |
| `hosting_type` | CharField | 'igny8_sites', 'wordpress', 'shopify', 'custom', 'multi' |
| `integrations` | ManyToMany(SiteIntegration) | Multiple integrations per site |
**Current Site Model** (needs extension):
- Already has: `wp_url`, `wp_username`, `wp_app_password` (WordPress specific)
- **NEW**: Move to `SiteIntegration` model for flexibility
- **NEW**: Add `site_type` and `hosting_type` fields
### 11.8 Publishing Workflow
```
Content/Site Publishing Flow:
1. User selects content/site to publish
2. System checks Site.integrations (available destinations)
3. User selects destinations (can be multiple)
4. PublisherService routes to appropriate adapters:
- WordPressAdapter.publish()
- SitesRendererAdapter.deploy()
- ShopifyAdapter.publish()
5. PublishingRecord created for each destination
6. Status tracked per destination
7. Two-way sync enabled if integration.sync_enabled = True
```
### 11.9 Implementation Files
| Component | File | Purpose |
|-----------|------|---------|
| **SiteIntegration Model** | `business/integration/models.py` | Store integration configs |
| **IntegrationService** | `business/integration/services/integration_service.py` | Manage integrations |
| **SyncService** | `business/integration/services/sync_service.py` | Handle two-way sync |
| **WordPressAdapter** | `business/publishing/services/adapters/wordpress_adapter.py` | WordPress publishing |
| **SitesRendererAdapter** | `business/publishing/services/adapters/sites_renderer_adapter.py` | IGNY8 Sites deployment |
| **ShopifyAdapter** | `business/publishing/services/adapters/shopify_adapter.py` | Shopify publishing (future) |
| **Integration API** | `modules/integration/views.py` | CRUD for integrations |
| **Integration UI** | `frontend/src/pages/Settings/Integrations.tsx` | Manage integrations |
### 11.10 Site Builder Integration Flow
```
Site Builder → Publishing Flow:
1. User builds site via Site Builder wizard
2. SiteBlueprint created with hosting_type
3. If hosting_type = 'wordpress' or 'shopify':
a. User must configure SiteIntegration
b. Site Builder generates platform-compatible structure
4. User clicks "Deploy"
5. DeploymentService:
- If 'igny8_sites': Deploy to Sites renderer
- If 'wordpress': Publish via WordPressAdapter
- If 'shopify': Publish via ShopifyAdapter
- If 'multi': Publish to all configured destinations
6. DeploymentRecord created
7. Site available on selected platforms
```
### 11.11 Content Type Support Per Platform
| Content Type | WordPress | IGNY8 Sites | Shopify | Notes |
|--------------|-----------|-------------|---------|-------|
| **Blog Posts** | ✅ | ✅ | ✅ (Blog) | All platforms support |
| **Pages** | ✅ | ✅ | ✅ (Pages) | All platforms support |
| **Products** | ✅ (WooCommerce) | ✅ | ✅ | Product adapter needed |
| **Product Categories** | ✅ (WooCommerce) | ✅ | ✅ (Collections) | Taxonomy mapping |
| **Product Attributes** | ✅ (WooCommerce) | ✅ | ✅ (Variants) | Attribute mapping |
| **Service Pages** | ✅ | ✅ | ❌ | Custom content type |
| **Taxonomy Terms** | ✅ | ✅ | ❌ | Blog-specific |
---
## SUMMARY
### Key Architectural Decisions
1. **Domain-Driven Structure**: Organize by business domains, not technical layers
2. **Service Layer**: All business logic in services, ViewSets are thin
3. **Unified Content Model**: Extend existing `Content` model, don't create duplicates
4. **Automation First**: All functions can be automated via Celery + AutomationRule
5. **Multi-Site Ready**: All features respect site boundaries and limits
6. **Volume Strategy**: Named volumes for data, bind mounts for code (dev)
7. **Single Repository**: Keep all code in one repo with clear structure
8. **Container Separation**: Separate containers for different UIs (Main, Site Builder, Sites)
### Implementation Order
1. **Phase 0**: Foundation (Service layer, Automation, Model extensions)
2. **Phase 1**: Site Builder
3. **Phase 2**: Linker & Optimizer
4. **Phase 3**: Sites Renderer
5. **Phase 4**: Universal Content Types
### Success Criteria
- ✅ No code duplication
- ✅ All functions automatable
- ✅ Multi-site isolation working
- ✅ Limits enforced per site
- ✅ Volume strategy optimized
- ✅ Repository structure clean
- ✅ All placeholders implemented
---
**END OF DOCUMENT**