igny8/docs/planning/IGNY8-IMPLEMENTATION-PLAN.md

# IGNY8 IMPLEMENTATION PLAN
**Preserving Existing Functionality + Phase 2 Features**

**Last Updated:** 2025-01-XX  
**Purpose:** Step-by-step implementation plan that keeps everything working while adding Phase 2 features.

---

## TABLE OF CONTENTS

1. [Implementation Strategy](#1-implementation-strategy)
2. [Phase 0: Foundation & Credit System](#phase-0-foundation--credit-system)
3. [Phase 1: Service Layer Refactoring](#phase-1-service-layer-refactoring)
4. [Phase 2: Automation System](#phase-2-automation-system)
5. [Phase 3: Site Builder](#phase-3-site-builder)
   - [3.0 Sites Folder Access & File Management](#30-sites-folder-access--file-management)
6. [Phase 4: Linker & Optimizer](#phase-4-linker--optimizer)
7. [Phase 5: Sites Renderer](#phase-5-sites-renderer)
8. [Phase 6: Site Integration & Multi-Destination Publishing](#phase-6-site-integration--multi-destination-publishing)
9. [Phase 7: UI Components & Module Settings](#phase-7-ui-components--module-settings)
10. [Phase 8: Universal Content Types](#phase-8-universal-content-types)
11. [Testing & Validation](#testing--validation)

---

## 1. IMPLEMENTATION STRATEGY

### 1.1 Core Principles

| Principle | Implementation |
|-----------|----------------|
| **Backward Compatibility** | All existing APIs, workflows, and features continue working |
| **Incremental Changes** | Add new features without breaking existing ones |
| **Feature Flags** | Use flags to enable/disable new features during rollout |
| **Parallel Development** | New features developed alongside existing code |
| **Gradual Migration** | Move to new architecture gradually, not all at once |

### 1.2 What's Currently Working (PRESERVE)

| Module | Features | Status |
|--------|----------|--------|
| **Planner** | Keywords, Clusters, Ideas | ✅ Working |
| **Writer** | Tasks, Content Generation, Images | ✅ Working |
| **Thinker** | Prompts, Author Profiles, Strategies | ✅ Working |
| **Billing** | Credits, Transactions, Usage Logs | ✅ Working |
| **System** | Settings, Integration Settings | ✅ Working |
| **Auth** | Multi-tenancy, Users, Sites, Sectors | ✅ Working |
| **AI Framework** | 5 AI functions (cluster, ideas, content, prompts, images) | ✅ Working |
| **WordPress Publishing** | Content publishing to WordPress | ✅ Working |

### 1.3 What Needs to Be Built (PHASE 2)

| Feature | Status | Priority |
|---------|--------|----------|
| **Credit-Only System** | 🔨 Build | HIGH |
| **Automation System** | 🔨 Build | HIGH |
| **Service Layer** | 🔨 Build | HIGH |
| **Site Builder** | 🔨 Build | HIGH |
| **Linker** | 🔨 Build | MEDIUM |
| **Optimizer** | 🔨 Build | MEDIUM |
| **Sites Renderer** | 🔨 Build | MEDIUM |
| **Site Integration** | 🔨 Build | MEDIUM |
| **Universal Content Types** | 🔨 Build | LOW |

---

## PHASE 0: FOUNDATION & CREDIT SYSTEM

**Goal**: Migrate to credit-only model while preserving all existing functionality.

### 0.0 Module Settings System (Enable/Disable Modules)

| Task | Files | Dependencies | Risk |
|------|-------|--------------|------|
| **Add Module Enable/Disable** | `business/system/models.py` | EXISTING (ModuleSettings) | LOW - Extend existing |
| **Module Settings API** | `modules/system/views.py` | EXISTING | LOW - Extend existing |
| **Module Settings UI** | `frontend/src/pages/Settings/Modules.tsx` | EXISTING (placeholder) | LOW - Implement UI |
| **Frontend Module Loader** | `frontend/src/config/modules.config.ts` | None | MEDIUM - Conditional loading |
| **Route Guard** | `frontend/src/components/common/ModuleGuard.tsx` | None | LOW - New component |

**Module Enable/Disable Logic**:
- Each module has `enabled` flag in ModuleSettings
- Frontend checks module status before loading routes
- Disabled modules don't appear in sidebar
- Disabled modules don't load code (lazy loading check)

**Modules to Control**:
- Planner
- Writer
- Thinker
- Automation
- Site Builder (NEW)
- Linker (NEW)
- Optimizer (NEW)
- Publisher (NEW)

### 0.1 Credit System Updates

| Task | Files | Dependencies | Risk |
|------|-------|--------------|------|
| **Remove Plan Limit Fields** | `core/auth/models.py` | None | LOW - Add migration to set defaults |
| **Update Plan Model** | `core/auth/models.py` | None | LOW - Keep only monthly_credits, support_level |
| **Update CreditService** | `business/billing/services/credit_service.py` | None | MEDIUM - Add credit cost constants |
| **Add Credit Costs** | `business/billing/constants.py` | None | LOW - Define credit costs per operation |
| **Update AI Engine** | `infrastructure/ai/engine.py` | CreditService | MEDIUM - Check credits before AI calls |
| **Update Content Generation** | `business/content/services/` | CreditService | MEDIUM - Check credits before generation |
| **Update Image Generation** | `infrastructure/ai/functions/generate_images.py` | CreditService | MEDIUM - Check credits before generation |
| **Remove Limit Checks** | All services | None | MEDIUM - Remove all plan limit validations |
| **Update Usage Logging** | `business/billing/models.py` | None | LOW - Ensure all operations log credits |
| **Update Frontend Limits UI** | `frontend/src/pages/Billing/` | Backend API | LOW - Replace limits with credit display |

**Credit Cost Constants**:
```python
# business/billing/constants.py
CREDIT_COSTS = {
    'clustering': 10,
    'idea_generation': 15,
    'content_generation': 1,  # per 100 words
    'image_prompt_extraction': 2,
    'image_generation': 5,
    'linking': 8,  # NEW
    'optimization': 1,  # per 200 words, NEW
    'site_structure_generation': 50,  # NEW
    'site_page_generation': 20,  # NEW
}
```

### 0.2 Operational Limits (Keep)

| Limit | Location | Implementation |
|-------|----------|----------------|
| **50 keywords per request** | `modules/planner/views.py` | Request validation |
| **6 images per request** | `modules/writer/views.py` | Request validation |

### 0.3 Database Migrations

| Migration | Purpose | Risk |
|-----------|---------|------|
| **Remove limit fields from Plan** | Clean up unused fields | LOW - Add defaults first |
| **Add credit cost tracking** | Enhance CreditUsageLog | LOW - Additive only |
| **Monthly credit replenishment** | Celery Beat task | LOW - New feature |

### 0.4 Testing

- ✅ All existing features work with credit checks
- ✅ Credit deduction happens correctly
- ✅ Insufficient credits show clear error
- ✅ Usage logging tracks all operations
- ✅ Frontend shows credit balance, not limits

**Timeline**: 1-2 weeks

---

## PHASE 1: SERVICE LAYER REFACTORING

**Goal**: Extract business logic from ViewSets into services, preserving all existing functionality.

### 1.1 Create Business Structure

| Task | Files | Dependencies |
|------|-------|--------------|
| **Create business/ folder** | `backend/igny8_core/business/` | None |
| **Move Content models** | `business/content/models.py` | Phase 0 |
| **Move Planning models** | `business/planning/models.py` | Phase 0 |
| **Create ContentService** | `business/content/services/content_generation_service.py` | Existing Writer logic |
| **Create PlanningService** | `business/planning/services/clustering_service.py` | Existing Planner logic |
| **Create IdeasService** | `business/planning/services/ideas_service.py` | Existing Planner logic |

### 1.2 Refactor ViewSets (Keep APIs Working)

| Module | Current | New | Risk |
|--------|---------|-----|------|
| **Planner ViewSets** | Business logic in views | Delegate to services | LOW - Internal refactor |
| **Writer ViewSets** | Business logic in views | Delegate to services | LOW - Internal refactor |
| **Billing ViewSets** | Already uses CreditService | Keep as-is | NONE |

**Strategy**: 
- ViewSets become thin wrappers
- All business logic moves to services
- APIs remain unchanged (backward compatible)

### 1.3 Testing

- ✅ All existing API endpoints work identically
- ✅ Response formats unchanged
- ✅ No breaking changes for frontend
- ✅ Services are testable independently

**Timeline**: 2-3 weeks

---

## PHASE 2: AUTOMATION SYSTEM

**Goal**: Implement automation rules and scheduled tasks.

### 2.1 Automation Models

| Task | Files | Dependencies |
|------|-------|--------------|
| **AutomationRule Model** | `business/automation/models.py` | Phase 1 |
| **ScheduledTask Model** | `business/automation/models.py` | Phase 1 |
| **Automation Migrations** | `business/automation/migrations/` | Phase 1 |

### 2.2 Automation Service

| Task | Files | Dependencies |
|------|-------|--------------|
| **AutomationService** | `business/automation/services/automation_service.py` | Phase 1 services |
| **Rule Execution Engine** | `business/automation/services/rule_engine.py` | Phase 1 services |
| **Condition Evaluator** | `business/automation/services/condition_evaluator.py` | None |
| **Action Executor** | `business/automation/services/action_executor.py` | Phase 1 services |

### 2.3 Celery Beat Tasks

| Task | Files | Dependencies |
|------|-------|--------------|
| **Scheduled Automation Task** | `infrastructure/messaging/automation_tasks.py` | AutomationService |
| **Monthly Credit Replenishment** | `infrastructure/messaging/automation_tasks.py` | CreditService |
| **Celery Beat Configuration** | `backend/igny8_core/celery.py` | None |

### 2.4 Automation API

| Task | Files | Dependencies |
|------|-------|--------------|
| **AutomationRule ViewSet** | `modules/automation/views.py` | AutomationService |
| **ScheduledTask ViewSet** | `modules/automation/views.py` | AutomationService |
| **Automation URLs** | `modules/automation/urls.py` | None |

### 2.5 Automation UI

| Task | Files | Dependencies |
|------|-------|--------------|
| **Automation Dashboard** | `frontend/src/pages/Automation/Dashboard.tsx` | EXISTING (placeholder) |
| **Rules Management** | `frontend/src/pages/Automation/Rules.tsx` | NEW |
| **Schedules (within Automation)** | Integrated into Automation Dashboard | Part of automation menu |
| **Automation API Client** | `frontend/src/services/automation.api.ts` | NEW |

### 2.6 Testing

- ✅ Automation rules execute correctly
- ✅ Scheduled tasks run on time
- ✅ Credit replenishment works monthly
- ✅ UI shows automation status

**Timeline**: 2-3 weeks

---

## PHASE 3: SITE BUILDER

**Goal**: Build Site Builder for creating sites via wizard.

### 3.0 Sites Folder Access & File Management

| Task | Files | Dependencies |
|------|-------|--------------|
| **Site File Management Service** | `business/site_building/services/file_management_service.py` | Phase 1 |
| **User Site Access Check** | `business/site_building/services/file_management_service.py` | EXISTING (SiteUserAccess) |
| **File Upload API** | `modules/site_builder/views.py` | File Management Service |
| **File Browser UI** | `site-builder/src/components/files/FileBrowser.tsx` | NEW |
| **Storage Quota Check** | `infrastructure/storage/file_storage.py` | Phase 1 |

**Sites Folder Structure**:
```
/data/app/sites-data/
└── clients/
    └── {site_id}/
        └── v{version}/
            ├── site.json
            ├── pages/
            └── assets/          # User-managed files
                ├── images/
                ├── documents/
                └── media/
```

**User Access Rules**:
- Owner/Admin: Full access to all account sites
- Editor: Access to granted sites (via SiteUserAccess)
- Viewer: Read-only access to granted sites
- File operations scoped to user's accessible sites only

### 3.1 Site Builder Models

| Task | Files | Dependencies |
|------|-------|--------------|
| **SiteBlueprint Model** | `business/site_building/models.py` | Phase 1 |
| **PageBlueprint Model** | `business/site_building/models.py` | Phase 1 |
| **Site Builder Migrations** | `business/site_building/migrations/` | Phase 1 |

### 3.2 Site Structure Generation

| Task | Files | Dependencies |
|------|-------|--------------|
| **Structure Generation AI Function** | `infrastructure/ai/functions/generate_site_structure.py` | Existing AI framework |
| **Structure Generation Service** | `business/site_building/services/structure_generation_service.py` | Phase 1, AI framework |
| **Site Structure Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system |

### 3.3 Site Builder API

| Task | Files | Dependencies |
|------|-------|--------------|
| **Site Builder ViewSet** | `modules/site_builder/views.py` | Structure Generation Service |
| **Site Builder URLs** | `modules/site_builder/urls.py` | None |
| **Site Builder Serializers** | `modules/site_builder/serializers.py` | None |

### 3.4 Site Builder Frontend (New Container)

**User-Friendly Name**: "Website Builder" or "Site Creator"

| Task | Files | Dependencies |
|------|-------|--------------|
| **Create Site Builder Container** | `docker-compose.app.yml` | None |
| **Wizard Steps** | `site-builder/src/pages/wizard/` | NEW |
| **Preview Canvas** | `site-builder/src/pages/preview/` | NEW |
| **Site Builder State** | `site-builder/src/state/builderStore.ts` | NEW |
| **Site Builder API Client** | `site-builder/src/api/builder.api.ts` | NEW |
| **Layout Selection** | `site-builder/src/components/layouts/` | NEW |
| **Template Library** | `site-builder/src/components/templates/` | NEW |
| **Block Components** | `site-builder/src/components/blocks/` | NEW (uses shared library) |

### 3.7 Global Component Library

| Task | Files | Dependencies |
|------|-------|--------------|
| **Create Shared Component Library** | `frontend/src/components/shared/` | None |
| **Block Components** | `frontend/src/components/shared/blocks/` | NEW |
| **Layout Components** | `frontend/src/components/shared/layouts/` | NEW |
| **Template Components** | `frontend/src/components/shared/templates/` | NEW |
| **Component Documentation** | `frontend/src/components/shared/README.md` | None |

**Component Library Structure**:
```
frontend/src/components/shared/
├── blocks/
│   ├── Hero.tsx
│   ├── Features.tsx
│   ├── Services.tsx
│   ├── Products.tsx
│   ├── Testimonials.tsx
│   ├── ContactForm.tsx
│   └── ...
├── layouts/
│   ├── DefaultLayout.tsx
│   ├── MinimalLayout.tsx
│   ├── MagazineLayout.tsx
│   ├── EcommerceLayout.tsx
│   └── ...
└── templates/
    ├── BlogTemplate.tsx
    ├── BusinessTemplate.tsx
    ├── PortfolioTemplate.tsx
    └── ...
```

**Usage**: Site Builder, Sites Renderer, and Main App all use same components (no duplicates)

### 3.5 Page Generation (Reuse Content Service)

| Task | Files | Dependencies |
|------|-------|--------------|
| **Extend ContentService** | `business/content/services/content_generation_service.py` | Phase 1 |
| **Add Site Page Type** | `business/content/models.py` | Phase 1 |
| **Page Generation Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system |

### 3.6 Testing

- ✅ Site Builder wizard works end-to-end
- ✅ Structure generation creates valid blueprints
- ✅ Preview renders correctly
- ✅ Page generation reuses existing content service

**Timeline**: 3-4 weeks

---

## PHASE 4: LINKER & OPTIMIZER

**Goal**: Add linking and optimization as post-processing stages with multiple entry points.

### 4.0 Content Workflow & Entry Points

**Content Sources**:
1. **IGNY8 Generated** - Content created via Writer module
2. **WordPress Synced** - Content synced from WordPress via plugin
3. **3rd Party Synced** - Content synced from external sources (Shopify, custom APIs)

**Content Storage Strategy**:
- All content stored in unified `Content` model
- `source` field: `'igny8'`, `'wordpress'`, `'shopify'`, `'custom'`
- `sync_status` field: `'native'`, `'imported'`, `'synced'`
- Same location, filtered by source/sync_status

**Workflow Entry Points**:

```
Entry Point 1: Writer → Linker → Optimizer → Publish
Entry Point 2: WordPress Sync → Optimizer → Publish
Entry Point 3: 3rd Party Sync → Optimizer → Publish
Entry Point 4: Manual Selection → Linker/Optimizer
```

### 4.0.1 Writer → Linker Handover

| Step | Action | Implementation |
|------|--------|---------------|
| **1. Content Generated** | Writer generates content | `ContentService.generate()` |
| **2. Content Saved** | Content saved to DB | `Content` model with `source='igny8'` |
| **3. Linker Trigger** | Auto-trigger or manual | `LinkerService.process(content_id)` |
| **4. Links Injected** | Links added to content | `Content.internal_links` updated |
| **5. Content Updated** | Content version incremented | `Content.linker_version++` |
| **6. Optimizer Ready** | Content ready for optimization | Status: `'linked'` or `'ready_for_optimization'` |

**Auto-Trigger Options**:
- Immediate (after content generation)
- Scheduled (via AutomationRule)
- Manual (user clicks "Add Links")

### 4.0.2 Optimizer Entry Points

| Entry Point | Source | Trigger | Content Status |
|-------------|--------|---------|----------------|
| **Writer → Optimizer** | IGNY8 generated | After linking or direct | `source='igny8'`, `status='draft'` |
| **WordPress Sync → Optimizer** | WordPress plugin | After sync or manual | `source='wordpress'`, `sync_status='synced'` |
| **3rd Party Sync → Optimizer** | External API | After sync or manual | `source='shopify'/'custom'`, `sync_status='imported'` |
| **Manual Selection** | Any content | User selects content | Any source, any status |

**Optimizer Processing**:
- All entry points use same `OptimizerService`
- Content source doesn't matter - optimizer works on any content
- Results stored in `OptimizationTask` model
- Original content preserved, optimized version stored separately

### 4.1 Content Model Extensions (For Synced Content)

| Task | Files | Dependencies |
|------|-------|--------------|
| **Add source field** | `business/content/models.py` | Phase 1 |
| **Add sync_status field** | `business/content/models.py` | Phase 1 |
| **Add external_id field** | `business/content/models.py` | Phase 1 |
| **Add sync_metadata field** | `business/content/models.py` | Phase 1 |
| **Content Migrations** | `business/content/migrations/` | Phase 1 |

**Content Model Extensions**:
```python
class Content(SiteSectorBaseModel):
    # Existing fields...
    
    # NEW: Source tracking
    source = models.CharField(
        max_length=50,
        choices=[
            ('igny8', 'IGNY8 Generated'),
            ('wordpress', 'WordPress Synced'),
            ('shopify', 'Shopify Synced'),
            ('custom', 'Custom API Synced'),
        ],
        default='igny8'
    )
    
    # NEW: Sync status
    sync_status = models.CharField(
        max_length=50,
        choices=[
            ('native', 'Native IGNY8 Content'),
            ('imported', 'Imported from External'),
            ('synced', 'Synced from External'),
        ],
        default='native'
    )
    
    # NEW: External reference
    external_id = models.CharField(max_length=255, blank=True, null=True)
    external_url = models.URLField(blank=True, null=True)
    sync_metadata = models.JSONField(default=dict)  # Platform-specific metadata
```

### 4.2 Linker Models

| Task | Files | Dependencies |
|------|-------|--------------|
| **InternalLink Model** | `business/linking/models.py` | Phase 1 |
| **LinkGraph Model** | `business/linking/models.py` | Phase 1 |
| **Linker Migrations** | `business/linking/migrations/` | Phase 1 |

### 4.3 Linker Service

| Task | Files | Dependencies |
|------|-------|--------------|
| **LinkerService** | `business/linking/services/linker_service.py` | Phase 1, ContentService |
| **Link Candidate Engine** | `business/linking/services/candidate_engine.py` | Phase 1 |
| **Link Injection Engine** | `business/linking/services/injection_engine.py` | Phase 1 |

### 4.4 Content Sync Service (For WordPress/3rd Party)

| Task | Files | Dependencies |
|------|-------|--------------|
| **ContentSyncService** | `business/integration/services/content_sync_service.py` | Phase 1, Phase 6 |
| **WordPress Content Sync** | `business/integration/services/wordpress_sync.py` | Phase 6 |
| **3rd Party Content Sync** | `business/integration/services/external_sync.py` | Phase 6 |
| **Content Import Logic** | `business/integration/services/import_service.py` | Phase 1 |

**Sync Service Flow**:
```python
class ContentSyncService:
    def sync_from_wordpress(self, site, sync_type='full'):
        """Sync content from WordPress"""
        # Fetch posts from WordPress
        # Create Content records with source='wordpress', sync_status='synced'
        # Store external_id, external_url
        # Content ready for Optimizer
    
    def sync_from_shopify(self, site, sync_type='full'):
        """Sync content from Shopify"""
        # Fetch products/pages from Shopify
        # Create Content records with source='shopify', sync_status='synced'
        # Store external_id, external_url
        # Content ready for Optimizer
    
    def import_content(self, content_data, source, external_id):
        """Import content from any source"""
        # Create Content record
        # Mark with source and sync_status='imported'
        # Content ready for Optimizer
```

### 4.5 Optimizer Models

| Task | Files | Dependencies |
|------|-------|--------------|
| **OptimizationTask Model** | `business/optimization/models.py` | Phase 1 |
| **OptimizationScores Model** | `business/optimization/models.py` | Phase 1 |
| **Optimizer Migrations** | `business/optimization/migrations/` | Phase 1 |

### 4.6 Optimizer Service (Multiple Entry Points)

| Task | Files | Dependencies |
|------|-------|--------------|
| **OptimizerService** | `business/optimization/services/optimizer_service.py` | Phase 1, ContentService |
| **Content Analyzer** | `business/optimization/services/analyzer.py` | Phase 1 |
| **Entry Point Handler** | `business/optimization/services/entry_handler.py` | Phase 1 |
| **Optimization AI Function** | `infrastructure/ai/functions/optimize_content.py` | Existing AI framework |
| **Optimization Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system |

**Optimizer Entry Point Handler**:
```python
class OptimizerService:
    def optimize_from_writer(self, content_id):
        """Entry Point 1: Writer → Optimizer"""
        content = Content.objects.get(id=content_id, source='igny8')
        return self.optimize(content)
    
    def optimize_from_wordpress_sync(self, content_id):
        """Entry Point 2: WordPress Sync → Optimizer"""
        content = Content.objects.get(id=content_id, source='wordpress')
        return self.optimize(content)
    
    def optimize_from_external_sync(self, content_id):
        """Entry Point 3: External Sync → Optimizer"""
        content = Content.objects.get(id=content_id, source__in=['shopify', 'custom'])
        return self.optimize(content)
    
    def optimize_manual(self, content_id):
        """Entry Point 4: Manual Selection → Optimizer"""
        content = Content.objects.get(id=content_id)
        return self.optimize(content)  # Works for any source
    
    def optimize(self, content):
        """Unified optimization logic"""
        # Works the same regardless of source
        # Analyzes content
        # Generates optimized version
        # Stores in OptimizationTask
        # Updates content.optimizer_version
```

| Task | Files | Dependencies |
|------|-------|--------------|
| **OptimizerService** | `business/optimization/services/optimizer_service.py` | Phase 1, ContentService |
| **Content Analyzer** | `business/optimization/services/analyzer.py` | Phase 1 |
| **Optimization AI Function** | `infrastructure/ai/functions/optimize_content.py` | Existing AI framework |
| **Optimization Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system |

### 4.7 Content Pipeline Service

| Task | Files | Dependencies |
|------|-------|--------------|
| **ContentPipelineService** | `business/content/services/content_pipeline_service.py` | LinkerService, OptimizerService |
| **Pipeline Orchestration** | `business/content/services/pipeline_service.py` | Phase 1 services |
| **Workflow State Machine** | `business/content/services/workflow_state.py` | Phase 1 services |

**Pipeline Workflow States**:
```
Content States:
- 'draft' → Generated, not processed
- 'linked' → Links added, ready for optimization
- 'optimized' → Optimized, ready for review
- 'review' → Ready for publishing
- 'published' → Published to destination(s)
```

**Pipeline Orchestration**:
```python
class ContentPipelineService:
    def process_writer_content(self, content_id, stages=['linking', 'optimization']):
        """Writer → Linker → Optimizer pipeline"""
        content = Content.objects.get(id=content_id, source='igny8')
        
        if 'linking' in stages:
            content = self.link_content(content)
        
        if 'optimization' in stages:
            content = self.optimize_content(content)
        
        return content
    
    def process_synced_content(self, content_id, stages=['optimization']):
        """Synced Content → Optimizer (skip linking if needed)"""
        content = Content.objects.get(id=content_id)
        
        if 'optimization' in stages:
            content = self.optimize_content(content)
        
        return content
```

### 4.8 Linker & Optimizer APIs

| Task | Files | Dependencies |
|------|-------|--------------|
| **Linker ViewSet** | `modules/linker/views.py` | LinkerService |
| **Optimizer ViewSet** | `modules/optimizer/views.py` | OptimizerService |
| **Linker URLs** | `modules/linker/urls.py` | None |
| **Optimizer URLs** | `modules/optimizer/urls.py` | None |

### 4.9 Linker & Optimizer UI

| Task | Files | Dependencies |
|------|-------|--------------|
| **Linker Dashboard** | `frontend/src/pages/Linker/Dashboard.tsx` | NEW |
| **Optimizer Dashboard** | `frontend/src/pages/Optimizer/Dashboard.tsx` | NEW |
| **Linker API Client** | `frontend/src/services/linker.api.ts` | NEW |
| **Optimizer API Client** | `frontend/src/services/optimizer.api.ts` | NEW |
| **Content Selection UI** | `frontend/src/components/optimizer/ContentSelector.tsx` | NEW |
| **Entry Point Handler UI** | `frontend/src/components/optimizer/EntryPointHandler.tsx` | NEW |
| **Source Badge Component** | `frontend/src/components/content/SourceBadge.tsx` | NEW |

**Optimizer UI Features**:
- Show content source (IGNY8, WordPress, Shopify badge)
- Show sync status (Native, Synced, Imported badge)
- Entry point selection (from Writer, from Sync, Manual)
- Content list with source filters
- "Send to Optimizer" button (works for any source)

### 4.10 Content Filtering & Display

| Task | Files | Dependencies |
|------|-------|--------------|
| **Content Filter Component** | `frontend/src/components/content/ContentFilter.tsx` | NEW |
| **Source Filter** | `frontend/src/components/content/SourceFilter.tsx` | NEW |
| **Sync Status Filter** | `frontend/src/components/content/SyncStatusFilter.tsx` | NEW |
| **Content List with Filters** | `frontend/src/pages/Writer/Content.tsx` | EXISTING (update) |

**Content Filtering**:
- Filter by source: IGNY8, WordPress, Shopify, Custom
- Filter by sync_status: Native, Imported, Synced
- Filter by optimization status: Not optimized, Optimized, Needs optimization
- Filter by linking status: Not linked, Linked, Needs linking

### 4.11 Testing

- ✅ Writer → Linker handover works
- ✅ Linker finds appropriate link candidates
- ✅ Links inject correctly into content
- ✅ Optimizer works from Writer entry point
- ✅ Optimizer works from WordPress sync entry point
- ✅ Optimizer works from 3rd party sync entry point
- ✅ Optimizer works from manual selection
- ✅ Synced content stored correctly with source flags
- ✅ Content filtering works (by source, sync_status)
- ✅ Pipeline orchestrates correctly
- ✅ All entry points use same optimization logic

**Timeline**: 4-5 weeks (extended for multiple entry points)

---

## PHASE 5: SITES RENDERER

**Goal**: Build Sites renderer for hosting public sites.

### 5.1 Sites Renderer Container

**User-Friendly Name**: "My Websites" or "Published Sites"

| Task | Files | Dependencies |
|------|-------|--------------|
| **Create Sites Container** | `docker-compose.app.yml` | None |
| **Sites Renderer Frontend** | `sites/src/` | NEW |
| **Block Components** | `sites/src/components/blocks/` | NEW (imports from shared) |
| **Layout Components** | `sites/src/components/layouts/` | NEW (imports from shared) |
| **Template Components** | `sites/src/components/templates/` | NEW (imports from shared) |
| **Site Definition Loader** | `sites/src/loaders/loadSiteDefinition.ts` | NEW |
| **Multiple Layout Support** | `sites/src/utils/layoutResolver.ts` | NEW |
| **Template System** | `sites/src/utils/templateEngine.ts` | NEW |
| **CMS Styling** | `sites/src/styles/cms.css` | NEW |

### 5.6 Multiple Layout Options

| Task | Files | Dependencies |
|------|-------|--------------|
| **Layout Configuration** | `business/site_building/models.py` | Phase 3 |
| **Layout Selector UI** | `site-builder/src/components/layouts/LayoutSelector.tsx` | Phase 3 |
| **Layout Renderer** | `sites/src/utils/layoutRenderer.ts` | Phase 5 |
| **Layout Preview** | `site-builder/src/components/preview/LayoutPreview.tsx` | Phase 3 |

**Layout Types**:
- Default (Standard)
- Minimal (Clean, simple)
- Magazine (Editorial, content-focused)
- Ecommerce (Product-focused)
- Portfolio (Showcase)
- Blog (Content-first)
- Corporate (Business)

### 5.2 Publisher Service

| Task | Files | Dependencies |
|------|-------|--------------|
| **PublisherService** | `business/publishing/services/publisher_service.py` | Phase 1 |
| **SitesRendererAdapter** | `business/publishing/services/adapters/sites_renderer_adapter.py` | Phase 3 |
| **DeploymentService** | `business/publishing/services/deployment_service.py` | Phase 3 |

### 5.3 Publishing Models

| Task | Files | Dependencies |
|------|-------|--------------|
| **PublishingRecord Model** | `business/publishing/models.py` | Phase 1 |
| **DeploymentRecord Model** | `business/publishing/models.py` | Phase 3 |
| **Publishing Migrations** | `business/publishing/migrations/` | Phase 1 |

### 5.4 Publisher API

| Task | Files | Dependencies |
|------|-------|--------------|
| **Publisher ViewSet** | `modules/publisher/views.py` | PublisherService |
| **Publisher URLs** | `modules/publisher/urls.py` | None |

### 5.5 Testing

- ✅ Sites renderer loads site definitions
- ✅ Blocks render correctly
- ✅ Deployment works end-to-end
- ✅ Sites are accessible publicly

**Timeline**: 2-3 weeks

---

## PHASE 6: SITE INTEGRATION & MULTI-DESTINATION PUBLISHING

**Goal**: Support multiple publishing destinations (WordPress, Sites, Shopify).

### 6.1 Site Integration Models

| Task | Files | Dependencies |
|------|-------|--------------|
| **SiteIntegration Model** | `business/integration/models.py` | Phase 1 |
| **Integration Migrations** | `business/integration/migrations/` | Phase 1 |

### 6.2 Integration Service

| Task | Files | Dependencies |
|------|-------|--------------|
| **IntegrationService** | `business/integration/services/integration_service.py` | Phase 1 |
| **SyncService** | `business/integration/services/sync_service.py` | Phase 1 |

### 6.3 Publishing Adapters

| Task | Files | Dependencies |
|------|-------|--------------|
| **BaseAdapter** | `business/publishing/services/adapters/base_adapter.py` | Phase 5 |
| **WordPressAdapter** | `business/publishing/services/adapters/wordpress_adapter.py` | EXISTING (refactor) |
| **SitesRendererAdapter** | `business/publishing/services/adapters/sites_renderer_adapter.py` | Phase 5 |
| **ShopifyAdapter** | `business/publishing/services/adapters/shopify_adapter.py` | Phase 5 (future) |

### 6.4 Multi-Destination Publishing

| Task | Files | Dependencies |
|------|-------|--------------|
| **Extend PublisherService** | `business/publishing/services/publisher_service.py` | Phase 5 |
| **Multi-destination Support** | `business/publishing/services/publisher_service.py` | Phase 5 |
| **Update PublishingRecord** | `business/publishing/models.py` | Phase 5 |

### 6.5 Site Model Extensions

| Task | Files | Dependencies |
|------|-------|--------------|
| **Add site_type field** | `core/auth/models.py` | None |
| **Add hosting_type field** | `core/auth/models.py` | None |
| **Add integrations relationship** | `core/auth/models.py` | Phase 6.1 |
| **Migration** | `core/auth/migrations/` | None |

### 6.6 Integration API

| Task | Files | Dependencies |
|------|-------|--------------|
| **Integration ViewSet** | `modules/integration/views.py` | IntegrationService |
| **Integration URLs** | `modules/integration/urls.py` | None |

### 6.7 Integration UI (Update Existing)

| Task | Files | Dependencies |
|------|-------|--------------|
| **Update Integration Settings** | `frontend/src/pages/Settings/Integration.tsx` | EXISTING (update) |
| **Site Integration Management** | `frontend/src/pages/Settings/Integration.tsx` | EXISTING (update) |
| **Integration API Client** | `frontend/src/services/integration.api.ts` | NEW |
| **Multi-Platform Support** | `frontend/src/components/integration/PlatformSelector.tsx` | NEW |
| **Integration Status** | `frontend/src/components/integration/IntegrationStatus.tsx` | NEW |

**Update Existing Integration Page**:
- Add SiteIntegration model support
- Show multiple integrations per site
- Add WordPress, Shopify, Custom API options
- Two-way sync toggle
- Connection testing

### 6.8 Publishing Settings UI

| Task | Files | Dependencies |
|------|-------|--------------|
| **Publishing Settings Page** | `frontend/src/pages/Settings/Publishing.tsx` | NEW |
| **Destination Management** | `frontend/src/pages/Settings/Publishing.tsx` | Phase 6 |
| **Publishing Rules** | `frontend/src/components/publishing/PublishingRules.tsx` | NEW |
| **Auto-Publish Settings** | `frontend/src/components/publishing/AutoPublishSettings.tsx` | NEW |
| **Publishing API Client** | `frontend/src/services/publisher.api.ts` | NEW |

### 6.9 Individual Site Management (CMS)

**User-Friendly Name**: "Site Manager" or "Content Manager"

| Task | Files | Dependencies |
|------|-------|--------------|
| **Site Management Dashboard** | `frontend/src/pages/Sites/Manage.tsx` | NEW |
| **Site Content Editor** | `frontend/src/pages/Sites/Editor.tsx` | NEW |
| **Post Editor** | `frontend/src/pages/Sites/PostEditor.tsx` | NEW |
| **Page Manager** | `frontend/src/pages/Sites/PageManager.tsx` | NEW |
| **Add/Remove Pages** | `frontend/src/components/sites/PageManager.tsx` | NEW |
| **Content List** | `frontend/src/components/sites/ContentList.tsx` | NEW |
| **Site Settings** | `frontend/src/pages/Sites/Settings.tsx` | NEW |
| **Site API Client** | `frontend/src/services/sites.api.ts` | NEW |

**Site Management Features**:
- View all pages/posts for a site
- Add new pages
- Remove pages
- Edit page content
- Manage page order
- Change page templates
- Update site settings
- Preview site

### 6.11 WordPress Plugin Connection (API Only)

| Task | Files | Dependencies |
|------|-------|--------------|
| **WordPress Sync Endpoints** | `modules/integration/views.py` | IntegrationService |
| **Two-way Sync Logic** | `business/integration/services/sync_service.py` | Phase 6.2 |
| **WordPress Webhook Handler** | `modules/integration/views.py` | Phase 6.2 |

**Note**: WordPress plugin itself is built separately. Only API endpoints for plugin connection are built here.

### 6.9 Testing

- ✅ Site integrations work correctly
- ✅ Multi-destination publishing works
- ✅ WordPress sync works (when plugin connected)
- ✅ Two-way sync functions properly

**Timeline**: 2-3 weeks

---

## PHASE 7: UI COMPONENTS & MODULE SETTINGS

**Goal**: Build comprehensive UI system with shared components, module settings, and site management.

### 7.1 Global Component Library

| Task | Files | Dependencies |
|------|-------|--------------|
| **Component Library Structure** | `frontend/src/components/shared/` | None |
| **Block Components** | `frontend/src/components/shared/blocks/` | None |
| **Layout Components** | `frontend/src/components/shared/layouts/` | None |
| **Template Components** | `frontend/src/components/shared/templates/` | None |
| **Component Documentation** | `frontend/src/components/shared/README.md` | None |
| **Component Storybook** | `frontend/.storybook/` | Optional |
| **Component Tests** | `frontend/src/components/shared/**/*.test.tsx` | None |

**Component Standards**:
- All components use TypeScript
- All components have props interfaces
- All components are responsive
- All components support dark mode
- All components are accessible (ARIA)
- No duplicate components

### 7.2 Module Settings UI

| Task | Files | Dependencies |
|------|-------|--------------|
| **Module Settings Page** | `frontend/src/pages/Settings/Modules.tsx` | EXISTING (implement) |
| **Module Toggle Component** | `frontend/src/components/settings/ModuleToggle.tsx` | NEW |
| **Module Status Indicator** | `frontend/src/components/settings/ModuleStatus.tsx` | NEW |
| **Module Configuration** | `frontend/src/components/settings/ModuleConfig.tsx` | NEW |
| **Module Settings API** | `frontend/src/services/modules.api.ts` | NEW |

**Module Settings Features**:
- Enable/disable modules per account
- Module-specific configuration
- Module status display
- Module usage statistics
- Module dependencies check

### 7.3 Frontend Module Loader

| Task | Files | Dependencies |
|------|-------|--------------|
| **Module Config** | `frontend/src/config/modules.config.ts` | Phase 0 |
| **Module Guard** | `frontend/src/components/common/ModuleGuard.tsx` | Phase 0 |
| **Conditional Route Loading** | `frontend/src/App.tsx` | Phase 0 |
| **Sidebar Module Filter** | `frontend/src/layout/AppSidebar.tsx` | Phase 0 |

**Module Loading Logic**:
```typescript
// Check if module is enabled before loading
const isModuleEnabled = (moduleName: string) => {
  const settings = useSettingsStore(state => state.moduleSettings);
  return settings[moduleName]?.enabled ?? true; // Default enabled
};

// Conditional route loading
{isModuleEnabled('site_builder') && (
  <Route path="/site-builder" element={<SiteBuilder />} />
)}
```

### 7.4 Site Management UI

| Task | Files | Dependencies |
|------|-------|--------------|
| **Site List View** | `frontend/src/pages/Sites/List.tsx` | NEW |
| **Site Dashboard** | `frontend/src/pages/Sites/Dashboard.tsx` | NEW |
| **Site Content Manager** | `frontend/src/pages/Sites/Content.tsx` | NEW |
| **Post Editor** | `frontend/src/pages/Sites/PostEditor.tsx` | NEW |
| **Page Manager** | `frontend/src/pages/Sites/Pages.tsx` | NEW |
| **Site Settings** | `frontend/src/pages/Sites/Settings.tsx` | NEW |
| **Site Preview** | `frontend/src/pages/Sites/Preview.tsx` | NEW |

**Site Management Features**:
- List all sites
- Site dashboard with stats
- Content management (posts, pages)
- Add/remove pages
- Edit content
- Change templates
- Update site settings
- Preview site

### 7.5 Layout & Template System

| Task | Files | Dependencies |
|------|-------|--------------|
| **Layout Selector** | `frontend/src/components/sites/LayoutSelector.tsx` | NEW |
| **Template Library** | `frontend/src/components/sites/TemplateLibrary.tsx` | NEW |
| **Layout Preview** | `frontend/src/components/sites/LayoutPreview.tsx` | NEW |
| **Template Customizer** | `frontend/src/components/sites/TemplateCustomizer.tsx` | NEW |
| **Style Editor** | `frontend/src/components/sites/StyleEditor.tsx` | NEW |

**Layout Options**:
- Default Layout
- Minimal Layout
- Magazine Layout
- Ecommerce Layout
- Portfolio Layout
- Blog Layout
- Corporate Layout

**Template System**:
- Pre-built templates
- Custom templates
- Template customization
- Style presets
- Color schemes
- Typography options

### 7.6 CMS Styling System

| Task | Files | Dependencies |
|------|-------|--------------|
| **CMS Theme System** | `frontend/src/styles/cms/` | NEW |
| **Style Presets** | `frontend/src/styles/cms/presets.ts` | NEW |
| **Color Schemes** | `frontend/src/styles/cms/colors.ts` | NEW |
| **Typography System** | `frontend/src/styles/cms/typography.ts` | NEW |
| **Component Styles** | `frontend/src/styles/cms/components.ts` | NEW |

**CMS Features**:
- Theme customization
- Color palette management
- Typography settings
- Component styling
- Responsive breakpoints
- Dark/light mode

### 7.7 Testing

- ✅ All components render correctly
- ✅ Module settings enable/disable modules
- ✅ Disabled modules don't load
- ✅ Site management works end-to-end
- ✅ Layout system works
- ✅ Template system works
- ✅ No duplicate components

**Timeline**: 3-4 weeks

---

## PHASE 8: UNIVERSAL CONTENT TYPES

**Goal**: Extend content system to support products, services, taxonomies.

### 8.1 Content Model Extensions

| Task | Files | Dependencies |
|------|-------|--------------|
| **Add entity_type field** | `business/content/models.py` | Phase 1 |
| **Add json_blocks field** | `business/content/models.py` | Phase 1 |
| **Add structure_data field** | `business/content/models.py` | Phase 1 |
| **Content Migrations** | `business/content/migrations/` | Phase 1 |

### 8.2 Content Type Prompts

| Task | Files | Dependencies |
|------|-------|--------------|
| **Product Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system |
| **Service Page Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system |
| **Taxonomy Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system |

### 8.3 Content Service Extensions

| Task | Files | Dependencies |
|------|-------|--------------|
| **Product Content Generation** | `business/content/services/content_generation_service.py` | Phase 1 |
| **Service Page Generation** | `business/content/services/content_generation_service.py` | Phase 1 |
| **Taxonomy Generation** | `business/content/services/content_generation_service.py` | Phase 1 |

### 8.4 Linker & Optimizer Extensions

| Task | Files | Dependencies |
|------|-------|--------------|
| **Product Linking** | `business/linking/services/linker_service.py` | Phase 4 |
| **Taxonomy Linking** | `business/linking/services/linker_service.py` | Phase 4 |
| **Product Optimization** | `business/optimization/services/optimizer_service.py` | Phase 4 |
| **Taxonomy Optimization** | `business/optimization/services/optimizer_service.py` | Phase 4 |

### 8.5 Testing

- ✅ Product content generates correctly
- ✅ Service pages work
- ✅ Taxonomy pages work
- ✅ Linking works for all types
- ✅ Optimization works for all types

**Timeline**: 2-3 weeks

---

## TESTING & VALIDATION

### Testing Strategy

| Phase | Testing Focus | Validation |
|-------|--------------|------------|
| **Phase 0** | Credit system, existing features | All existing features work with credits |
| **Phase 1** | Service layer, API compatibility | APIs unchanged, services work |
| **Phase 2** | Automation execution | Rules execute, schedules work |
| **Phase 3** | Site Builder end-to-end | Sites build and deploy |
| **Phase 4** | Linker/Optimizer quality | Content improves, links work |
| **Phase 5** | Sites renderer | Sites render correctly |
| **Phase 6** | Multi-destination publishing | Content publishes to all destinations |
| **Phase 7** | UI components & module settings | Components work, modules can be disabled, site management works |
| **Phase 8** | Universal content types | All content types work |

### Backward Compatibility Checklist

- ✅ All existing API endpoints work
- ✅ All existing workflows function
- ✅ Frontend continues working
- ✅ Database migrations are safe
- ✅ No data loss during migration

### Rollout Strategy

1. **Feature Flags**: Enable new features gradually
2. **Beta Testing**: Test with select accounts first
3. **Gradual Rollout**: Enable for all users after validation
4. **Monitoring**: Track errors, performance, usage

---

## SUMMARY

### Implementation Timeline

| Phase | Duration | Dependencies |
|-------|----------|--------------|
| **Phase 0: Credit System** | 1-2 weeks | None |
| **Phase 1: Service Layer** | 2-3 weeks | Phase 0 |
| **Phase 2: Automation** | 2-3 weeks | Phase 1 |
| **Phase 3: Site Builder** | 3-4 weeks | Phase 1, Phase 2 |
| **Phase 4: Linker/Optimizer** | 3-4 weeks | Phase 1 |
| **Phase 5: Sites Renderer** | 2-3 weeks | Phase 3 |
| **Phase 6: Site Integration** | 2-3 weeks | Phase 5 |
| **Phase 7: UI Components & Module Settings** | 3-4 weeks | Phase 0, Phase 3, Phase 5 |
| **Phase 8: Universal Types** | 2-3 weeks | Phase 4 |

**Total Estimated Time**: 20-29 weeks (5-7 months)

### Key Success Criteria

- ✅ All existing features continue working
- ✅ Credit system is universal and consistent
- ✅ Automation system is functional
- ✅ Site Builder creates and deploys sites ("Website Builder" in UI)
- ✅ Sites Renderer hosts sites ("My Websites" in UI)
- ✅ Linker and Optimizer improve content
- ✅ Multi-destination publishing works
- ✅ Module settings enable/disable modules (modules don't load when disabled)
- ✅ Global component library (no duplicates, shared across all apps)
- ✅ Multiple layout options for sites (7 layout types)
- ✅ Site management UI (CMS) for content editing, page management
- ✅ Site Integration UI updated with new SiteIntegration model
- ✅ Publishing Settings UI functional
- ✅ All content types supported

---

**END OF DOCUMENT**