igny8/docs/planning/PHASE-3-4-IMPLEMENTATION-PLAN.md

PHASE 3 & 4 IMPLEMENTATION PLAN

Detailed Configuration Plan for Site Builder & Linker/Optimizer

Created: 2025-01-XX
Status: Planning Phase

---

TABLE OF CONTENTS

1. Overview
2. Phase 3: Site Builder Implementation Plan
3. Phase 4: Linker & Optimizer Implementation Plan
4. Integration Points
5. File Structure
6. Dependencies & Order
7. Testing Strategy

---

OVERVIEW

Implementation Approach

• Phase 3: Build Site Builder with wizard, AI structure generation, and file management
• Phase 4: Implement Linker and Optimizer as post-processing stages with multiple entry points
• Shared Components: Create global component library for reuse across apps
• Integration: Ensure seamless integration with existing Phase 1 & 2 services

Key Principles

• Service Layer Pattern: All business logic in services (Phase 1 pattern)
• Credit-Aware: All operations check credits before execution
• Multiple Entry Points: Optimizer works from Writer, WordPress sync, 3rd party, manual
• Component Reuse: Shared components across Site Builder, Sites Renderer, Main App

---

PHASE 3: SITE BUILDER IMPLEMENTATION PLAN

3.1 Backend Structure

Business Layer (business/site_building/)

business/site_building/
├── __init__.py
├── models.py                    # SiteBlueprint, PageBlueprint
├── migrations/
│   └── 0001_initial.py
└── services/
    ├── __init__.py
    ├── file_management_service.py
    ├── structure_generation_service.py
    └── page_generation_service.py

Models to Create:

1. SiteBlueprint (business/site_building/models.py)

   • Fields:
     • name, description
     • config_json (wizard choices: business_type, style, objectives)
     • structure_json (AI-generated structure: pages, layout, theme)
     • status (draft, generating, ready, deployed)
     • hosting_type (igny8_sites, wordpress, shopify, multi)
     • version, deployed_version
     • Inherits from SiteSectorBaseModel

2. PageBlueprint (business/site_building/models.py)

   • Fields:
     • site_blueprint (ForeignKey)
     • slug, title
     • type (home, about, services, products, blog, contact, custom)
     • blocks_json (page content blocks)
     • status (draft, generating, ready)
     • order
     • Inherits from SiteSectorBaseModel

Services to Create

1. FileManagementService (business/site_building/services/file_management_service.py)

   class SiteBuilderFileService:
       def get_user_accessible_sites(self, user)
       def check_file_access(self, user, site_id)
       def upload_file(self, user, site_id, file, folder='images')
       def delete_file(self, user, site_id, file_path)
       def list_files(self, user, site_id, folder='images')
       def check_storage_quota(self, site_id, file_size)
   

2. StructureGenerationService (business/site_building/services/structure_generation_service.py)

   class StructureGenerationService:
       def __init__(self):
           self.ai_function = GenerateSiteStructureFunction()
           self.credit_service = CreditService()
   
       def generate_structure(self, site_blueprint, business_brief, objectives, style)
       def _create_page_blueprints(self, site_blueprint, structure)
   

3. PageGenerationService (business/site_building/services/page_generation_service.py)

   class PageGenerationService:
       def __init__(self):
           self.content_service = ContentGenerationService()
           self.credit_service = CreditService()
   
       def generate_page_content(self, page_blueprint, account)
       def regenerate_page(self, page_blueprint, account)
   

AI Functions (infrastructure/ai/functions/)

1. GenerateSiteStructureFunction (infrastructure/ai/functions/generate_site_structure.py)
   • Operation type: site_structure_generation
   • Credit cost: 50 credits (from constants)
   • Generates site structure JSON from business brief

API Layer (modules/site_builder/)

modules/site_builder/
├── __init__.py
├── views.py                    # SiteBuilderViewSet, PageBlueprintViewSet, FileUploadView
├── serializers.py              # SiteBlueprintSerializer, PageBlueprintSerializer
├── urls.py
└── apps.py

ViewSets to Create:

1. SiteBuilderViewSet (modules/site_builder/views.py)

   • CRUD for SiteBlueprint
   • Actions:
     • generate_structure/ (POST) - Trigger AI structure generation
     • deploy/ (POST) - Deploy site to hosting
     • preview/ (GET) - Get preview JSON

2. PageBlueprintViewSet (modules/site_builder/views.py)

   • CRUD for PageBlueprint
   • Actions:
     • generate_content/ (POST) - Generate page content
     • regenerate/ (POST) - Regenerate page content

3. FileUploadView (modules/site_builder/views.py)

   • upload/ (POST) - Upload file to site assets
   • delete/ (DELETE) - Delete file
   • list/ (GET) - List files

File Storage Structure

/data/app/sites-data/
└── clients/
    └── {site_id}/
        └── v{version}/
            ├── site.json              # Site definition
            ├── pages/                 # Page definitions
            │   ├── home.json
            │   ├── about.json
            │   └── ...
            └── assets/                # User-managed files
                ├── images/
                ├── documents/
                └── media/

3.2 Frontend Structure

Site Builder Container (site-builder/)

site-builder/
├── src/
│   ├── pages/
│   │   ├── wizard/
│   │   │   ├── Step1TypeSelection.tsx
│   │   │   ├── Step2BusinessBrief.tsx
│   │   │   ├── Step3Objectives.tsx
│   │   │   └── Step4Style.tsx
│   │   ├── preview/
│   │   │   └── PreviewCanvas.tsx
│   │   └── dashboard/
│   │       └── SiteList.tsx
│   ├── components/
│   │   ├── blocks/              # Block components (import from shared)
│   │   ├── forms/
│   │   ├── files/
│   │   │   └── FileBrowser.tsx
│   │   └── preview-canvas/
│   ├── state/
│   │   ├── builderStore.ts
│   │   └── siteDefinitionStore.ts
│   ├── api/
│   │   ├── builder.api.ts
│   │   └── sites.api.ts
│   └── main.tsx
├── package.json
├── vite.config.ts
└── Dockerfile

Shared Component Library (frontend/src/components/shared/)

frontend/src/components/shared/
├── blocks/
│   ├── Hero.tsx
│   ├── Features.tsx
│   ├── Services.tsx
│   ├── Products.tsx
│   ├── Testimonials.tsx
│   ├── ContactForm.tsx
│   └── ...
├── layouts/
│   ├── DefaultLayout.tsx
│   ├── MinimalLayout.tsx
│   ├── MagazineLayout.tsx
│   ├── EcommerceLayout.tsx
│   ├── PortfolioLayout.tsx
│   ├── BlogLayout.tsx
│   └── CorporateLayout.tsx
└── templates/
    ├── BlogTemplate.tsx
    ├── BusinessTemplate.tsx
    └── PortfolioTemplate.tsx

3.3 Implementation Tasks

Backend Tasks (Priority Order)

1. Create Business Models

   • Create business/site_building/ folder
   • Create SiteBlueprint model
   • Create PageBlueprint model
   • Create migrations

2. Create Services

   • Create FileManagementService
   • Create StructureGenerationService
   • Create PageGenerationService
   • Integrate with CreditService

3. Create AI Function

   • Create GenerateSiteStructureFunction
   • Add prompts for site structure generation
   • Test AI function

4. Create API Layer

   • Create modules/site_builder/ folder
   • Create SiteBuilderViewSet
   • Create PageBlueprintViewSet
   • Create FileUploadView
   • Create serializers
   • Register URLs

Frontend Tasks (Priority Order)

1. Create Site Builder Container

   • Create site-builder/ folder structure
   • Set up Vite + React + TypeScript
   • Configure Docker container
   • Set up routing

2. Create Wizard

   • Step 1: Type Selection
   • Step 2: Business Brief
   • Step 3: Objectives
   • Step 4: Style Preferences
   • Wizard state management

3. Create Preview Canvas

   • Preview renderer
   • Block rendering
   • Layout rendering

4. Create Shared Components

   • Block components
   • Layout components
   • Template components

---

PHASE 4: LINKER & OPTIMIZER IMPLEMENTATION PLAN

4.1 Backend Structure

Business Layer

business/
├── linking/
│   ├── __init__.py
│   ├── models.py                    # InternalLink (optional)
│   └── services/
│       ├── __init__.py
│       ├── linker_service.py
│       ├── candidate_engine.py
│       └── injection_engine.py
│
├── optimization/
│   ├── __init__.py
│   ├── models.py                    # OptimizationTask, OptimizationScores
│   └── services/
│       ├── __init__.py
│       ├── optimizer_service.py
│       └── analyzer.py
│
└── content/
    └── services/
        └── content_pipeline_service.py  # NEW: Orchestrates pipeline

Content Model Extensions

Extend business/content/models.py:

class Content(SiteSectorBaseModel):
    # Existing fields...
    
    # NEW: Source tracking (Phase 4)
    source = models.CharField(
        max_length=50,
        choices=[
            ('igny8', 'IGNY8 Generated'),
            ('wordpress', 'WordPress Synced'),
            ('shopify', 'Shopify Synced'),
            ('custom', 'Custom API Synced'),
        ],
        default='igny8'
    )
    
    sync_status = models.CharField(
        max_length=50,
        choices=[
            ('native', 'Native IGNY8 Content'),
            ('imported', 'Imported from External'),
            ('synced', 'Synced from External'),
        ],
        default='native'
    )
    
    external_id = models.CharField(max_length=255, blank=True, null=True)
    external_url = models.URLField(blank=True, null=True)
    sync_metadata = models.JSONField(default=dict)
    
    # NEW: Linking fields
    internal_links = models.JSONField(default=list)
    linker_version = models.IntegerField(default=0)
    
    # NEW: Optimization fields
    optimizer_version = models.IntegerField(default=0)
    optimization_scores = models.JSONField(default=dict)

Models to Create

1. OptimizationTask (business/optimization/models.py)

   • Fields:
     • content (ForeignKey to Content)
     • scores_before, scores_after (JSON)
     • html_before, html_after (Text)
     • status (pending, completed, failed)
     • credits_used
     • Inherits from AccountBaseModel

2. OptimizationScores (business/optimization/models.py) - Optional

   • Store detailed scoring metrics

Services to Create

1. LinkerService (business/linking/services/linker_service.py)

   class LinkerService:
       def __init__(self):
           self.candidate_engine = CandidateEngine()
           self.injection_engine = InjectionEngine()
           self.credit_service = CreditService()
   
       def process(self, content_id)
       def batch_process(self, content_ids)
   

2. CandidateEngine (business/linking/services/candidate_engine.py)

   class CandidateEngine:
       def find_candidates(self, content)
       def _find_relevant_content(self, content)
       def _score_candidates(self, content, candidates)
   

3. InjectionEngine (business/linking/services/injection_engine.py)

   class InjectionEngine:
       def inject_links(self, content, candidates)
       def _inject_link_into_html(self, html, link_data)
   

4. OptimizerService (business/optimization/services/optimizer_service.py)

   class OptimizerService:
       def __init__(self):
           self.analyzer = ContentAnalyzer()
           self.ai_function = OptimizeContentFunction()
           self.credit_service = CreditService()
   
       # Multiple entry points
       def optimize_from_writer(self, content_id)
       def optimize_from_wordpress_sync(self, content_id)
       def optimize_from_external_sync(self, content_id)
       def optimize_manual(self, content_id)
   
       # Unified optimization logic
       def optimize(self, content)
   

5. ContentAnalyzer (business/optimization/services/analyzer.py)

   class ContentAnalyzer:
       def analyze(self, content)
       def _calculate_seo_score(self, content)
       def _calculate_readability_score(self, content)
       def _calculate_engagement_score(self, content)
   

6. ContentPipelineService (business/content/services/content_pipeline_service.py)

   class ContentPipelineService:
       def __init__(self):
           self.linker_service = LinkerService()
           self.optimizer_service = OptimizerService()
   
       def process_writer_content(self, content_id, stages=['linking', 'optimization'])
       def process_synced_content(self, content_id, stages=['optimization'])
   

AI Functions

1. OptimizeContentFunction (infrastructure/ai/functions/optimize_content.py)
   • Operation type: optimization
   • Credit cost: 1 credit per 200 words
   • Optimizes content for SEO, readability, engagement

API Layer (modules/linker/ and modules/optimizer/)

modules/
├── linker/
│   ├── __init__.py
│   ├── views.py                    # LinkerViewSet
│   ├── serializers.py
│   ├── urls.py
│   └── apps.py
│
└── optimizer/
    ├── __init__.py
    ├── views.py                    # OptimizerViewSet
    ├── serializers.py
    ├── urls.py
    └── apps.py

ViewSets to Create:

1. LinkerViewSet (modules/linker/views.py)

   • Actions:
     • process/ (POST) - Process content for linking
     • batch_process/ (POST) - Process multiple content items

2. OptimizerViewSet (modules/optimizer/views.py)

   • Actions:
     • optimize/ (POST) - Optimize content (auto-detects source)
     • optimize_from_writer/ (POST) - Entry point 1
     • optimize_from_sync/ (POST) - Entry point 2 & 3
     • optimize_manual/ (POST) - Entry point 4
     • analyze/ (GET) - Analyze content without optimizing

4.2 Frontend Structure

Linker UI (frontend/src/pages/Linker/)

frontend/src/pages/Linker/
├── Dashboard.tsx
├── ContentList.tsx
└── LinkResults.tsx

Optimizer UI (frontend/src/pages/Optimizer/)

frontend/src/pages/Optimizer/
├── Dashboard.tsx
├── ContentSelector.tsx
├── OptimizationResults.tsx
└── ScoreComparison.tsx

Shared Components

frontend/src/components/
├── content/
│   ├── SourceBadge.tsx          # Show content source (IGNY8, WordPress, etc.)
│   ├── SyncStatusBadge.tsx      # Show sync status
│   ├── ContentFilter.tsx        # Filter by source, sync_status
│   └── SourceFilter.tsx

4.3 Implementation Tasks

Backend Tasks (Priority Order)

1. Extend Content Model

   • Add source field
   • Add sync_status field
   • Add external_id, external_url, sync_metadata
   • Add internal_links, linker_version
   • Add optimizer_version, optimization_scores
   • Create migration

2. Create Linking Services

   • Create business/linking/ folder
   • Create LinkerService
   • Create CandidateEngine
   • Create InjectionEngine
   • Integrate with CreditService

3. Create Optimization Services

   • Create business/optimization/ folder
   • Create OptimizationTask model
   • Create OptimizerService (with multiple entry points)
   • Create ContentAnalyzer
   • Integrate with CreditService

4. Create AI Function

   • Create OptimizeContentFunction
   • Add optimization prompts
   • Test AI function

5. Create Pipeline Service

   • Create ContentPipelineService
   • Integrate Linker and Optimizer

6. Create API Layer

   • Create modules/linker/ folder
   • Create LinkerViewSet
   • Create modules/optimizer/ folder
   • Create OptimizerViewSet
   • Create serializers
   • Register URLs

Frontend Tasks (Priority Order)

1. Create Linker UI

   • Linker Dashboard
   • Content List
   • Link Results display

2. Create Optimizer UI

   • Optimizer Dashboard
   • Content Selector (with source filters)
   • Optimization Results
   • Score Comparison

3. Create Shared Components

   • SourceBadge component
   • SyncStatusBadge component
   • ContentFilter component
   • SourceFilter component

4. Update Content List

   • Add source badges
   • Add sync status badges
   • Add filters (by source, sync_status)
   • Add "Send to Optimizer" button

---

INTEGRATION POINTS

Phase 3 Integration

1. With Phase 1 Services

   • StructureGenerationService uses CreditService
   • PageGenerationService uses ContentGenerationService
   • All operations check credits before execution

2. With Phase 2 Automation

   • Automation rules can trigger site structure generation
   • Automation can deploy sites automatically

3. With Content Service

   • Page generation reuses ContentGenerationService
   • Site pages stored as Content records

Phase 4 Integration

1. With Phase 1 Services

   • LinkerService uses CreditService
   • OptimizerService uses CreditService
   • ContentPipelineService orchestrates services

2. With Writer Module

   • Writer → Linker → Optimizer pipeline
   • Content generated in Writer flows to Linker/Optimizer

3. With WordPress Sync (Phase 6)

   • WordPress content synced with source='wordpress'
   • Optimizer works on synced content

4. With 3rd Party Sync (Phase 6)

   • External content synced with source='shopify' or source='custom'
   • Optimizer works on all sources

Cross-Phase Integration

1. Site Builder → Linker/Optimizer

   • Site pages can be optimized
   • Site content can be linked internally

2. Content Pipeline

   • Unified pipeline: Writer → Linker → Optimizer → Publish
   • Works for all content sources

---

FILE STRUCTURE

Complete Backend Structure

backend/igny8_core/
├── business/
│   ├── automation/          # Phase 2 ✅
│   ├── billing/             # Phase 0, 1 ✅
│   ├── content/             # Phase 1 ✅
│   │   └── services/
│   │       └── content_pipeline_service.py  # Phase 4 NEW
│   ├── linking/             # Phase 4 NEW
│   │   ├── models.py
│   │   └── services/
│   │       ├── linker_service.py
│   │       ├── candidate_engine.py
│   │       └── injection_engine.py
│   ├── optimization/        # Phase 4 NEW
│   │   ├── models.py
│   │   └── services/
│   │       ├── optimizer_service.py
│   │       └── analyzer.py
│   ├── planning/            # Phase 1 ✅
│   └── site_building/       # Phase 3 NEW
│       ├── models.py
│       └── services/
│           ├── file_management_service.py
│           ├── structure_generation_service.py
│           └── page_generation_service.py
│
├── modules/
│   ├── automation/          # Phase 2 ✅
│   ├── billing/             # Phase 0, 1 ✅
│   ├── linker/              # Phase 4 NEW
│   ├── optimizer/           # Phase 4 NEW
│   ├── planner/             # Phase 1 ✅
│   ├── site_builder/        # Phase 3 NEW
│   └── writer/              # Phase 1 ✅
│
└── infrastructure/
    └── ai/
        └── functions/
            ├── generate_site_structure.py  # Phase 3 NEW
            └── optimize_content.py          # Phase 4 NEW

Complete Frontend Structure

frontend/src/
├── components/
│   ├── shared/              # Phase 3 NEW
│   │   ├── blocks/
│   │   ├── layouts/
│   │   └── templates/
│   └── content/             # Phase 4 NEW
│       ├── SourceBadge.tsx
│       ├── SyncStatusBadge.tsx
│       └── ContentFilter.tsx
│
└── pages/
    ├── Linker/              # Phase 4 NEW
    │   ├── Dashboard.tsx
    │   └── ContentList.tsx
    ├── Optimizer/           # Phase 4 NEW
    │   ├── Dashboard.tsx
    │   └── ContentSelector.tsx
    └── Writer/              # Phase 1 ✅
        └── ...

site-builder/src/            # Phase 3 NEW
├── pages/
│   ├── wizard/
│   ├── preview/
│   └── dashboard/
└── components/

---

DEPENDENCIES & ORDER

Phase 3 Dependencies

1. Required (Already Complete)

   • ✅ Phase 0: Credit system
   • ✅ Phase 1: Service layer (ContentGenerationService, CreditService)
   • ✅ Phase 2: Automation system (optional integration)

2. Phase 3 Implementation Order

   • Step 1: Create models (SiteBlueprint, PageBlueprint)
   • Step 2: Create FileManagementService
   • Step 3: Create StructureGenerationService + AI function
   • Step 4: Create PageGenerationService
   • Step 5: Create API layer (ViewSets)
   • Step 6: Create frontend container structure
   • Step 7: Create wizard UI
   • Step 8: Create preview canvas
   • Step 9: Create shared component library

Phase 4 Dependencies

1. Required (Already Complete)

   • ✅ Phase 0: Credit system
   • ✅ Phase 1: Service layer (ContentGenerationService, CreditService)
   • ✅ Content model (needs extension)

2. Phase 4 Implementation Order

   • Step 1: Extend Content model (add source, sync_status, linking, optimization fields)
   • Step 2: Create linking services (LinkerService, CandidateEngine, InjectionEngine)
   • Step 3: Create optimization services (OptimizerService, ContentAnalyzer)
   • Step 4: Create optimization AI function
   • Step 5: Create ContentPipelineService
   • Step 6: Create API layer (LinkerViewSet, OptimizerViewSet)
   • Step 7: Create frontend UI (Linker Dashboard, Optimizer Dashboard)
   • Step 8: Create shared components (SourceBadge, ContentFilter)

Parallel Implementation

• Phase 3 and Phase 4 can be implemented in parallel after:
  • Content model extensions (Phase 4 Step 1) are complete
  • Both phases use Phase 1 services independently

---

TESTING STRATEGY

Phase 3 Testing

1. Backend Tests

   • Test SiteBlueprint CRUD
   • Test PageBlueprint CRUD
   • Test structure generation (AI function)
   • Test file upload/delete/access
   • Test credit deduction

2. Frontend Tests

   • Test wizard flow
   • Test preview rendering
   • Test file browser
   • Test component library

Phase 4 Testing

1. Backend Tests

   • Test Content model extensions
   • Test LinkerService (find candidates, inject links)
   • Test OptimizerService (all entry points)
   • Test ContentPipelineService
   • Test credit deduction

2. Frontend Tests

   • Test Linker UI
   • Test Optimizer UI
   • Test source filtering
   • Test content selection

Integration Tests

1. Writer → Linker → Optimizer Pipeline

   • Test full pipeline flow
   • Test credit deduction at each stage

2. WordPress Sync → Optimizer (Phase 6)

   • Test synced content optimization
   • Test source tracking

---

CREDIT COSTS

Phase 3 Credit Costs

• site_structure_generation: 50 credits (per site blueprint)
• site_page_generation: 20 credits (per page)
• File storage: No credits (storage quota based)

Phase 4 Credit Costs

• linking: 8 credits (per content piece)
• optimization: 1 credit per 200 words

---

SUCCESS CRITERIA

Phase 3 Success Criteria

• ✅ Site Builder wizard works end-to-end
• ✅ AI structure generation creates valid blueprints
• ✅ Preview renders correctly
• ✅ File management works
• ✅ Shared components work across apps
• ✅ Page generation reuses ContentGenerationService

Phase 4 Success Criteria

• ✅ Linker finds appropriate link candidates
• ✅ Links inject correctly into content
• ✅ Optimizer works from all entry points (Writer, WordPress, 3rd party, Manual)
• ✅ Content source tracking works
• ✅ Pipeline orchestrates correctly
• ✅ UI shows content sources and filters

---

RISK MITIGATION

Phase 3 Risks

1. AI Structure Generation Quality

   • Mitigation: Prompt engineering, validation, user feedback loop

2. Component Compatibility

   • Mitigation: Shared component library, comprehensive testing

3. File Management Security

   • Mitigation: Access control, validation, quota checks

Phase 4 Risks

1. Link Quality

   • Mitigation: Candidate scoring algorithm, relevance checks

2. Optimization Quality

   • Mitigation: Content analysis, before/after comparison, user review

3. Multiple Entry Points Complexity

   • Mitigation: Unified optimization logic, clear entry point methods

---

END OF IMPLEMENTATION PLAN