igny8/docs/planning/phases/PHASE-4-LINKER-OPTIMIZER.md

PHASE 4: LINKER & OPTIMIZER

Detailed Implementation Plan

Goal: Add linking and optimization as post-processing stages with multiple entry points.

Timeline: 4-5 weeks
Priority: MEDIUM
Dependencies: Phase 1

---

TABLE OF CONTENTS

1. Overview
2. Content Workflow & Entry Points
3. Content Model Extensions
4. Linker Implementation
5. Optimizer Implementation
6. Content Pipeline Service
7. Linker & Optimizer APIs
8. Linker & Optimizer UI
9. Testing & Validation
10. Implementation Checklist

---

OVERVIEW

Objectives

• ✅ Add internal linking to content
• ✅ Add content optimization
• ✅ Support multiple entry points (Writer, WordPress Sync, 3rd Party, Manual)
• ✅ Create content pipeline service
• ✅ Build UI for linker and optimizer

Key Principles

• Multiple Entry Points: Optimizer works from any content source
• Unified Content Model: All content stored in same model with source tracking
• Pipeline Orchestration: Linker → Optimizer → Publish workflow
• Source Agnostic: Optimizer works on any content regardless of source

---

CONTENT WORKFLOW & ENTRY POINTS

4.0 Content Workflow & Entry Points

Content Sources:

1. IGNY8 Generated - Content created via Writer module
2. WordPress Synced - Content synced from WordPress via plugin
3. 3rd Party Synced - Content synced from external sources (Shopify, custom APIs)

Workflow Entry Points:

Entry Point 1: Writer → Linker → Optimizer → Publish
Entry Point 2: WordPress Sync → Optimizer → Publish
Entry Point 3: 3rd Party Sync → Optimizer → Publish
Entry Point 4: Manual Selection → Linker/Optimizer

Content Storage Strategy:

• All content stored in unified Content model
• source field: 'igny8', 'wordpress', 'shopify', 'custom'
• sync_status field: 'native', 'imported', 'synced'

---

CONTENT MODEL EXTENSIONS

4.1 Content Model Extensions

Purpose: Add fields to track content source and sync status.

Task	File	Dependencies	Implementation
Add source field	domain/content/models.py	Phase 1	Track content source
Add sync_status field	domain/content/models.py	Phase 1	Track sync status
Add external_id field	domain/content/models.py	Phase 1	Store external platform ID
Add sync_metadata field	domain/content/models.py	Phase 1	Store platform-specific metadata

Content Model Extensions:

# domain/content/models.py
class Content(SiteSectorBaseModel):
    # Existing fields...
    
    # NEW: Source tracking
    source = models.CharField(
        max_length=50,
        choices=[
            ('igny8', 'IGNY8 Generated'),
            ('wordpress', 'WordPress Synced'),
            ('shopify', 'Shopify Synced'),
            ('custom', 'Custom API Synced'),
        ],
        default='igny8'
    )
    
    # NEW: Sync status
    sync_status = models.CharField(
        max_length=50,
        choices=[
            ('native', 'Native IGNY8 Content'),
            ('imported', 'Imported from External'),
            ('synced', 'Synced from External'),
        ],
        default='native'
    )
    
    # NEW: External reference
    external_id = models.CharField(max_length=255, blank=True, null=True)
    external_url = models.URLField(blank=True, null=True)
    sync_metadata = models.JSONField(default=dict)
    
    # NEW: Linking fields
    internal_links = models.JSONField(default=list)
    linker_version = models.IntegerField(default=0)
    
    # NEW: Optimization fields
    optimizer_version = models.IntegerField(default=0)
    optimization_scores = models.JSONField(default=dict)

---

LINKER IMPLEMENTATION

4.2 Linker Models

Task	File	Dependencies	Implementation
InternalLink Model	domain/linking/models.py	Phase 1	Store link relationships
LinkGraph Model	domain/linking/models.py	Phase 1	Store link graph

4.3 Linker Service

Task	File	Dependencies	Implementation
LinkerService	domain/linking/services/linker_service.py	Phase 1, ContentService	Main linking service
Link Candidate Engine	domain/linking/services/candidate_engine.py	Phase 1	Find link candidates
Link Injection Engine	domain/linking/services/injection_engine.py	Phase 1	Inject links into content

LinkerService:

# domain/linking/services/linker_service.py
class LinkerService:
    def process(self, content_id):
        """Process content for linking"""
        content = Content.objects.get(id=content_id)
        
        # Check credits
        credit_service.check_credits(content.account, 'linking')
        
        # Find link candidates
        candidates = self.candidate_engine.find_candidates(content)
        
        # Inject links
        linked_content = self.injection_engine.inject_links(content, candidates)
        
        # Update content
        content.internal_links = linked_content['links']
        content.linker_version += 1
        content.save()
        
        # Deduct credits
        credit_service.deduct_credits(content.account, 'linking')
        
        return content

---

OPTIMIZER IMPLEMENTATION

4.5 Optimizer Models

Task	File	Dependencies	Implementation
OptimizationTask Model	domain/optimization/models.py	Phase 1	Store optimization results
OptimizationScores Model	domain/optimization/models.py	Phase 1	Store optimization scores

4.6 Optimizer Service (Multiple Entry Points)

Task	File	Dependencies	Implementation
OptimizerService	domain/optimization/services/optimizer_service.py	Phase 1, ContentService	Main optimization service
Content Analyzer	domain/optimization/services/analyzer.py	Phase 1	Analyze content quality
Optimization AI Function	infrastructure/ai/functions/optimize_content.py	Existing AI framework	AI optimization function

OptimizerService:

# domain/optimization/services/optimizer_service.py
class OptimizerService:
    def optimize_from_writer(self, content_id):
        """Entry Point 1: Writer → Optimizer"""
        content = Content.objects.get(id=content_id, source='igny8')
        return self.optimize(content)
    
    def optimize_from_wordpress_sync(self, content_id):
        """Entry Point 2: WordPress Sync → Optimizer"""
        content = Content.objects.get(id=content_id, source='wordpress')
        return self.optimize(content)
    
    def optimize_from_external_sync(self, content_id):
        """Entry Point 3: External Sync → Optimizer"""
        content = Content.objects.get(id=content_id, source__in=['shopify', 'custom'])
        return self.optimize(content)
    
    def optimize_manual(self, content_id):
        """Entry Point 4: Manual Selection → Optimizer"""
        content = Content.objects.get(id=content_id)
        return self.optimize(content)
    
    def optimize(self, content):
        """Unified optimization logic"""
        # Check credits
        credit_service.check_credits(content.account, 'optimization', content.word_count)
        
        # Analyze content
        scores_before = self.analyzer.analyze(content)
        
        # Optimize content
        optimized = self.ai_function.optimize(content)
        
        # Analyze optimized content
        scores_after = self.analyzer.analyze(optimized)
        
        # Store optimization task
        OptimizationTask.objects.create(
            content=content,
            scores_before=scores_before,
            scores_after=scores_after,
            html_before=content.html_content,
            html_after=optimized['html_content'],
        )
        
        # Update content
        content.optimizer_version += 1
        content.optimization_scores = scores_after
        content.save()
        
        # Deduct credits
        credit_service.deduct_credits(content.account, 'optimization', content.word_count)
        
        return content

---

CONTENT PIPELINE SERVICE

4.7 Content Pipeline Service

Task	File	Dependencies	Implementation
ContentPipelineService	domain/content/services/content_pipeline_service.py	LinkerService, OptimizerService	Orchestrate content pipeline

Pipeline Workflow States:

Content States:
- 'draft' → Generated, not processed
- 'linked' → Links added, ready for optimization
- 'optimized' → Optimized, ready for review
- 'review' → Ready for publishing
- 'published' → Published to destination(s)

ContentPipelineService:

# domain/content/services/content_pipeline_service.py
class ContentPipelineService:
    def process_writer_content(self, content_id, stages=['linking', 'optimization']):
        """Writer → Linker → Optimizer pipeline"""
        content = Content.objects.get(id=content_id, source='igny8')
        
        if 'linking' in stages:
            content = linker_service.process(content.id)
        
        if 'optimization' in stages:
            content = optimizer_service.optimize_from_writer(content.id)
        
        return content
    
    def process_synced_content(self, content_id, stages=['optimization']):
        """Synced Content → Optimizer (skip linking if needed)"""
        content = Content.objects.get(id=content_id)
        
        if 'optimization' in stages:
            content = optimizer_service.optimize_manual(content.id)
        
        return content

---

LINKER & OPTIMIZER APIs

4.8 Linker & Optimizer APIs

Task	File	Dependencies	Implementation
Linker ViewSet	modules/linker/views.py	LinkerService	API for linker operations
Optimizer ViewSet	modules/optimizer/views.py	OptimizerService	API for optimizer operations

---

LINKER & OPTIMIZER UI

4.9 Linker & Optimizer UI

Task	File	Dependencies	Implementation
Linker Dashboard	frontend/src/pages/Linker/Dashboard.tsx	NEW	Linker overview
Optimizer Dashboard	frontend/src/pages/Optimizer/Dashboard.tsx	NEW	Optimizer overview
Content Selection UI	frontend/src/components/optimizer/ContentSelector.tsx	NEW	Select content for optimization
Source Badge Component	frontend/src/components/content/SourceBadge.tsx	NEW	Show content source

Optimizer UI Features:

• Show content source (IGNY8, WordPress, Shopify badge)
• Show sync status (Native, Synced, Imported badge)
• Entry point selection (from Writer, from Sync, Manual)
• Content list with source filters
• "Send to Optimizer" button (works for any source)

4.10 Content Filtering & Display

Task	File	Dependencies	Implementation
Content Filter Component	frontend/src/components/content/ContentFilter.tsx	NEW	Filter content by source
Source Filter	frontend/src/components/content/SourceFilter.tsx	NEW	Filter by source
Sync Status Filter	frontend/src/components/content/SyncStatusFilter.tsx	NEW	Filter by sync status

---

TESTING & VALIDATION

4.11 Testing

Test Cases:

• ✅ Writer → Linker handover works
• ✅ Linker finds appropriate link candidates
• ✅ Links inject correctly into content
• ✅ Optimizer works from Writer entry point
• ✅ Optimizer works from WordPress sync entry point
• ✅ Optimizer works from 3rd party sync entry point
• ✅ Optimizer works from manual selection
• ✅ Synced content stored correctly with source flags
• ✅ Content filtering works (by source, sync_status)
• ✅ Pipeline orchestrates correctly
• ✅ All entry points use same optimization logic

---

IMPLEMENTATION CHECKLIST

Backend Tasks

• Extend Content model with source/sync fields
• Create domain/linking/models.py
• Create LinkerService
• Create domain/optimization/models.py
• Create OptimizerService
• Create optimization AI function
• Create ContentPipelineService
• Create Linker ViewSet
• Create Optimizer ViewSet
• Create content sync service (for Phase 6)

Frontend Tasks

• Create Linker Dashboard
• Create Optimizer Dashboard
• Create content selection UI
• Create source badge component
• Create content filters
• Update content list with filters

---

SUCCESS CRITERIA

• ✅ Writer → Linker handover works
• ✅ Optimizer works from all entry points
• ✅ Content source tracking works
• ✅ Pipeline orchestrates correctly
• ✅ UI shows content sources and filters

---

END OF PHASE 4 DOCUMENT