9 phases

2025-11-16 23:15:45 +05:00
parent 1c8c44ebe0
commit f84be4194f
10 changed files with 3514 additions and 0 deletions
--- a/docs/planning/phases/PHASE-4-LINKER-OPTIMIZER.md
+++ b/docs/planning/phases/PHASE-4-LINKER-OPTIMIZER.md
@@ -0,0 +1,391 @@
+# PHASE 4: LINKER & OPTIMIZER
+**Detailed Implementation Plan**
+
+**Goal**: Add linking and optimization as post-processing stages with multiple entry points.
+
+**Timeline**: 4-5 weeks  
+**Priority**: MEDIUM  
+**Dependencies**: Phase 1
+
+---
+
+## TABLE OF CONTENTS
+
+1. [Overview](#overview)
+2. [Content Workflow & Entry Points](#content-workflow--entry-points)
+3. [Content Model Extensions](#content-model-extensions)
+4. [Linker Implementation](#linker-implementation)
+5. [Optimizer Implementation](#optimizer-implementation)
+6. [Content Pipeline Service](#content-pipeline-service)
+7. [Linker & Optimizer APIs](#linker--optimizer-apis)
+8. [Linker & Optimizer UI](#linker--optimizer-ui)
+9. [Testing & Validation](#testing--validation)
+10. [Implementation Checklist](#implementation-checklist)
+
+---
+
+## OVERVIEW
+
+### Objectives
+- ✅ Add internal linking to content
+- ✅ Add content optimization
+- ✅ Support multiple entry points (Writer, WordPress Sync, 3rd Party, Manual)
+- ✅ Create content pipeline service
+- ✅ Build UI for linker and optimizer
+
+### Key Principles
+- **Multiple Entry Points**: Optimizer works from any content source
+- **Unified Content Model**: All content stored in same model with source tracking
+- **Pipeline Orchestration**: Linker → Optimizer → Publish workflow
+- **Source Agnostic**: Optimizer works on any content regardless of source
+
+---
+
+## CONTENT WORKFLOW & ENTRY POINTS
+
+### 4.0 Content Workflow & Entry Points
+
+**Content Sources**:
+1. **IGNY8 Generated** - Content created via Writer module
+2. **WordPress Synced** - Content synced from WordPress via plugin
+3. **3rd Party Synced** - Content synced from external sources (Shopify, custom APIs)
+
+**Workflow Entry Points**:
+```
+Entry Point 1: Writer → Linker → Optimizer → Publish
+Entry Point 2: WordPress Sync → Optimizer → Publish
+Entry Point 3: 3rd Party Sync → Optimizer → Publish
+Entry Point 4: Manual Selection → Linker/Optimizer
+```
+
+**Content Storage Strategy**:
+- All content stored in unified `Content` model
+- `source` field: `'igny8'`, `'wordpress'`, `'shopify'`, `'custom'`
+- `sync_status` field: `'native'`, `'imported'`, `'synced'`
+
+---
+
+## CONTENT MODEL EXTENSIONS
+
+### 4.1 Content Model Extensions
+
+**Purpose**: Add fields to track content source and sync status.
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **Add source field** | `domain/content/models.py` | Phase 1 | Track content source |
+| **Add sync_status field** | `domain/content/models.py` | Phase 1 | Track sync status |
+| **Add external_id field** | `domain/content/models.py` | Phase 1 | Store external platform ID |
+| **Add sync_metadata field** | `domain/content/models.py` | Phase 1 | Store platform-specific metadata |
+
+**Content Model Extensions**:
+```python
+# domain/content/models.py
+class Content(SiteSectorBaseModel):
+    # Existing fields...
+    
+    # NEW: Source tracking
+    source = models.CharField(
+        max_length=50,
+        choices=[
+            ('igny8', 'IGNY8 Generated'),
+            ('wordpress', 'WordPress Synced'),
+            ('shopify', 'Shopify Synced'),
+            ('custom', 'Custom API Synced'),
+        ],
+        default='igny8'
+    )
+    
+    # NEW: Sync status
+    sync_status = models.CharField(
+        max_length=50,
+        choices=[
+            ('native', 'Native IGNY8 Content'),
+            ('imported', 'Imported from External'),
+            ('synced', 'Synced from External'),
+        ],
+        default='native'
+    )
+    
+    # NEW: External reference
+    external_id = models.CharField(max_length=255, blank=True, null=True)
+    external_url = models.URLField(blank=True, null=True)
+    sync_metadata = models.JSONField(default=dict)
+    
+    # NEW: Linking fields
+    internal_links = models.JSONField(default=list)
+    linker_version = models.IntegerField(default=0)
+    
+    # NEW: Optimization fields
+    optimizer_version = models.IntegerField(default=0)
+    optimization_scores = models.JSONField(default=dict)
+```
+
+---
+
+## LINKER IMPLEMENTATION
+
+### 4.2 Linker Models
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **InternalLink Model** | `domain/linking/models.py` | Phase 1 | Store link relationships |
+| **LinkGraph Model** | `domain/linking/models.py` | Phase 1 | Store link graph |
+
+### 4.3 Linker Service
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **LinkerService** | `domain/linking/services/linker_service.py` | Phase 1, ContentService | Main linking service |
+| **Link Candidate Engine** | `domain/linking/services/candidate_engine.py` | Phase 1 | Find link candidates |
+| **Link Injection Engine** | `domain/linking/services/injection_engine.py` | Phase 1 | Inject links into content |
+
+**LinkerService**:
+```python
+# domain/linking/services/linker_service.py
+class LinkerService:
+    def process(self, content_id):
+        """Process content for linking"""
+        content = Content.objects.get(id=content_id)
+        
+        # Check credits
+        credit_service.check_credits(content.account, 'linking')
+        
+        # Find link candidates
+        candidates = self.candidate_engine.find_candidates(content)
+        
+        # Inject links
+        linked_content = self.injection_engine.inject_links(content, candidates)
+        
+        # Update content
+        content.internal_links = linked_content['links']
+        content.linker_version += 1
+        content.save()
+        
+        # Deduct credits
+        credit_service.deduct_credits(content.account, 'linking')
+        
+        return content
+```
+
+---
+
+## OPTIMIZER IMPLEMENTATION
+
+### 4.5 Optimizer Models
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **OptimizationTask Model** | `domain/optimization/models.py` | Phase 1 | Store optimization results |
+| **OptimizationScores Model** | `domain/optimization/models.py` | Phase 1 | Store optimization scores |
+
+### 4.6 Optimizer Service (Multiple Entry Points)
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **OptimizerService** | `domain/optimization/services/optimizer_service.py` | Phase 1, ContentService | Main optimization service |
+| **Content Analyzer** | `domain/optimization/services/analyzer.py` | Phase 1 | Analyze content quality |
+| **Optimization AI Function** | `infrastructure/ai/functions/optimize_content.py` | Existing AI framework | AI optimization function |
+
+**OptimizerService**:
+```python
+# domain/optimization/services/optimizer_service.py
+class OptimizerService:
+    def optimize_from_writer(self, content_id):
+        """Entry Point 1: Writer → Optimizer"""
+        content = Content.objects.get(id=content_id, source='igny8')
+        return self.optimize(content)
+    
+    def optimize_from_wordpress_sync(self, content_id):
+        """Entry Point 2: WordPress Sync → Optimizer"""
+        content = Content.objects.get(id=content_id, source='wordpress')
+        return self.optimize(content)
+    
+    def optimize_from_external_sync(self, content_id):
+        """Entry Point 3: External Sync → Optimizer"""
+        content = Content.objects.get(id=content_id, source__in=['shopify', 'custom'])
+        return self.optimize(content)
+    
+    def optimize_manual(self, content_id):
+        """Entry Point 4: Manual Selection → Optimizer"""
+        content = Content.objects.get(id=content_id)
+        return self.optimize(content)
+    
+    def optimize(self, content):
+        """Unified optimization logic"""
+        # Check credits
+        credit_service.check_credits(content.account, 'optimization', content.word_count)
+        
+        # Analyze content
+        scores_before = self.analyzer.analyze(content)
+        
+        # Optimize content
+        optimized = self.ai_function.optimize(content)
+        
+        # Analyze optimized content
+        scores_after = self.analyzer.analyze(optimized)
+        
+        # Store optimization task
+        OptimizationTask.objects.create(
+            content=content,
+            scores_before=scores_before,
+            scores_after=scores_after,
+            html_before=content.html_content,
+            html_after=optimized['html_content'],
+        )
+        
+        # Update content
+        content.optimizer_version += 1
+        content.optimization_scores = scores_after
+        content.save()
+        
+        # Deduct credits
+        credit_service.deduct_credits(content.account, 'optimization', content.word_count)
+        
+        return content
+```
+
+---
+
+## CONTENT PIPELINE SERVICE
+
+### 4.7 Content Pipeline Service
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **ContentPipelineService** | `domain/content/services/content_pipeline_service.py` | LinkerService, OptimizerService | Orchestrate content pipeline |
+
+**Pipeline Workflow States**:
+```
+Content States:
+- 'draft' → Generated, not processed
+- 'linked' → Links added, ready for optimization
+- 'optimized' → Optimized, ready for review
+- 'review' → Ready for publishing
+- 'published' → Published to destination(s)
+```
+
+**ContentPipelineService**:
+```python
+# domain/content/services/content_pipeline_service.py
+class ContentPipelineService:
+    def process_writer_content(self, content_id, stages=['linking', 'optimization']):
+        """Writer → Linker → Optimizer pipeline"""
+        content = Content.objects.get(id=content_id, source='igny8')
+        
+        if 'linking' in stages:
+            content = linker_service.process(content.id)
+        
+        if 'optimization' in stages:
+            content = optimizer_service.optimize_from_writer(content.id)
+        
+        return content
+    
+    def process_synced_content(self, content_id, stages=['optimization']):
+        """Synced Content → Optimizer (skip linking if needed)"""
+        content = Content.objects.get(id=content_id)
+        
+        if 'optimization' in stages:
+            content = optimizer_service.optimize_manual(content.id)
+        
+        return content
+```
+
+---
+
+## LINKER & OPTIMIZER APIs
+
+### 4.8 Linker & Optimizer APIs
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **Linker ViewSet** | `modules/linker/views.py` | LinkerService | API for linker operations |
+| **Optimizer ViewSet** | `modules/optimizer/views.py` | OptimizerService | API for optimizer operations |
+
+---
+
+## LINKER & OPTIMIZER UI
+
+### 4.9 Linker & Optimizer UI
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **Linker Dashboard** | `frontend/src/pages/Linker/Dashboard.tsx` | NEW | Linker overview |
+| **Optimizer Dashboard** | `frontend/src/pages/Optimizer/Dashboard.tsx` | NEW | Optimizer overview |
+| **Content Selection UI** | `frontend/src/components/optimizer/ContentSelector.tsx` | NEW | Select content for optimization |
+| **Source Badge Component** | `frontend/src/components/content/SourceBadge.tsx` | NEW | Show content source |
+
+**Optimizer UI Features**:
+- Show content source (IGNY8, WordPress, Shopify badge)
+- Show sync status (Native, Synced, Imported badge)
+- Entry point selection (from Writer, from Sync, Manual)
+- Content list with source filters
+- "Send to Optimizer" button (works for any source)
+
+### 4.10 Content Filtering & Display
+
+| Task | File | Dependencies | Implementation |
+|------|------|--------------|----------------|
+| **Content Filter Component** | `frontend/src/components/content/ContentFilter.tsx` | NEW | Filter content by source |
+| **Source Filter** | `frontend/src/components/content/SourceFilter.tsx` | NEW | Filter by source |
+| **Sync Status Filter** | `frontend/src/components/content/SyncStatusFilter.tsx` | NEW | Filter by sync status |
+
+---
+
+## TESTING & VALIDATION
+
+### 4.11 Testing
+
+**Test Cases**:
+- ✅ Writer → Linker handover works
+- ✅ Linker finds appropriate link candidates
+- ✅ Links inject correctly into content
+- ✅ Optimizer works from Writer entry point
+- ✅ Optimizer works from WordPress sync entry point
+- ✅ Optimizer works from 3rd party sync entry point
+- ✅ Optimizer works from manual selection
+- ✅ Synced content stored correctly with source flags
+- ✅ Content filtering works (by source, sync_status)
+- ✅ Pipeline orchestrates correctly
+- ✅ All entry points use same optimization logic
+
+---
+
+## IMPLEMENTATION CHECKLIST
+
+### Backend Tasks
+
+- [ ] Extend Content model with source/sync fields
+- [ ] Create `domain/linking/models.py`
+- [ ] Create LinkerService
+- [ ] Create `domain/optimization/models.py`
+- [ ] Create OptimizerService
+- [ ] Create optimization AI function
+- [ ] Create ContentPipelineService
+- [ ] Create Linker ViewSet
+- [ ] Create Optimizer ViewSet
+- [ ] Create content sync service (for Phase 6)
+
+### Frontend Tasks
+
+- [ ] Create Linker Dashboard
+- [ ] Create Optimizer Dashboard
+- [ ] Create content selection UI
+- [ ] Create source badge component
+- [ ] Create content filters
+- [ ] Update content list with filters
+
+---
+
+## SUCCESS CRITERIA
+
+- ✅ Writer → Linker handover works
+- ✅ Optimizer works from all entry points
+- ✅ Content source tracking works
+- ✅ Pipeline orchestrates correctly
+- ✅ UI shows content sources and filters
+
+---
+
+**END OF PHASE 4 DOCUMENT**
+