diff --git a/CHANGELOG.md b/CHANGELOG.md index 489e48eb..d8b9b3fb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -26,6 +26,19 @@ Each entry follows this format: ## [Unreleased] +### Added +- **Planning Documents Organization**: Organized architecture and implementation planning documents + - Created `docs/planning/` directory for all planning documents + - Moved `IGNY8-HOLISTIC-ARCHITECTURE-PLAN.md` to `docs/planning/` + - Moved `IGNY8-IMPLEMENTATION-PLAN.md` to `docs/planning/` + - Moved `Igny8-phase-2-plan.md` to `docs/planning/` + - Moved `CONTENT-WORKFLOW-DIAGRAM.md` to `docs/planning/` + - Moved `ARCHITECTURE_CONTEXT.md` to `docs/planning/` + - Moved `sample-usage-limits-credit-system` to `docs/planning/` + - Created `docs/refactor/` directory for refactoring plans + - Updated `README.md` to reflect new document structure + - **Impact**: Better organization of planning documents, easier to find and maintain + ### Changed - **API Documentation Consolidation**: Consolidated all API documentation into single comprehensive reference - Created `docs/API-COMPLETE-REFERENCE.md` - Unified API documentation covering all endpoints, authentication, response formats, error handling, rate limiting, permissions, and integration examples diff --git a/IGNY8-IMPLEMENTATION-PLAN.md b/IGNY8-IMPLEMENTATION-PLAN.md deleted file mode 100644 index e25b9f64..00000000 --- a/IGNY8-IMPLEMENTATION-PLAN.md +++ /dev/null @@ -1,591 +0,0 @@ -# IGNY8 IMPLEMENTATION PLAN -**Preserving Existing Functionality + Phase 2 Features** - -**Last Updated:** 2025-01-XX -**Purpose:** Step-by-step implementation plan that keeps everything working while adding Phase 2 features. - ---- - -## TABLE OF CONTENTS - -1. [Implementation Strategy](#1-implementation-strategy) -2. [Phase 0: Foundation & Credit System](#phase-0-foundation--credit-system) -3. [Phase 1: Service Layer Refactoring](#phase-1-service-layer-refactoring) -4. [Phase 2: Automation System](#phase-2-automation-system) -5. [Phase 3: Site Builder](#phase-3-site-builder) -6. [Phase 4: Linker & Optimizer](#phase-4-linker--optimizer) -7. [Phase 5: Sites Renderer](#phase-5-sites-renderer) -8. [Phase 6: Site Integration & Multi-Destination Publishing](#phase-6-site-integration--multi-destination-publishing) -9. [Phase 7: Universal Content Types](#phase-7-universal-content-types) -10. [Testing & Validation](#testing--validation) - ---- - -## 1. IMPLEMENTATION STRATEGY - -### 1.1 Core Principles - -| Principle | Implementation | -|-----------|----------------| -| **Backward Compatibility** | All existing APIs, workflows, and features continue working | -| **Incremental Changes** | Add new features without breaking existing ones | -| **Feature Flags** | Use flags to enable/disable new features during rollout | -| **Parallel Development** | New features developed alongside existing code | -| **Gradual Migration** | Move to new architecture gradually, not all at once | - -### 1.2 What's Currently Working (PRESERVE) - -| Module | Features | Status | -|--------|----------|--------| -| **Planner** | Keywords, Clusters, Ideas | ✅ Working | -| **Writer** | Tasks, Content Generation, Images | ✅ Working | -| **Thinker** | Prompts, Author Profiles, Strategies | ✅ Working | -| **Billing** | Credits, Transactions, Usage Logs | ✅ Working | -| **System** | Settings, Integration Settings | ✅ Working | -| **Auth** | Multi-tenancy, Users, Sites, Sectors | ✅ Working | -| **AI Framework** | 5 AI functions (cluster, ideas, content, prompts, images) | ✅ Working | -| **WordPress Publishing** | Content publishing to WordPress | ✅ Working | - -### 1.3 What Needs to Be Built (PHASE 2) - -| Feature | Status | Priority | -|---------|--------|----------| -| **Credit-Only System** | 🔨 Build | HIGH | -| **Automation System** | 🔨 Build | HIGH | -| **Service Layer** | 🔨 Build | HIGH | -| **Site Builder** | 🔨 Build | HIGH | -| **Linker** | 🔨 Build | MEDIUM | -| **Optimizer** | 🔨 Build | MEDIUM | -| **Sites Renderer** | 🔨 Build | MEDIUM | -| **Site Integration** | 🔨 Build | MEDIUM | -| **Universal Content Types** | 🔨 Build | LOW | - ---- - -## PHASE 0: FOUNDATION & CREDIT SYSTEM - -**Goal**: Migrate to credit-only model while preserving all existing functionality. - -### 0.1 Credit System Updates - -| Task | Files | Dependencies | Risk | -|------|-------|--------------|------| -| **Remove Plan Limit Fields** | `core/auth/models.py` | None | LOW - Add migration to set defaults | -| **Update Plan Model** | `core/auth/models.py` | None | LOW - Keep only monthly_credits, support_level | -| **Update CreditService** | `domain/billing/services/credit_service.py` | None | MEDIUM - Add credit cost constants | -| **Add Credit Costs** | `domain/billing/constants.py` | None | LOW - Define credit costs per operation | -| **Update AI Engine** | `infrastructure/ai/engine.py` | CreditService | MEDIUM - Check credits before AI calls | -| **Update Content Generation** | `domain/content/services/` | CreditService | MEDIUM - Check credits before generation | -| **Update Image Generation** | `infrastructure/ai/functions/generate_images.py` | CreditService | MEDIUM - Check credits before generation | -| **Remove Limit Checks** | All services | None | MEDIUM - Remove all plan limit validations | -| **Update Usage Logging** | `domain/billing/models.py` | None | LOW - Ensure all operations log credits | -| **Update Frontend Limits UI** | `frontend/src/pages/Billing/` | Backend API | LOW - Replace limits with credit display | - -**Credit Cost Constants**: -```python -# domain/billing/constants.py -CREDIT_COSTS = { - 'clustering': 10, - 'idea_generation': 15, - 'content_generation': 1, # per 100 words - 'image_prompt_extraction': 2, - 'image_generation': 5, - 'linking': 8, # NEW - 'optimization': 1, # per 200 words, NEW - 'site_structure_generation': 50, # NEW - 'site_page_generation': 20, # NEW -} -``` - -### 0.2 Operational Limits (Keep) - -| Limit | Location | Implementation | -|-------|----------|----------------| -| **50 keywords per request** | `modules/planner/views.py` | Request validation | -| **6 images per request** | `modules/writer/views.py` | Request validation | - -### 0.3 Database Migrations - -| Migration | Purpose | Risk | -|-----------|---------|------| -| **Remove limit fields from Plan** | Clean up unused fields | LOW - Add defaults first | -| **Add credit cost tracking** | Enhance CreditUsageLog | LOW - Additive only | -| **Monthly credit replenishment** | Celery Beat task | LOW - New feature | - -### 0.4 Testing - -- ✅ All existing features work with credit checks -- ✅ Credit deduction happens correctly -- ✅ Insufficient credits show clear error -- ✅ Usage logging tracks all operations -- ✅ Frontend shows credit balance, not limits - -**Timeline**: 1-2 weeks - ---- - -## PHASE 1: SERVICE LAYER REFACTORING - -**Goal**: Extract business logic from ViewSets into services, preserving all existing functionality. - -### 1.1 Create Domain Structure - -| Task | Files | Dependencies | -|------|-------|--------------| -| **Create domain/ folder** | `backend/igny8_core/domain/` | None | -| **Move Content models** | `domain/content/models.py` | Phase 0 | -| **Move Planning models** | `domain/planning/models.py` | Phase 0 | -| **Create ContentService** | `domain/content/services/content_generation_service.py` | Existing Writer logic | -| **Create PlanningService** | `domain/planning/services/clustering_service.py` | Existing Planner logic | -| **Create IdeasService** | `domain/planning/services/ideas_service.py` | Existing Planner logic | - -### 1.2 Refactor ViewSets (Keep APIs Working) - -| Module | Current | New | Risk | -|--------|---------|-----|------| -| **Planner ViewSets** | Business logic in views | Delegate to services | LOW - Internal refactor | -| **Writer ViewSets** | Business logic in views | Delegate to services | LOW - Internal refactor | -| **Billing ViewSets** | Already uses CreditService | Keep as-is | NONE | - -**Strategy**: -- ViewSets become thin wrappers -- All business logic moves to services -- APIs remain unchanged (backward compatible) - -### 1.3 Testing - -- ✅ All existing API endpoints work identically -- ✅ Response formats unchanged -- ✅ No breaking changes for frontend -- ✅ Services are testable independently - -**Timeline**: 2-3 weeks - ---- - -## PHASE 2: AUTOMATION SYSTEM - -**Goal**: Implement automation rules and scheduled tasks. - -### 2.1 Automation Models - -| Task | Files | Dependencies | -|------|-------|--------------| -| **AutomationRule Model** | `domain/automation/models.py` | Phase 1 | -| **ScheduledTask Model** | `domain/automation/models.py` | Phase 1 | -| **Automation Migrations** | `domain/automation/migrations/` | Phase 1 | - -### 2.2 Automation Service - -| Task | Files | Dependencies | -|------|-------|--------------| -| **AutomationService** | `domain/automation/services/automation_service.py` | Phase 1 services | -| **Rule Execution Engine** | `domain/automation/services/rule_engine.py` | Phase 1 services | -| **Condition Evaluator** | `domain/automation/services/condition_evaluator.py` | None | -| **Action Executor** | `domain/automation/services/action_executor.py` | Phase 1 services | - -### 2.3 Celery Beat Tasks - -| Task | Files | Dependencies | -|------|-------|--------------| -| **Scheduled Automation Task** | `infrastructure/messaging/automation_tasks.py` | AutomationService | -| **Monthly Credit Replenishment** | `infrastructure/messaging/automation_tasks.py` | CreditService | -| **Celery Beat Configuration** | `backend/igny8_core/celery.py` | None | - -### 2.4 Automation API - -| Task | Files | Dependencies | -|------|-------|--------------| -| **AutomationRule ViewSet** | `modules/automation/views.py` | AutomationService | -| **ScheduledTask ViewSet** | `modules/automation/views.py` | AutomationService | -| **Automation URLs** | `modules/automation/urls.py` | None | - -### 2.5 Automation UI - -| Task | Files | Dependencies | -|------|-------|--------------| -| **Automation Dashboard** | `frontend/src/pages/Automation/Dashboard.tsx` | EXISTING (placeholder) | -| **Rules Management** | `frontend/src/pages/Automation/Rules.tsx` | NEW | -| **Schedules Page** | `frontend/src/pages/Schedules.tsx` | EXISTING (placeholder) | -| **Automation API Client** | `frontend/src/services/automation.api.ts` | NEW | - -### 2.6 Testing - -- ✅ Automation rules execute correctly -- ✅ Scheduled tasks run on time -- ✅ Credit replenishment works monthly -- ✅ UI shows automation status - -**Timeline**: 2-3 weeks - ---- - -## PHASE 3: SITE BUILDER - -**Goal**: Build Site Builder for creating sites via wizard. - -### 3.1 Site Builder Models - -| Task | Files | Dependencies | -|------|-------|--------------| -| **SiteBlueprint Model** | `domain/site_building/models.py` | Phase 1 | -| **PageBlueprint Model** | `domain/site_building/models.py` | Phase 1 | -| **Site Builder Migrations** | `domain/site_building/migrations/` | Phase 1 | - -### 3.2 Site Structure Generation - -| Task | Files | Dependencies | -|------|-------|--------------| -| **Structure Generation AI Function** | `infrastructure/ai/functions/generate_site_structure.py` | Existing AI framework | -| **Structure Generation Service** | `domain/site_building/services/structure_generation_service.py` | Phase 1, AI framework | -| **Site Structure Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system | - -### 3.3 Site Builder API - -| Task | Files | Dependencies | -|------|-------|--------------| -| **Site Builder ViewSet** | `modules/site_builder/views.py` | Structure Generation Service | -| **Site Builder URLs** | `modules/site_builder/urls.py` | None | -| **Site Builder Serializers** | `modules/site_builder/serializers.py` | None | - -### 3.4 Site Builder Frontend (New Container) - -| Task | Files | Dependencies | -|------|-------|--------------| -| **Create Site Builder Container** | `docker-compose.app.yml` | None | -| **Wizard Steps** | `site-builder/src/pages/wizard/` | NEW | -| **Preview Canvas** | `site-builder/src/pages/preview/` | NEW | -| **Site Builder State** | `site-builder/src/state/builderStore.ts` | NEW | -| **Site Builder API Client** | `site-builder/src/api/builder.api.ts` | NEW | - -### 3.5 Page Generation (Reuse Content Service) - -| Task | Files | Dependencies | -|------|-------|--------------| -| **Extend ContentService** | `domain/content/services/content_generation_service.py` | Phase 1 | -| **Add Site Page Type** | `domain/content/models.py` | Phase 1 | -| **Page Generation Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system | - -### 3.6 Testing - -- ✅ Site Builder wizard works end-to-end -- ✅ Structure generation creates valid blueprints -- ✅ Preview renders correctly -- ✅ Page generation reuses existing content service - -**Timeline**: 3-4 weeks - ---- - -## PHASE 4: LINKER & OPTIMIZER - -**Goal**: Add linking and optimization as post-processing stages. - -### 4.1 Linker Models - -| Task | Files | Dependencies | -|------|-------|--------------| -| **InternalLink Model** | `domain/linking/models.py` | Phase 1 | -| **LinkGraph Model** | `domain/linking/models.py` | Phase 1 | -| **Linker Migrations** | `domain/linking/migrations/` | Phase 1 | - -### 4.2 Linker Service - -| Task | Files | Dependencies | -|------|-------|--------------| -| **LinkerService** | `domain/linking/services/linker_service.py` | Phase 1, ContentService | -| **Link Candidate Engine** | `domain/linking/services/candidate_engine.py` | Phase 1 | -| **Link Injection Engine** | `domain/linking/services/injection_engine.py` | Phase 1 | - -### 4.3 Optimizer Models - -| Task | Files | Dependencies | -|------|-------|--------------| -| **OptimizationTask Model** | `domain/optimization/models.py` | Phase 1 | -| **OptimizationScores Model** | `domain/optimization/models.py` | Phase 1 | -| **Optimizer Migrations** | `domain/optimization/migrations/` | Phase 1 | - -### 4.4 Optimizer Service - -| Task | Files | Dependencies | -|------|-------|--------------| -| **OptimizerService** | `domain/optimization/services/optimizer_service.py` | Phase 1, ContentService | -| **Content Analyzer** | `domain/optimization/services/analyzer.py` | Phase 1 | -| **Optimization AI Function** | `infrastructure/ai/functions/optimize_content.py` | Existing AI framework | -| **Optimization Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system | - -### 4.5 Content Pipeline Service - -| Task | Files | Dependencies | -|------|-------|--------------| -| **ContentPipelineService** | `domain/content/services/content_pipeline_service.py` | LinkerService, OptimizerService | -| **Pipeline Orchestration** | `domain/content/services/pipeline_service.py` | Phase 1 services | - -### 4.6 Linker & Optimizer APIs - -| Task | Files | Dependencies | -|------|-------|--------------| -| **Linker ViewSet** | `modules/linker/views.py` | LinkerService | -| **Optimizer ViewSet** | `modules/optimizer/views.py` | OptimizerService | -| **Linker URLs** | `modules/linker/urls.py` | None | -| **Optimizer URLs** | `modules/optimizer/urls.py` | None | - -### 4.7 Linker & Optimizer UI - -| Task | Files | Dependencies | -|------|-------|--------------| -| **Linker Dashboard** | `frontend/src/pages/Linker/Dashboard.tsx` | NEW | -| **Optimizer Dashboard** | `frontend/src/pages/Optimizer/Dashboard.tsx` | NEW | -| **Linker API Client** | `frontend/src/services/linker.api.ts` | NEW | -| **Optimizer API Client** | `frontend/src/services/optimizer.api.ts` | NEW | - -### 4.8 Testing - -- ✅ Linker finds appropriate link candidates -- ✅ Links inject correctly into content -- ✅ Optimizer improves content quality -- ✅ Pipeline orchestrates correctly - -**Timeline**: 3-4 weeks - ---- - -## PHASE 5: SITES RENDERER - -**Goal**: Build Sites renderer for hosting public sites. - -### 5.1 Sites Renderer Container - -| Task | Files | Dependencies | -|------|-------|--------------| -| **Create Sites Container** | `docker-compose.app.yml` | None | -| **Sites Renderer Frontend** | `sites/src/` | NEW | -| **Block Component Library** | `sites/src/components/Blocks/` | NEW | -| **Site Definition Loader** | `sites/src/loaders/loadSiteDefinition.ts` | NEW | - -### 5.2 Publisher Service - -| Task | Files | Dependencies | -|------|-------|--------------| -| **PublisherService** | `domain/publishing/services/publisher_service.py` | Phase 1 | -| **SitesRendererAdapter** | `domain/publishing/services/adapters/sites_renderer_adapter.py` | Phase 3 | -| **DeploymentService** | `domain/publishing/services/deployment_service.py` | Phase 3 | - -### 5.3 Publishing Models - -| Task | Files | Dependencies | -|------|-------|--------------| -| **PublishingRecord Model** | `domain/publishing/models.py` | Phase 1 | -| **DeploymentRecord Model** | `domain/publishing/models.py` | Phase 3 | -| **Publishing Migrations** | `domain/publishing/migrations/` | Phase 1 | - -### 5.4 Publisher API - -| Task | Files | Dependencies | -|------|-------|--------------| -| **Publisher ViewSet** | `modules/publisher/views.py` | PublisherService | -| **Publisher URLs** | `modules/publisher/urls.py` | None | - -### 5.5 Testing - -- ✅ Sites renderer loads site definitions -- ✅ Blocks render correctly -- ✅ Deployment works end-to-end -- ✅ Sites are accessible publicly - -**Timeline**: 2-3 weeks - ---- - -## PHASE 6: SITE INTEGRATION & MULTI-DESTINATION PUBLISHING - -**Goal**: Support multiple publishing destinations (WordPress, Sites, Shopify). - -### 6.1 Site Integration Models - -| Task | Files | Dependencies | -|------|-------|--------------| -| **SiteIntegration Model** | `domain/integration/models.py` | Phase 1 | -| **Integration Migrations** | `domain/integration/migrations/` | Phase 1 | - -### 6.2 Integration Service - -| Task | Files | Dependencies | -|------|-------|--------------| -| **IntegrationService** | `domain/integration/services/integration_service.py` | Phase 1 | -| **SyncService** | `domain/integration/services/sync_service.py` | Phase 1 | - -### 6.3 Publishing Adapters - -| Task | Files | Dependencies | -|------|-------|--------------| -| **BaseAdapter** | `domain/publishing/services/adapters/base_adapter.py` | Phase 5 | -| **WordPressAdapter** | `domain/publishing/services/adapters/wordpress_adapter.py` | EXISTING (refactor) | -| **SitesRendererAdapter** | `domain/publishing/services/adapters/sites_renderer_adapter.py` | Phase 5 | -| **ShopifyAdapter** | `domain/publishing/services/adapters/shopify_adapter.py` | Phase 5 (future) | - -### 6.4 Multi-Destination Publishing - -| Task | Files | Dependencies | -|------|-------|--------------| -| **Extend PublisherService** | `domain/publishing/services/publisher_service.py` | Phase 5 | -| **Multi-destination Support** | `domain/publishing/services/publisher_service.py` | Phase 5 | -| **Update PublishingRecord** | `domain/publishing/models.py` | Phase 5 | - -### 6.5 Site Model Extensions - -| Task | Files | Dependencies | -|------|-------|--------------| -| **Add site_type field** | `core/auth/models.py` | None | -| **Add hosting_type field** | `core/auth/models.py` | None | -| **Add integrations relationship** | `core/auth/models.py` | Phase 6.1 | -| **Migration** | `core/auth/migrations/` | None | - -### 6.6 Integration API - -| Task | Files | Dependencies | -|------|-------|--------------| -| **Integration ViewSet** | `modules/integration/views.py` | IntegrationService | -| **Integration URLs** | `modules/integration/urls.py` | None | - -### 6.7 Integration UI - -| Task | Files | Dependencies | -|------|-------|--------------| -| **Integration Settings** | `frontend/src/pages/Settings/Integrations.tsx` | NEW | -| **Integration API Client** | `frontend/src/services/integration.api.ts` | NEW | - -### 6.8 WordPress Plugin Connection (API Only) - -| Task | Files | Dependencies | -|------|-------|--------------| -| **WordPress Sync Endpoints** | `modules/integration/views.py` | IntegrationService | -| **Two-way Sync Logic** | `domain/integration/services/sync_service.py` | Phase 6.2 | -| **WordPress Webhook Handler** | `modules/integration/views.py` | Phase 6.2 | - -**Note**: WordPress plugin itself is built separately. Only API endpoints for plugin connection are built here. - -### 6.9 Testing - -- ✅ Site integrations work correctly -- ✅ Multi-destination publishing works -- ✅ WordPress sync works (when plugin connected) -- ✅ Two-way sync functions properly - -**Timeline**: 2-3 weeks - ---- - -## PHASE 7: UNIVERSAL CONTENT TYPES - -**Goal**: Extend content system to support products, services, taxonomies. - -### 7.1 Content Model Extensions - -| Task | Files | Dependencies | -|------|-------|--------------| -| **Add entity_type field** | `domain/content/models.py` | Phase 1 | -| **Add json_blocks field** | `domain/content/models.py` | Phase 1 | -| **Add structure_data field** | `domain/content/models.py` | Phase 1 | -| **Content Migrations** | `domain/content/migrations/` | Phase 1 | - -### 7.2 Content Type Prompts - -| Task | Files | Dependencies | -|------|-------|--------------| -| **Product Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system | -| **Service Page Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system | -| **Taxonomy Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system | - -### 7.3 Content Service Extensions - -| Task | Files | Dependencies | -|------|-------|--------------| -| **Product Content Generation** | `domain/content/services/content_generation_service.py` | Phase 1 | -| **Service Page Generation** | `domain/content/services/content_generation_service.py` | Phase 1 | -| **Taxonomy Generation** | `domain/content/services/content_generation_service.py` | Phase 1 | - -### 7.4 Linker & Optimizer Extensions - -| Task | Files | Dependencies | -|------|-------|--------------| -| **Product Linking** | `domain/linking/services/linker_service.py` | Phase 4 | -| **Taxonomy Linking** | `domain/linking/services/linker_service.py` | Phase 4 | -| **Product Optimization** | `domain/optimization/services/optimizer_service.py` | Phase 4 | -| **Taxonomy Optimization** | `domain/optimization/services/optimizer_service.py` | Phase 4 | - -### 7.5 Testing - -- ✅ Product content generates correctly -- ✅ Service pages work -- ✅ Taxonomy pages work -- ✅ Linking works for all types -- ✅ Optimization works for all types - -**Timeline**: 2-3 weeks - ---- - -## TESTING & VALIDATION - -### Testing Strategy - -| Phase | Testing Focus | Validation | -|-------|--------------|------------| -| **Phase 0** | Credit system, existing features | All existing features work with credits | -| **Phase 1** | Service layer, API compatibility | APIs unchanged, services work | -| **Phase 2** | Automation execution | Rules execute, schedules work | -| **Phase 3** | Site Builder end-to-end | Sites build and deploy | -| **Phase 4** | Linker/Optimizer quality | Content improves, links work | -| **Phase 5** | Sites renderer | Sites render correctly | -| **Phase 6** | Multi-destination publishing | Content publishes to all destinations | -| **Phase 7** | Universal content types | All content types work | - -### Backward Compatibility Checklist - -- ✅ All existing API endpoints work -- ✅ All existing workflows function -- ✅ Frontend continues working -- ✅ Database migrations are safe -- ✅ No data loss during migration - -### Rollout Strategy - -1. **Feature Flags**: Enable new features gradually -2. **Beta Testing**: Test with select accounts first -3. **Gradual Rollout**: Enable for all users after validation -4. **Monitoring**: Track errors, performance, usage - ---- - -## SUMMARY - -### Implementation Timeline - -| Phase | Duration | Dependencies | -|-------|----------|--------------| -| **Phase 0: Credit System** | 1-2 weeks | None | -| **Phase 1: Service Layer** | 2-3 weeks | Phase 0 | -| **Phase 2: Automation** | 2-3 weeks | Phase 1 | -| **Phase 3: Site Builder** | 3-4 weeks | Phase 1, Phase 2 | -| **Phase 4: Linker/Optimizer** | 3-4 weeks | Phase 1 | -| **Phase 5: Sites Renderer** | 2-3 weeks | Phase 3 | -| **Phase 6: Site Integration** | 2-3 weeks | Phase 5 | -| **Phase 7: Universal Types** | 2-3 weeks | Phase 4 | - -**Total Estimated Time**: 17-25 weeks (4-6 months) - -### Key Success Criteria - -- ✅ All existing features continue working -- ✅ Credit system is universal and consistent -- ✅ Automation system is functional -- ✅ Site Builder creates and deploys sites -- ✅ Linker and Optimizer improve content -- ✅ Multi-destination publishing works -- ✅ All content types supported - ---- - -**END OF DOCUMENT** - diff --git a/README.md b/README.md index 9a342817..21653ed4 100644 --- a/README.md +++ b/README.md @@ -45,7 +45,15 @@ igny8/ │ ├── 04-BACKEND-IMPLEMENTATION.md │ ├── 05-AI-FRAMEWORK-IMPLEMENTATION.md │ ├── 06-FUNCTIONAL-BUSINESS-LOGIC.md -│ └── API-COMPLETE-REFERENCE.md # Complete unified API documentation +│ ├── API-COMPLETE-REFERENCE.md # Complete unified API documentation +│ ├── planning/ # Architecture & implementation planning documents +│ │ ├── IGNY8-HOLISTIC-ARCHITECTURE-PLAN.md # Complete architecture plan +│ │ ├── IGNY8-IMPLEMENTATION-PLAN.md # Step-by-step implementation plan +│ │ ├── Igny8-phase-2-plan.md # Phase 2 feature specifications +│ │ ├── CONTENT-WORKFLOW-DIAGRAM.md # Content workflow diagrams +│ │ ├── ARCHITECTURE_CONTEXT.md # Architecture context reference +│ │ └── sample-usage-limits-credit-system # Credit system specification +│ └── refactor/ # Refactoring plans and documentation ├── CHANGELOG.md # Version history and changes (only updated after user confirmation) └── docker-compose.app.yml ``` diff --git a/docs/00-DOCUMENTATION-MANAGEMENT.md b/docs/00-DOCUMENTATION-MANAGEMENT.md index ace6cf11..7fa86485 100644 --- a/docs/00-DOCUMENTATION-MANAGEMENT.md +++ b/docs/00-DOCUMENTATION-MANAGEMENT.md @@ -146,7 +146,21 @@ docs/ ├── 03-FRONTEND-ARCHITECTURE.md ├── 04-BACKEND-IMPLEMENTATION.md ├── 05-AI-FRAMEWORK-IMPLEMENTATION.md -└── 06-FUNCTIONAL-BUSINESS-LOGIC.md +├── 06-FUNCTIONAL-BUSINESS-LOGIC.md +├── API-COMPLETE-REFERENCE.md +├── WORDPRESS-PLUGIN-INTEGRATION.md +├── planning/ # Architecture & implementation planning +│ ├── IGNY8-HOLISTIC-ARCHITECTURE-PLAN.md +│ ├── IGNY8-IMPLEMENTATION-PLAN.md +│ ├── Igny8-phase-2-plan.md +│ ├── CONTENT-WORKFLOW-DIAGRAM.md +│ ├── ARCHITECTURE_CONTEXT.md +│ └── sample-usage-limits-credit-system +└── refactor/ # Refactoring plans and documentation + ├── README.md + ├── routes/ + ├── folder-structure/ + └── migrations/ ``` ### Documentation Update Checklist diff --git a/docs/planning/ARCHITECTURE_CONTEXT.md b/docs/planning/ARCHITECTURE_CONTEXT.md new file mode 100644 index 00000000..a8ac181d --- /dev/null +++ b/docs/planning/ARCHITECTURE_CONTEXT.md @@ -0,0 +1,924 @@ +# IGNY8 Complete Architecture Context +**Created:** 2025-01-XX +**Purpose:** Comprehensive context document for understanding the complete IGNY8 system architecture, workflows, and implementation details. + +--- + +## Executive Summary + +IGNY8 is a full-stack SaaS platform for SEO keyword management and AI-driven content generation. The system operates on a multi-tenant architecture with complete account isolation, hierarchical organization (Account > Site > Sector > Content), and unified AI processing framework. + +**Key Characteristics:** +- Multi-tenant SaaS with account isolation +- Django 5.2+ backend with DRF API +- React 19 frontend with TypeScript +- PostgreSQL 15 database +- Celery + Redis for async tasks +- Docker-based containerization +- Caddy reverse proxy for HTTPS + +--- + +## System Architecture Overview + +### High-Level Architecture + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Client Layer (Browser) │ +│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ +│ │ Main App │ │ Marketing │ │ Admin │ │ +│ │ (app.igny8) │ │ (igny8.com) │ │ Panel │ │ +│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │ +└─────────┼──────────────────┼──────────────────┼─────────────┘ + │ │ │ + └──────────────────┼──────────────────┘ + │ +┌────────────────────────────┼──────────────────────────────┐ +│ Reverse Proxy Layer │ +│ ┌───────────────┐ │ +│ │ Caddy │ │ +│ │ (HTTPS/443) │ │ +│ └───────┬───────┘ │ +└────────────────────────────┼──────────────────────────────┘ + │ +┌────────────────────────────┼──────────────────────────────┐ +│ Application Layer │ +│ ┌──────────────┐ ┌──────────────┐ │ +│ │ Frontend │ │ Backend │ │ +│ │ (React) │◄─────────────┤ (Django) │ │ +│ │ Port 8021 │ REST API │ Port 8011 │ │ +│ └──────────────┘ └──────┬───────┘ │ +│ │ │ +│ ┌────────┴────────┐ │ +│ │ Celery Worker │ │ +│ │ (Async Tasks) │ │ +│ └────────┬────────┘ │ +└───────────────────────────────────────┼──────────────────┘ + │ +┌───────────────────────────────────────┼──────────────────┐ +│ Data Layer │ +│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ +│ │ PostgreSQL │ │ Redis │ │ Storage │ │ +│ │ (Database) │ │ (Cache/Broker)│ │ (Files) │ │ +│ └──────────────┘ └──────────────┘ └──────────────┘ │ +└──────────────────────────────────────────────────────────┘ + │ +┌───────────────────────────────────────┼──────────────────┐ +│ External Services │ +│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ +│ │ OpenAI │ │ Runware │ │ WordPress │ │ +│ │ (GPT/DALL-E)│ │ (Images) │ │ (Publish) │ │ +│ └──────────────┘ └──────────────┘ └──────────────┘ │ +└──────────────────────────────────────────────────────────┘ +``` + +### Current Infrastructure Status + +**Running Containers:** +- `igny8_backend` - Django API (Port 8011, healthy) +- `igny8_frontend` - React app (Port 8021) +- `igny8_marketing_dev` - Marketing site (Port 8023) +- `igny8_celery_worker` - Async task processor +- `igny8_celery_beat` - Scheduled tasks +- `igny8_postgres` - Database (healthy) +- `igny8_redis` - Cache/Broker (healthy) +- `igny8_caddy` - Reverse proxy (Ports 80, 443) +- `igny8_pgadmin` - DB admin (Port 5050) +- `igny8_filebrowser` - File manager (Port 8080) +- `portainer` - Container management (Ports 8000, 9443) + +**Network:** `igny8_net` (bridge network, external) + +--- + +## Technology Stack + +### Backend Stack +- **Framework:** Django 5.2.7+ +- **API:** Django REST Framework +- **Database:** PostgreSQL 15 +- **Task Queue:** Celery 5.3.0+ with Redis 7 +- **Auth:** JWT (PyJWT 2.8.0+) +- **Server:** Gunicorn +- **Static Files:** WhiteNoise + +### Frontend Stack +- **Framework:** React 19.0.0 +- **Language:** TypeScript 5.7.2 +- **Build Tool:** Vite 6.1.0 +- **Styling:** Tailwind CSS 4.0.8 +- **State:** Zustand 5.0.8 +- **Routing:** React Router v7.9.5 +- **Icons:** @heroicons/react 2.2.0 + +### Infrastructure +- **Containerization:** Docker + Docker Compose +- **Reverse Proxy:** Caddy (HTTPS termination) +- **Container Management:** Portainer + +--- + +## Core Architecture Principles + +### 1. Multi-Tenancy Foundation +- **Account Isolation:** All models inherit `AccountBaseModel` with `account` ForeignKey +- **Automatic Filtering:** All ViewSets inherit `AccountModelViewSet` with automatic filtering +- **Middleware:** `AccountContextMiddleware` sets `request.account` from JWT token +- **Hierarchy:** Account > Site > Sector > Content + +### 2. Configuration-Driven Everything +- **Frontend:** Config files in `/config/pages/` and `/config/snippets/` +- **Backend:** DRF serializers and ViewSet actions +- **Templates:** 4 universal templates (Dashboard, Table, Form, System) + +### 3. Unified AI Framework +- **Single Interface:** All AI operations use `AIEngine` orchestrator +- **Base Class:** All AI functions inherit from `BaseAIFunction` +- **Execution Pipeline:** 6 phases (INIT, PREP, AI_CALL, PARSE, SAVE, DONE) +- **Progress Tracking:** Real-time updates via Celery + +### 4. Module-Based Organization +- **Planner:** Keywords, Clusters, Ideas +- **Writer:** Tasks, Content, Images +- **Thinker:** Prompts, Author Profiles, Strategies +- **System:** Settings, Integrations, AI Configuration +- **Billing:** Credits, Transactions, Usage +- **Auth:** Accounts, Users, Sites, Sectors + +--- + +## System Hierarchy + +### Entity Relationships + +``` +Account (1) ──< (N) User +Account (1) ──< (1) Subscription ──> (1) Plan +Account (1) ──< (N) Site +Site (1) ──< (1-5) Sector +Sector (1) ──< (N) Keywords, Clusters, ContentIdeas, Tasks +Cluster (1) ──< (N) Keywords (Many-to-Many) +Cluster (1) ──< (N) ContentIdeas +ContentIdeas (1) ──< (N) Tasks +Task (1) ──> (1) Content +Task (1) ──< (N) Images +``` + +### Hierarchy Details + +**Account Level:** +- Top-level organization/workspace +- Contains users, sites, subscriptions, and all data +- Has credit balance and plan assignment +- Status: active, suspended, trial, cancelled + +**User Level:** +- Individual user accounts within an account +- Has role (developer, owner, admin, editor, viewer) +- Can belong to only one account +- Access controlled by role and site permissions + +**Site Level:** +- Workspace within an account (1-N relationship) +- Can have multiple active sites simultaneously +- Has WordPress integration settings (URL, username, password) +- Can be associated with an industry +- Status: active, inactive, suspended + +**Sector Level:** +- Content category within a site (1-5 per site) +- Organizes keywords, clusters, ideas, and tasks +- Can reference an industry sector template +- Status: active, inactive + +**Content Level:** +- Keywords, Clusters, ContentIdeas belong to Sector +- Tasks, Content, Images belong to Sector +- All content is automatically associated with Account and Site + +--- + +## User Roles & Access Control + +### Role Hierarchy +``` +developer > owner > admin > editor > viewer > system_bot +``` + +### Role Permissions + +| Role | Account Access | Site Access | Data Access | User Management | Billing | +|------|----------------|-------------|-------------|-----------------|---------| +| Developer | All accounts | All sites | All data | Yes | Yes | +| System Bot | All accounts | All sites | All data | No | No | +| Owner | Own account | All sites in account | All data in account | Yes | Yes | +| Admin | Own account | All sites in account | All data in account | Yes | No | +| Editor | Own account | Granted sites only | Data in granted sites | No | No | +| Viewer | Own account | Granted sites only | Read-only in granted sites | No | No | + +### Access Control Implementation + +**Automatic Access:** +- Owners and Admins: Automatic access to all sites in their account +- Developers and System Bot: Access to all sites across all accounts + +**Explicit Access:** +- Editors and Viewers: Require explicit `SiteUserAccess` records +- Access granted by Owner or Admin +- Access can be revoked at any time + +--- + +## Complete Workflows + +### 1. Account Setup Workflow + +**Steps:** +1. User signs up via `/signup` +2. Account created with default plan +3. Owner user created and linked to account +4. User signs in via `/signin` +5. JWT token generated and returned +6. Frontend stores token and redirects to dashboard +7. User creates first site (optional) +8. User creates sectors (1-5 per site, optional) +9. User configures integration settings (OpenAI, Runware) +10. System ready for use + +**Data Created:** +- 1 Account record +- 1 User record (owner role) +- 1 Subscription record (default plan) +- 0-N Site records +- 0-N Sector records (per site) +- 1 IntegrationSettings record (per integration type) + +### 2. Keyword Management Workflow + +**Steps:** +1. User navigates to `/planner/keywords` +2. User imports keywords via CSV or manual entry +3. Keywords validated and stored in database +4. Keywords displayed in table with filters +5. User filters keywords by sector, status, intent, etc. +6. User selects keywords for clustering +7. User clicks "Auto Cluster" action +8. Backend validates keyword IDs +9. Celery task queued (`run_ai_task` with function `auto_cluster`) +10. Task ID returned to frontend +11. Frontend polls progress endpoint +12. Celery worker processes task: + - Loads keywords from database + - Builds AI prompt with keyword data + - Calls OpenAI API for clustering + - Parses cluster response + - Creates Cluster records + - Links keywords to clusters +13. Progress updates sent to frontend +14. Task completes +15. Frontend displays new clusters +16. Credits deducted from account + +**AI Function:** Auto Cluster Keywords + +### 3. Content Generation Workflow + +**Steps:** +1. User navigates to `/planner/ideas` +2. User selects content ideas +3. User clicks "Create Tasks" action +4. Task records created for each idea +5. User navigates to `/writer/tasks` +6. User selects tasks for content generation +7. User clicks "Generate Content" action +8. Backend validates task IDs +9. Celery task queued (`run_ai_task` with function `generate_content`) +10. Task ID returned to frontend +11. Frontend polls progress endpoint +12. Celery worker processes task: + - Loads tasks and related data (cluster, keywords, idea) + - Builds AI prompt with task data + - Calls OpenAI API for content generation + - Parses HTML content response + - Creates/updates Content records + - Updates task status +13. Progress updates sent to frontend +14. Task completes +15. Frontend displays generated content +16. Credits deducted from account + +**AI Function:** Generate Content + +### 4. WordPress Publishing Workflow + +**Steps:** +1. User navigates to `/writer/content` +2. User selects content to publish +3. User clicks "Publish to WordPress" action +4. Backend validates: + - Site has WordPress URL configured + - Site has WordPress credentials + - Content is ready (status: review or draft) +5. Backend calls WordPress REST API: + - Creates post with content HTML + - Uploads featured image (if available) + - Uploads in-article images (if available) + - Sets post status (draft, publish) +6. WordPress post ID stored in Content record +7. Content status updated to "published" +8. Frontend displays success message + +**Integration:** WordPress REST API + +--- + +## AI Framework Architecture + +### Unified Execution Pipeline + +**Entry Point:** `run_ai_task` (Celery task) +- Location: `backend/igny8_core/ai/tasks.py` +- Parameters: `function_name`, `payload`, `account_id` +- Flow: Loads function from registry → Creates AIEngine → Executes function + +**Engine Orchestrator:** `AIEngine` +- Location: `backend/igny8_core/ai/engine.py` +- Purpose: Central orchestrator managing lifecycle, progress, logging, cost tracking +- Methods: + - `execute` - Main execution pipeline (6 phases) + - `_handle_error` - Centralized error handling + - `_log_to_database` - Logs to AITaskLog model + +**Base Function Class:** `BaseAIFunction` +- Location: `backend/igny8_core/ai/base.py` +- Purpose: Abstract base class defining interface for all AI functions +- Abstract Methods: + - `get_name()` - Returns function name + - `prepare()` - Loads and prepares data + - `build_prompt()` - Builds AI prompt + - `parse_response()` - Parses AI response + - `save_output()` - Saves results to database + +### AI Function Execution Flow + +``` +1. API Endpoint (views.py) + ↓ +2. run_ai_task (tasks.py) + - Gets account from account_id + - Gets function instance from registry + - Creates AIEngine + ↓ +3. AIEngine.execute (engine.py) + Phase 1: INIT (0-10%) + - Calls function.validate() + - Updates progress tracker + ↓ + Phase 2: PREP (10-25%) + - Calls function.prepare() + - Calls function.build_prompt() + - Updates progress tracker + ↓ + Phase 3: AI_CALL (25-70%) + - Gets model config from settings + - Calls AICore.run_ai_request() or AICore.generate_image() + - Tracks cost and tokens + - Updates progress tracker + ↓ + Phase 4: PARSE (70-85%) + - Calls function.parse_response() + - Updates progress tracker + ↓ + Phase 5: SAVE (85-98%) + - Calls function.save_output() + - Logs credit usage + - Updates progress tracker + ↓ + Phase 6: DONE (98-100%) + - Logs to AITaskLog + - Returns result +``` + +### AI Functions + +1. **Auto Cluster Keywords** (`auto_cluster`) + - Purpose: Group related keywords into semantic clusters + - Input: Keyword IDs (max 20) + - Output: Cluster records created, keywords linked + - Credits: 1 credit per 30 keywords + +2. **Generate Ideas** (`generate_ideas`) + - Purpose: Generate content ideas from keyword clusters + - Input: Cluster IDs (max 1 per batch) + - Output: ContentIdeas records created + - Credits: 1 credit per idea + +3. **Generate Content** (`generate_content`) + - Purpose: Generate blog post and article content + - Input: Task IDs (max 50 per batch) + - Output: Content records created/updated with HTML + - Credits: 3 credits per content piece + +4. **Generate Image Prompts** (`generate_image_prompts`) + - Purpose: Extract image prompts from content HTML + - Input: Content IDs + - Output: Images records updated with prompts + - Credits: Included in content generation + +5. **Generate Images** (`generate_images`) + - Purpose: Generate images using OpenAI DALL-E or Runware + - Input: Image IDs (with prompts) + - Output: Images records updated with image URLs + - Credits: 1 credit per image + +--- + +## Frontend Architecture + +### Application Structure + +**Dual Application Architecture:** +1. **Main Application** (`app.igny8.com`): Authenticated SaaS platform +2. **Marketing Site** (`igny8.com`): Public-facing marketing website + +**Entry Points:** +- Main App: `src/main.tsx` → `src/App.tsx` +- Marketing: `src/marketing/index.tsx` → `src/marketing/MarketingApp.tsx` + +### State Management + +**Zustand Stores:** +- `authStore` - Authentication & user +- `siteStore` - Active site management +- `sectorStore` - Active sector management +- `plannerStore` - Planner module state +- `billingStore` - Billing & credits +- `settingsStore` - Application settings +- `pageSizeStore` - Table pagination +- `columnVisibilityStore` - Table column visibility + +**React Contexts:** +- `ThemeContext` - Light/dark theme +- `SidebarContext` - Sidebar state +- `HeaderMetricsContext` - Header metrics +- `ToastProvider` - Toast notifications + +### Template System + +**4 Universal Templates:** +1. **DashboardTemplate** - Module home pages (KPIs, workflow steps, charts) +2. **TablePageTemplate** - CRUD table pages (filtering, sorting, pagination) +3. **FormPageTemplate** - Settings/form pages (sectioned forms) +4. **SystemPageTemplate** - System/admin pages (status cards, logs) + +### API Integration + +**API Service Layer:** +- Location: `frontend/src/services/api.ts` +- Function: `fetchAPI()` - Centralized API client +- Features: + - Automatic token injection + - Token refresh on 401 + - Site/sector context injection + - Unified error handling + - Timeout handling + +**Request Flow:** +1. User action in frontend +2. Frontend makes API request via `fetchAPI()` +3. JWT token included in Authorization header +4. Backend middleware extracts account from JWT +5. Backend ViewSet processes request +6. Backend returns JSON response (unified format) +7. Frontend updates state +8. Frontend updates UI + +--- + +## Backend Architecture + +### Multi-Tenancy Implementation + +**Account Isolation:** +- **Model Level:** All models inherit `AccountBaseModel` with `account` ForeignKey +- **ViewSet Level:** All ViewSets inherit `AccountModelViewSet` with automatic filtering +- **Middleware Level:** `AccountContextMiddleware` sets `request.account` from JWT + +**Middleware Flow:** +``` +Request with JWT Token + ↓ +AccountContextMiddleware + ├── Extract Account ID from JWT + ├── Load Account Object + └── Set request.account + ↓ +ViewSet.get_queryset() + ├── Check User Role + ├── Filter by Account (if not admin/developer) + └── Filter by Accessible Sites (if not owner/admin) + ↓ +Database Query + ↓ +Results (Account-Isolated) +``` + +### Base Classes + +**AccountModelViewSet:** +- Location: `backend/igny8_core/api/base.py` +- Purpose: Base ViewSet with automatic account filtering +- Features: + - Automatic account filtering + - Admin/Developer override + - Account context in serializers + +**SiteSectorModelViewSet:** +- Location: `backend/igny8_core/api/base.py` +- Purpose: Base ViewSet with site/sector filtering +- Features: + - Account filtering (inherited) + - Site access control + - Sector validation + - Accessible sites/sectors in serializer context + +### API Response Format + +**Unified Format:** +```json +{ + "success": true, + "data": {...}, + "message": "Optional message", + "request_id": "uuid" +} +``` + +**Error Format:** +```json +{ + "success": false, + "error": "Error message", + "errors": { + "field_name": ["Field-specific errors"] + }, + "request_id": "uuid" +} +``` + +**Paginated Format:** +```json +{ + "success": true, + "count": 120, + "next": "url", + "previous": "url", + "results": [...], + "request_id": "uuid" +} +``` + +--- + +## Module Organization + +### Planner Module +- **Purpose:** Keyword management & content planning +- **Models:** Keywords, Clusters, ContentIdeas +- **ViewSets:** KeywordViewSet, ClusterViewSet, ContentIdeasViewSet +- **Celery Tasks:** `auto_cluster_keywords_task`, `auto_generate_ideas_task` +- **Features:** + - Keyword import (CSV/manual) + - Keyword filtering and organization + - AI-powered keyword clustering + - Content idea generation from clusters + - Keyword-to-cluster mapping + +### Writer Module +- **Purpose:** Content generation & management +- **Models:** Tasks, Content, Images +- **ViewSets:** TasksViewSet, ImagesViewSet +- **Celery Tasks:** `auto_generate_content_task`, `auto_generate_images_task` +- **Features:** + - Task creation from content ideas + - AI-powered content generation + - Content editing and review + - Image prompt extraction + - AI-powered image generation + - WordPress publishing + +### Thinker Module +- **Purpose:** AI configuration and strategy +- **Models:** AIPrompt, AuthorProfile, Strategy +- **ViewSets:** AIPromptViewSet, AuthorProfileViewSet +- **Features:** + - AI prompt management + - Author profile management + - Content strategy management + - Image testing + +### System Module +- **Purpose:** System configuration and AI settings +- **Models:** IntegrationSettings, AIPrompt, AuthorProfile, Strategy +- **ViewSets:** IntegrationSettingsViewSet, AIPromptViewSet, AuthorProfileViewSet +- **Features:** + - Integration settings (OpenAI, Runware) + - AI prompt management + - System status and monitoring + +### Billing Module +- **Purpose:** Credit management and usage tracking +- **Models:** CreditTransaction, CreditUsageLog +- **ViewSets:** CreditTransactionViewSet, CreditUsageLogViewSet +- **Services:** CreditService +- **Features:** + - Credit balance tracking + - Credit transactions + - Usage logging + - Cost tracking + +### Auth Module +- **Purpose:** Multi-tenancy and user management +- **Models:** Account, User, Plan, Site, Sector, Industry +- **ViewSets:** AccountViewSet, UserViewSet, SiteViewSet, SectorViewSet +- **Features:** + - Account management + - User management + - Plan management + - Site and sector management + - Industry templates + +--- + +## Credit System + +### Credit Balance Management + +**Account Credits:** +- Each account has a `credits` field (integer) +- Credits start at 0 or plan-included credits +- Credits are deducted for AI operations +- Credits can be added via transactions + +**Credit Checking:** +- Before AI operation: System checks if account has sufficient credits +- If insufficient: Operation fails with `InsufficientCreditsError` +- If sufficient: Operation proceeds + +**Credit Deduction:** +- After AI operation completes: Credits deducted via `CreditService.deduct_credits()` +- Account credits field updated +- CreditTransaction record created (type: deduction, amount: negative) +- CreditUsageLog record created with operation details + +### Credit Costs per Operation + +- **Clustering:** 1 credit per 30 keywords (base: 1 credit) +- **Ideas:** 1 credit per idea (base: 1 credit) +- **Content:** 3 credits per content piece (base: 3 credits) +- **Images:** 1 credit per image (base: 1 credit) +- **Reparse:** 1 credit per reparse (base: 1 credit) + +--- + +## WordPress Integration + +### Publishing Process + +**Workflow:** +1. User selects content to publish +2. System validates WordPress configuration +3. System authenticates with WordPress REST API +4. System creates WordPress post: + - Title: Content meta_title or task title + - Content: Content HTML + - Status: Draft or Publish (based on content status) + - Featured image: Uploaded if available + - In-article images: Uploaded if available + - Meta fields: Primary keyword, secondary keywords +5. WordPress returns post ID +6. System updates Content record: + - Sets `wp_post_id` field + - Sets `status` to "published" + +**Requirements:** +- Site must have WordPress URL configured (`wp_url`) +- Site must have WordPress username and app password +- Content must have status "review" or "draft" +- WordPress REST API must be accessible + +--- + +## Docker Architecture + +### Infrastructure Stack (`igny8-infra`) +- **PostgreSQL** - Database (Port 5432 internal) +- **Redis** - Cache & Celery broker (Port 6379 internal) +- **pgAdmin** - Database admin (Port 5050) +- **FileBrowser** - File management (Port 8080) +- **Caddy** - Reverse proxy (Ports 80, 443) +- **Setup Helper** - Utility container + +### Application Stack (`igny8-app`) +- **Backend** - Django API (Port 8011:8010) +- **Frontend** - React app (Port 8021:5173) +- **Marketing Dev** - Marketing site (Port 8023:5174) +- **Celery Worker** - Async task processing +- **Celery Beat** - Scheduled tasks + +### Network Configuration +- **Network Name:** `igny8_net` +- **Type:** External bridge network +- **Purpose:** Inter-container communication + +--- + +## Key Files and Locations + +### Backend Key Files +- `backend/igny8_core/auth/middleware.py` - AccountContextMiddleware +- `backend/igny8_core/api/base.py` - AccountModelViewSet, SiteSectorModelViewSet +- `backend/igny8_core/ai/engine.py` - AIEngine orchestrator +- `backend/igny8_core/ai/base.py` - BaseAIFunction +- `backend/igny8_core/ai/tasks.py` - run_ai_task entrypoint +- `backend/igny8_core/api/response.py` - Unified response helpers + +### Frontend Key Files +- `frontend/src/services/api.ts` - API client +- `frontend/src/store/authStore.ts` - Authentication state +- `frontend/src/store/siteStore.ts` - Site management +- `frontend/src/templates/` - 4 universal templates +- `frontend/src/config/pages/` - Page configurations + +### Documentation +- `docs/01-TECH-STACK-AND-INFRASTRUCTURE.md` - Tech stack +- `docs/02-APPLICATION-ARCHITECTURE.md` - Application architecture +- `docs/03-FRONTEND-ARCHITECTURE.md` - Frontend architecture +- `docs/04-BACKEND-IMPLEMENTATION.md` - Backend implementation +- `docs/05-AI-FRAMEWORK-IMPLEMENTATION.md` - AI framework +- `docs/06-FUNCTIONAL-BUSINESS-LOGIC.md` - Business logic +- `docs/API-COMPLETE-REFERENCE.md` - Complete API reference + +--- + +## Data Flow Examples + +### Request Flow +``` +1. User Action (e.g., "Auto Cluster Keywords") + ↓ +2. Frontend API Call (fetchAPI) + ↓ +3. Backend Endpoint (ViewSet Action) + ↓ +4. Celery Task Queued + ↓ +5. Task ID Returned to Frontend + ↓ +6. Frontend Polls Progress Endpoint + ↓ +7. Celery Worker Processes Task + ↓ +8. AIProcessor Makes API Calls + ↓ +9. Results Saved to Database + ↓ +10. Progress Updates Sent + ↓ +11. Frontend Displays Results +``` + +### Authentication Flow +``` +1. User Signs In + ↓ +2. Backend Validates Credentials + ↓ +3. JWT Token Generated + ↓ +4. Token Returned to Frontend + ↓ +5. Frontend Stores Token (localStorage) + ↓ +6. Frontend Includes Token in Requests (Authorization: Bearer {token}) + ↓ +7. Backend Validates Token + ↓ +8. Account Context Set (AccountContextMiddleware) + ↓ +9. Request Processed +``` + +--- + +## Security Architecture + +### Authentication +- **Primary:** JWT Bearer tokens +- **Fallback:** Session-based auth (admin panel) +- **Token Storage:** localStorage (frontend) +- **Token Expiry:** 15 minutes (access), 7 days (refresh) + +### Authorization +- **Role-Based Access Control (RBAC):** Role checked on every request +- **Data Access Control:** + - Account-level: Automatic filtering by account + - Site-level: Filtering by accessible sites + - Action-level: Permission checks in ViewSet actions + +### Account Isolation +- All queries filtered by account +- Admin/Developer override for system accounts +- No cross-account data leakage + +--- + +## External Service Integrations + +### OpenAI Integration +- **Purpose:** Text generation and image generation +- **Configuration:** API key stored per account in `IntegrationSettings` +- **Services Used:** + - GPT models for text generation + - DALL-E for image generation +- **Cost Tracking:** Tracked per request + +### Runware Integration +- **Purpose:** Alternative image generation service +- **Configuration:** API key stored per account +- **Model Selection:** e.g., `runware:97@1` +- **Image Type:** realistic, artistic, cartoon + +### WordPress Integration +- **Purpose:** Content publishing +- **Configuration:** WordPress URL per site, username and password stored per site +- **Workflow:** + 1. Content generated in IGNY8 + 2. Images attached + 3. Content published to WordPress via REST API + 4. Status updated in IGNY8 + +--- + +## Development Workflow + +### Local Development +1. **Backend:** + ```bash + cd backend + pip install -r requirements.txt + python manage.py migrate + python manage.py runserver + ``` + +2. **Frontend:** + ```bash + cd frontend + npm install + npm run dev + ``` + +### Docker Development +1. **Build Images:** + ```bash + docker build -t igny8-backend -f backend/Dockerfile ./backend + docker build -t igny8-frontend-dev -f frontend/Dockerfile.dev ./frontend + ``` + +2. **Start Services:** + ```bash + docker compose -f docker-compose.app.yml -p igny8-app up -d + ``` + +--- + +## Summary + +This context document provides a comprehensive overview of the IGNY8 system architecture, including: + +1. **System Architecture:** High-level architecture, infrastructure status, technology stack +2. **Core Principles:** Multi-tenancy, configuration-driven, unified AI framework, module-based +3. **System Hierarchy:** Entity relationships, account/site/sector structure +4. **User Roles:** Role hierarchy, permissions, access control +5. **Workflows:** Complete workflows for account setup, keyword management, content generation, WordPress publishing +6. **AI Framework:** Unified execution pipeline, AI functions, progress tracking +7. **Frontend Architecture:** Dual application structure, state management, templates, API integration +8. **Backend Architecture:** Multi-tenancy implementation, base classes, API response format +9. **Module Organization:** Planner, Writer, Thinker, System, Billing, Auth modules +10. **Credit System:** Credit balance management, costs per operation +11. **WordPress Integration:** Publishing process, requirements +12. **Docker Architecture:** Infrastructure and application stacks +13. **Key Files:** Important file locations +14. **Data Flow:** Request and authentication flows +15. **Security:** Authentication, authorization, account isolation +16. **External Services:** OpenAI, Runware, WordPress integrations +17. **Development:** Local and Docker development workflows + +This document serves as a comprehensive reference for understanding the complete IGNY8 system architecture and implementation details. + +--- + +**Last Updated:** 2025-01-XX +**Version:** 1.0.0 + diff --git a/docs/planning/CONTENT-WORKFLOW-DIAGRAM.md b/docs/planning/CONTENT-WORKFLOW-DIAGRAM.md new file mode 100644 index 00000000..29876516 --- /dev/null +++ b/docs/planning/CONTENT-WORKFLOW-DIAGRAM.md @@ -0,0 +1,309 @@ +# CONTENT WORKFLOW & ENTRY POINTS +**Complete Workflow Diagrams for Writer → Linker → Optimizer** + +--- + +## WORKFLOW 1: WRITER → LINKER → OPTIMIZER → PUBLISH + +``` +┌─────────────┐ +│ Writer │ +│ Generates │ +│ Content │ +└──────┬──────┘ + │ + ▼ +┌─────────────────────┐ +│ Content Saved │ +│ source='igny8' │ +│ sync_status='native'│ +│ status='draft' │ +└──────┬──────────────┘ + │ + ▼ +┌─────────────────────┐ +│ Linker Trigger │ +│ (Auto or Manual) │ +└──────┬──────────────┘ + │ + ▼ +┌─────────────────────┐ +│ LinkerService │ +│ - Finds candidates │ +│ - Injects links │ +│ - Updates content │ +└──────┬──────────────┘ + │ + ▼ +┌─────────────────────┐ +│ Content Updated │ +│ linker_version++ │ +│ internal_links[] │ +│ status='linked' │ +└──────┬──────────────┘ + │ + ▼ +┌─────────────────────┐ +│ Optimizer Trigger │ +│ (Auto or Manual) │ +└──────┬──────────────┘ + │ + ▼ +┌─────────────────────┐ +│ OptimizerService │ +│ - Analyzes content │ +│ - Optimizes │ +│ - Stores results │ +└──────┬──────────────┘ + │ + ▼ +┌─────────────────────┐ +│ Content Updated │ +│ optimizer_version++ │ +│ status='optimized' │ +└──────┬──────────────┘ + │ + ▼ +┌─────────────────────┐ +│ PublisherService │ +│ - WordPress │ +│ - Sites Renderer │ +│ - Shopify │ +└─────────────────────┘ +``` + +--- + +## WORKFLOW 2: WORDPRESS SYNC → OPTIMIZER → PUBLISH + +``` +┌─────────────────┐ +│ WordPress │ +│ Plugin Syncs │ +│ Posts to IGNY8 │ +└────────┬────────┘ + │ + ▼ +┌─────────────────────────┐ +│ ContentSyncService │ +│ sync_from_wordpress() │ +└────────┬────────────────┘ + │ + ▼ +┌─────────────────────────┐ +│ Content Created │ +│ source='wordpress' │ +│ sync_status='synced' │ +│ external_id=wp_post_id │ +│ external_url=wp_url │ +└────────┬────────────────┘ + │ + ▼ +┌─────────────────────────┐ +│ Content Visible │ +│ in Writer/Content List │ +│ (Filterable by source) │ +└────────┬────────────────┘ + │ + ▼ +┌─────────────────────────┐ +│ User Selects Content │ +│ Clicks "Optimize" │ +└────────┬────────────────┘ + │ + ▼ +┌─────────────────────────┐ +│ OptimizerService │ +│ optimize_from_wordpress_│ +│ sync(content_id) │ +└────────┬────────────────┘ + │ + ▼ +┌─────────────────────────┐ +│ Optimizer Processes │ +│ (Same logic as IGNY8) │ +│ - Analyzes │ +│ - Optimizes │ +│ - Stores results │ +└────────┬────────────────┘ + │ + ▼ +┌─────────────────────────┐ +│ OptimizationTask │ +│ Created │ +│ Original preserved │ +└────────┬────────────────┘ + │ + ▼ +┌─────────────────────────┐ +│ Optional: Sync Back │ +│ to WordPress │ +│ (Two-way sync) │ +└─────────────────────────┘ +``` + +--- + +## WORKFLOW 3: 3RD PARTY SYNC → OPTIMIZER → PUBLISH + +``` +┌─────────────────┐ +│ Shopify/API │ +│ Syncs Content │ +│ to IGNY8 │ +└────────┬────────┘ + │ + ▼ +┌─────────────────────────┐ +│ ContentSyncService │ +│ sync_from_shopify() │ +│ or sync_from_custom() │ +└────────┬────────────────┘ + │ + ▼ +┌─────────────────────────┐ +│ Content Created │ +│ source='shopify'/'custom'│ +│ sync_status='imported' │ +│ external_id=external_id │ +│ external_url=external_url│ +└────────┬────────────────┘ + │ + ▼ +┌─────────────────────────┐ +│ Content Visible │ +│ in Writer/Content List │ +│ (Filterable by source) │ +└────────┬────────────────┘ + │ + ▼ +┌─────────────────────────┐ +│ User Selects Content │ +│ Clicks "Optimize" │ +└────────┬────────────────┘ + │ + ▼ +┌─────────────────────────┐ +│ OptimizerService │ +│ optimize_from_external_ │ +│ sync(content_id) │ +└────────┬────────────────┘ + │ + ▼ +┌─────────────────────────┐ +│ Optimizer Processes │ +│ (Same logic) │ +└────────┬────────────────┘ + │ + ▼ +┌─────────────────────────┐ +│ OptimizationTask │ +│ Created │ +└─────────────────────────┘ +``` + +--- + +## WORKFLOW 4: MANUAL SELECTION → LINKER/OPTIMIZER + +``` +┌─────────────────────────┐ +│ User Views Content List │ +│ (Any source) │ +└────────┬────────────────┘ + │ + ▼ +┌─────────────────────────┐ +│ User Selects Content │ +│ (Can filter by source) │ +└────────┬────────────────┘ + │ + ▼ +┌─────────────────────────┐ +│ User Clicks Action: │ +│ - "Add Links" │ +│ - "Optimize" │ +│ - "Link & Optimize" │ +└────────┬────────────────┘ + │ + ▼ +┌─────────────────────────┐ +│ LinkerService or │ +│ OptimizerService │ +│ (Works for any source) │ +└────────┬────────────────┘ + │ + ▼ +┌─────────────────────────┐ +│ Content Processed │ +│ Results Stored │ +└─────────────────────────┘ +``` + +--- + +## CONTENT STORAGE STRATEGY + +### Unified Content Model + +All content stored in same `Content` model, differentiated by flags: + +| Field | Values | Purpose | +|-------|--------|---------| +| `source` | `'igny8'`, `'wordpress'`, `'shopify'`, `'custom'` | Where content came from | +| `sync_status` | `'native'`, `'imported'`, `'synced'` | How content was added | +| `external_id` | String | External platform ID | +| `external_url` | URL | External platform URL | +| `sync_metadata` | JSON | Platform-specific data | + +### Content Filtering + +**Frontend Filters**: +- By source: Show only IGNY8, WordPress, Shopify, or All +- By sync_status: Show Native, Imported, Synced, or All +- By optimization status: Not optimized, Optimized, Needs optimization +- By linking status: Not linked, Linked, Needs linking + +**Backend Queries**: +```python +# Get all IGNY8 content +Content.objects.filter(source='igny8', sync_status='native') + +# Get all WordPress synced content +Content.objects.filter(source='wordpress', sync_status='synced') + +# Get all content ready for optimization +Content.objects.filter(optimizer_version=0) + +# Get all content ready for linking +Content.objects.filter(linker_version=0) +``` + +--- + +## ENTRY POINT SUMMARY + +| Entry Point | Trigger | Content Source | Goes Through | +|-------------|---------|----------------|--------------| +| **Writer → Linker** | Auto or Manual | `source='igny8'` | Linker → Optimizer | +| **Writer → Optimizer** | Auto or Manual | `source='igny8'` | Optimizer (skip linker) | +| **WordPress Sync → Optimizer** | Manual or Auto | `source='wordpress'` | Optimizer only | +| **3rd Party Sync → Optimizer** | Manual or Auto | `source='shopify'/'custom'` | Optimizer only | +| **Manual Selection → Linker** | Manual | Any source | Linker only | +| **Manual Selection → Optimizer** | Manual | Any source | Optimizer only | + +--- + +## KEY PRINCIPLES + +1. **Unified Storage**: All content in same model, filtered by flags +2. **Source Agnostic**: Linker/Optimizer work on any content source +3. **Flexible Entry**: Multiple ways to enter pipeline +4. **Preserve Original**: Original content always preserved +5. **Version Tracking**: `linker_version` and `optimizer_version` track processing +6. **Filterable**: Content can be filtered by source, sync_status, processing status + +--- + +**END OF DOCUMENT** + diff --git a/IGNY8-HOLISTIC-ARCHITECTURE-PLAN.md b/docs/planning/IGNY8-HOLISTIC-ARCHITECTURE-PLAN.md similarity index 71% rename from IGNY8-HOLISTIC-ARCHITECTURE-PLAN.md rename to docs/planning/IGNY8-HOLISTIC-ARCHITECTURE-PLAN.md index daefdabf..76393a11 100644 --- a/IGNY8-HOLISTIC-ARCHITECTURE-PLAN.md +++ b/docs/planning/IGNY8-HOLISTIC-ARCHITECTURE-PLAN.md @@ -27,9 +27,9 @@ | Principle | Description | Implementation | |-----------|-------------|----------------| -| **Domain-Driven Design** | Organize by business domains, not technical layers | `domain/` folder with content, planning, linking, optimization, publishing domains | -| **Service Layer Pattern** | Business logic in services, not ViewSets | All modules delegate to domain services | -| **Single Responsibility** | Each layer has one clear purpose | Core → Domain → Module → Infrastructure | +| **Domain-Driven Design** | Organize by business domains, not technical layers | `business-modules/` folder with content, planning, linking, optimization, publishing business modules | +| **Service Layer Pattern** | Business logic in services, not ViewSets | All modules delegate to business module services | +| **Single Responsibility** | Each layer has one clear purpose | Core → Business Modules → Module → Infrastructure | | **No Duplication** | Reuse services across modules | ContentGenerationService used by Writer + Site Builder | | **Multi-Tenancy First** | All features respect account/site/sector boundaries | All models inherit AccountBaseModel or SiteSectorBaseModel | | **Automation Ready** | All functions can be automated via Celery | All AI functions exposed as Celery tasks with scheduling support | @@ -38,7 +38,7 @@ | System | Purpose | Technology | Container | |--------|---------|------------|-----------| -| **IGNY8 Core App** | Main backend API (all domains) | Django 5.2+ | `igny8_backend` | +| **IGNY8 Core App** | Main backend API (all business modules) | Django 5.2+ | `igny8_backend` | | **IGNY8 Main App** | Dashboard UI | React 19 + Vite | `igny8_frontend` | | **IGNY8 Site Builder** | Site creation wizard | React 19 + Vite | `igny8_site_builder` (NEW) | | **IGNY8 Sites** | Public site renderer | React 19 + Vite/Next.js | `igny8_sites` (NEW) | @@ -108,8 +108,8 @@ backend/igny8_core/ │ ├── permissions.py # IsAuthenticatedAndActive, HasTenantAccess │ └── throttles.py # DebugScopedRateThrottle │ -├── domain/ # DOMAIN LAYER (Business Logic) -│ ├── content/ # Content domain +├── business-modules/ # BUSINESS MODULES LAYER (Business Logic) +│ ├── content/ # Content business module │ │ ├── models.py # Content, Tasks, Images (unified, extended) │ │ ├── services/ │ │ │ ├── content_generation_service.py # Unified generation @@ -117,23 +117,23 @@ backend/igny8_core/ │ │ │ └── content_versioning_service.py # Version management │ │ └── repositories.py # ContentRepository (if needed) │ │ -│ ├── planning/ # Planning domain +│ ├── planning/ # Planning business module │ │ ├── models.py # Keywords, Clusters, Ideas │ │ └── services/ │ │ ├── clustering_service.py │ │ └── ideas_service.py │ │ -│ ├── linking/ # Linking domain (NEW) +│ ├── linking/ # Linking business module (NEW) │ │ ├── models.py # InternalLinks, LinkGraph │ │ └── services/ │ │ └── linker_service.py │ │ -│ ├── optimization/ # Optimization domain (NEW) +│ ├── optimization/ # Optimization business module (NEW) │ │ ├── models.py # OptimizationTask, OptimizationScores │ │ └── services/ │ │ └── optimizer_service.py │ │ -│ ├── publishing/ # Publishing domain (NEW) +│ ├── publishing/ # Publishing business module (NEW) │ │ ├── models.py # PublishingRecord, DeploymentRecord │ │ └── services/ │ │ ├── publisher_service.py @@ -143,13 +143,13 @@ backend/igny8_core/ │ │ │ └── sites_renderer_adapter.py │ │ └── deployment_service.py │ │ -│ ├── site_building/ # Site Building domain (NEW) +│ ├── site_building/ # Site Building business module (NEW) │ │ ├── models.py # SiteBlueprint, PageBlueprint │ │ └── services/ │ │ ├── structure_generation_service.py │ │ └── site_deployment_service.py │ │ -│ └── billing/ # Billing domain +│ └── billing/ # Billing business module │ ├── models.py # Credits, Transactions │ └── services/ │ └── credit_service.py @@ -248,20 +248,20 @@ backend/igny8_core/ | Model | Current Location | New Location | Extensions Needed | |-------|------------------|-------------|-------------------| -| `Content` | `modules/writer/models.py` | `domain/content/models.py` | Add: `entity_type`, `json_blocks`, `structure_data`, `linker_version`, `optimizer_version`, `internal_links`, `optimization_scores`, `published_to`, `external_ids` | -| `Tasks` | `modules/writer/models.py` | `domain/content/models.py` | Add: `entity_type` choices (product, service, taxonomy, etc.) | -| `Keywords` | `modules/planner/models.py` | `domain/planning/models.py` | No changes | -| `Clusters` | `modules/planner/models.py` | `domain/planning/models.py` | No changes | -| `ContentIdeas` | `modules/planner/models.py` | `domain/planning/models.py` | Add: `entity_type` support | -| `InternalLinks` | - | `domain/linking/models.py` | NEW: `source_id`, `target_id`, `anchor`, `position`, `link_type` | -| `OptimizationTask` | - | `domain/optimization/models.py` | NEW: `content_id`, `type`, `target_keyword`, `scores_before`, `scores_after`, `html_before`, `html_after` | -| `SiteBlueprint` | - | `domain/site_building/models.py` | NEW: `tenant`, `site`, `config_json`, `structure_json`, `status`, `hosting_type` | -| `PageBlueprint` | - | `domain/site_building/models.py` | NEW: `site_blueprint`, `slug`, `type`, `blocks_json`, `status` | -| `SiteIntegration` | - | `domain/integration/models.py` | NEW: `site`, `platform`, `platform_type`, `config_json`, `credentials`, `is_active`, `sync_enabled` | -| `PublishingRecord` | - | `domain/publishing/models.py` | NEW: `content_id`, `destination`, `destination_type`, `status`, `external_id`, `published_at`, `sync_status` | -| `DeploymentRecord` | - | `domain/publishing/models.py` | NEW: `site_blueprint`, `version`, `status`, `build_url`, `deployed_at`, `deployment_type` | -| `AutomationRule` | - | `domain/automation/models.py` | NEW: `name`, `trigger`, `conditions`, `actions`, `schedule`, `is_active` | -| `ScheduledTask` | - | `domain/automation/models.py` | NEW: `automation_rule`, `scheduled_at`, `status`, `executed_at` | +| `Content` | `modules/writer/models.py` | `business-modules/content/models.py` | Add: `entity_type`, `json_blocks`, `structure_data`, `linker_version`, `optimizer_version`, `internal_links`, `optimization_scores`, `published_to`, `external_ids`, `source`, `sync_status`, `external_id`, `external_url`, `sync_metadata` | +| `Tasks` | `modules/writer/models.py` | `business-modules/content/models.py` | Add: `entity_type` choices (product, service, taxonomy, etc.) | +| `Keywords` | `modules/planner/models.py` | `business-modules/planning/models.py` | No changes | +| `Clusters` | `modules/planner/models.py` | `business-modules/planning/models.py` | No changes | +| `ContentIdeas` | `modules/planner/models.py` | `business-modules/planning/models.py` | Add: `entity_type` support | +| `InternalLinks` | - | `business-modules/linking/models.py` | NEW: `source_id`, `target_id`, `anchor`, `position`, `link_type` | +| `OptimizationTask` | - | `business-modules/optimization/models.py` | NEW: `content_id`, `type`, `target_keyword`, `scores_before`, `scores_after`, `html_before`, `html_after` | +| `SiteBlueprint` | - | `business-modules/site_building/models.py` | NEW: `tenant`, `site`, `config_json`, `structure_json`, `status`, `hosting_type` | +| `PageBlueprint` | - | `business-modules/site_building/models.py` | NEW: `site_blueprint`, `slug`, `type`, `blocks_json`, `status` | +| `SiteIntegration` | - | `business-modules/integration/models.py` | NEW: `site`, `platform`, `platform_type`, `config_json`, `credentials`, `is_active`, `sync_enabled` | +| `PublishingRecord` | - | `business-modules/publishing/models.py` | NEW: `content_id`, `destination`, `destination_type`, `status`, `external_id`, `published_at`, `sync_status` | +| `DeploymentRecord` | - | `business-modules/publishing/models.py` | NEW: `site_blueprint`, `version`, `status`, `build_url`, `deployed_at`, `deployment_type` | +| `AutomationRule` | - | `business-modules/automation/models.py` | NEW: `name`, `trigger`, `conditions`, `actions`, `schedule`, `is_active` | +| `ScheduledTask` | - | `business-modules/automation/models.py` | NEW: `automation_rule`, `scheduled_at`, `status`, `executed_at` | --- @@ -367,8 +367,8 @@ sites/src/ | Component | Purpose | Implementation | |-----------|---------|----------------| -| **AutomationRule Model** | Store automation rules | `domain/automation/models.py` | -| **AutomationService** | Execute automation rules | `domain/automation/services/automation_service.py` | +| **AutomationRule Model** | Store automation rules | `business-modules/automation/models.py` | +| **AutomationService** | Execute automation rules | `business-modules/automation/services/automation_service.py` | | **Celery Beat Tasks** | Scheduled automation | `infrastructure/messaging/automation_tasks.py` | | **Automation API** | CRUD for rules | `modules/automation/views.py` | | **Automation UI** | Manage rules | `frontend/src/pages/Automation/` | @@ -410,8 +410,8 @@ class AutomationRule(SiteSectorBaseModel): | Task | File | Implementation | |------|------|----------------| -| **AutomationRule Model** | `domain/automation/models.py` | Create model with trigger, conditions, actions, schedule | -| **AutomationService** | `domain/automation/services/automation_service.py` | `execute_rule()`, `check_conditions()`, `execute_actions()` | +| **AutomationRule Model** | `business-modules/automation/models.py` | Create model with trigger, conditions, actions, schedule | +| **AutomationService** | `business-modules/automation/services/automation_service.py` | `execute_rule()`, `check_conditions()`, `execute_actions()` | | **Celery Beat Tasks** | `infrastructure/messaging/automation_tasks.py` | `@periodic_task` decorators for scheduled rules | | **Automation API** | `modules/automation/views.py` | CRUD ViewSet for AutomationRule | | **Automation UI** | `frontend/src/pages/Automation/` | Dashboard, Rules management, History | @@ -435,9 +435,9 @@ class AutomationRule(SiteSectorBaseModel): | Resource | Limit Type | Enforcement | Location | |-----------|------------|-------------|----------| -| **Daily Content Tasks** | Per site | `Plan.daily_content_tasks` | `domain/content/services/content_generation_service.py` | +| **Daily Content Tasks** | Per site | `Plan.daily_content_tasks` | `business-modules/content/services/content_generation_service.py` | | **Daily AI Requests** | Per site | `Plan.daily_ai_requests` | `infrastructure/ai/engine.py` | -| **Monthly Word Count** | Per site | `Plan.monthly_word_count_limit` | `domain/content/services/content_generation_service.py` | +| **Monthly Word Count** | Per site | `Plan.monthly_word_count_limit` | `business-modules/content/services/content_generation_service.py` | | **Daily Image Generation** | Per site | `Plan.daily_image_generation_limit` | `infrastructure/ai/functions/generate_images.py` | | **Storage Quota** | Per site | Configurable (default: 10GB) | `infrastructure/storage/file_storage.py` | | **Concurrent Tasks** | Per site | Configurable (default: 5) | Celery queue configuration | @@ -520,35 +520,203 @@ class FileStorageService: return (current_size + file_size) <= quota ``` +### 7.4 Sites Folder Access for Site Builder + +**User-Level File Management**: +- Each user has access to their account's sites folder: `/data/app/sites-data/clients/{site_id}/` +- Site Builder users can manage files for sites they have access to (via `SiteUserAccess`) +- File operations (upload, delete, organize) scoped to user's accessible sites +- Storage quota enforced per site (from Plan) +- File permissions: Owner/Admin (full access), Editor (upload/edit), Viewer (read-only) + +**Site Builder File Management**: +```python +# business-modules/site_building/services/file_management_service.py +class SiteBuilderFileService: + def get_user_accessible_sites(self, user) -> List[Site]: + """Get sites user can access for file management""" + return user.get_accessible_sites() + + def get_site_files_path(self, site_id: int) -> str: + """Get site's files directory""" + return f"/data/app/sites-data/clients/{site_id}/v{version}/assets/" + + def check_file_access(self, user, site_id: int) -> bool: + """Check if user can access site's files""" + accessible_sites = self.get_user_accessible_sites(user) + return any(site.id == site_id for site in accessible_sites) + + def upload_file(self, user, site_id: int, file) -> str: + """Upload file to site's assets folder""" + if not self.check_file_access(user, site_id): + raise PermissionDenied("No access to this site") + # Upload logic +``` + --- ## 8. REPOSITORY STRUCTURE -### 8.1 Main Repository Structure +### 8.1 Repository Structure + +#### 8.1.1 IGNY8 Repository (Main Development Repository) + +**Purpose**: Complete codebase including backend, frontend, and architecture documentation ``` -igny8/ # Main Git repository -├── backend/ # Django backend +igny8/ # Main Git repository (Complete Codebase) +├── backend/ # Django backend (ACTUAL CODE, not architecture) +│ └── igny8_core/ # Django project with all modules ├── frontend/ # React frontend (main app + marketing) -├── site-builder/ # Site Builder (NEW - separate or subfolder) -├── sites/ # Sites renderer (NEW - separate or subfolder) -├── docs/ # Documentation +├── site-builder/ # Site Builder frontend (NEW) +├── sites/ # Sites renderer frontend (NEW) +├── docs/ # Complete documentation +│ ├── planning/ # Architecture & implementation plans (ARCHITECTURE FILES) +│ ├── refactor/ # Refactoring plans (ARCHITECTURE FILES) +│ └── [core docs] # Technical documentation ├── docker-compose.app.yml # App stack ├── .gitignore # Excludes: node_modules, __pycache__, .env, logs, storage, sites-data └── README.md + +NOTE: +- backend/ contains ACTUAL IMPLEMENTATION CODE (not architecture) +- docs/planning/ contains ARCHITECTURE FILES (specifications, plans) +- Backend code can be developed here OR in separate repo for VPS deployment ``` +#### 8.1.2 Separate Backend Repository (Optional - VPS Deployment) + +**Purpose**: Backend code for VPS deployment (SSH-accessible files only) + +**When to Use:** +- Backend needs to be deployed separately to VPS +- Backend code is only accessible via SSH (not in igny8 folder) +- Separate deployment pipeline required + +**Structure:** +``` +igny8-backend/ # Separate Git repository (Backend Only) +├── igny8_core/ # Django project +│ ├── core/ # Core layer +│ ├── business-modules/ # Business modules layer +│ ├── infrastructure/ # Infrastructure layer +│ ├── modules/ # Module ViewSets (API layer) +│ ├── shared/ # Shared utilities +│ └── api/ # API base classes +├── requirements.txt +├── Dockerfile +├── docker-compose.backend.yml # Backend stack +├── .gitignore +└── README.md + +NOTE: +- This repo contains ACTUAL BACKEND CODE (not architecture) +- Used when backend is deployed separately from igny8 repo +- Follows architecture from igny8/docs/planning/ +- Files accessible only via SSH (VPS deployment) +``` + +#### 8.1.3 Development Workflow + +**Architecture First (igny8 repo):** +1. Architecture defined in `igny8/docs/planning/` (ARCHITECTURE FILES) +2. Module architecture docs created (`docs/planning/modules/{module}/`) +3. API specifications defined + +**Development Options:** + +**Option A: Single Repository (igny8)** +- Backend code in `igny8/backend/` (ACTUAL CODE) +- Frontend code in `igny8/frontend/` +- Architecture docs in `igny8/docs/planning/` (ARCHITECTURE FILES) +- All in one repo, easier coordination + +**Option B: Separate Backend Repository (VPS)** +- Backend code in separate `igny8-backend/` repo (ACTUAL CODE) +- Frontend code in `igny8/frontend/` +- Architecture docs in `igny8/docs/planning/` (ARCHITECTURE FILES) +- Backend deployed separately via SSH (VPS access only) +- Backend developers clone backend repo +- Frontend developers clone igny8 repo +- Both follow architecture from igny8 repo docs + +**Module Development:** +1. Architecture defined in `igny8/docs/planning/modules/{module}/` (ARCHITECTURE FILES) +2. Backend code developed in `igny8/backend/` OR `igny8-backend/` (ACTUAL CODE) +3. Frontend code developed in `igny8/frontend/` (ACTUAL CODE) +4. Module branches created (`module/{module-name}`) +5. Independent development with architecture as contract + ### 8.2 Branching Strategy +#### 8.2.1 IGNY8 Repository (Main Development Repository) + +**This Repository Contains:** +- **Backend code** (`backend/`) - ACTUAL IMPLEMENTATION CODE +- **Frontend code** (`frontend/`, `site-builder/`, `sites/`) - ACTUAL IMPLEMENTATION CODE +- **Architecture documentation** (`docs/planning/`) - ARCHITECTURE FILES ONLY +- Implementation plans +- API specifications + +**Branch Structure:** + | Branch | Purpose | Merge To | Protection | |--------|---------|----------|------------| -| `main` | Production-ready code | - | Protected, requires PR | +| `main` | Production-ready architecture/docs | - | Protected, requires PR | +| `develop` | Integration branch for docs | `main` | Protected, requires PR | +| `feature/*` | Feature documentation branches | `develop` | Not protected | +| `module/*` | Module-specific architecture docs | `develop` | Not protected | +| `hotfix/*` | Critical doc fixes | `main`, `develop` | Not protected | +| `release/*` | Release documentation | `main` | Protected | + +**Module-Level Branches:** +- `module/automation` - Automation module architecture +- `module/linker` - Linker module architecture +- `module/optimizer` - Optimizer module architecture +- `module/site-builder` - Site Builder architecture +- `module/integration` - Integration module architecture +- `module/publisher` - Publisher module architecture + +#### 8.2.2 Separate Backend Repository (Development) + +**Backend Repository Structure:** +- Separate Git repository (not in igny8) +- Contains all Django backend code +- Follows architecture from igny8 repo docs +- Has its own branching strategy + +**Backend Branch Structure:** + +| Branch | Purpose | Merge To | Protection | +|--------|---------|----------|------------| +| `main` | Production-ready backend | - | Protected, requires PR | | `develop` | Integration branch | `main` | Protected, requires PR | +| `module/automation` | Automation module backend | `develop` | Not protected | +| `module/linker` | Linker module backend | `develop` | Not protected | +| `module/optimizer` | Optimizer module backend | `develop` | Not protected | +| `module/site-builder` | Site Builder backend | `develop` | Not protected | +| `module/integration` | Integration module backend | `develop` | Not protected | +| `module/publisher` | Publisher module backend | `develop` | Not protected | | `feature/*` | Feature branches | `develop` | Not protected | -| `phase-2/*` | Phase 2 features | `develop` | Not protected | | `hotfix/*` | Critical fixes | `main`, `develop` | Not protected | | `release/*` | Release candidates | `main` | Protected | +**Module Development Workflow:** + +**If using single repository (igny8):** +1. Architecture defined in `igny8/docs/planning/modules/{module}/` (ARCHITECTURE FILES) +2. Backend code developed in `igny8/backend/` (`module/{module-name}` branch) +3. Frontend code developed in `igny8/frontend/` (`module/{module-name}` branch) +4. Integration testing in `develop` branches +5. Merge to `main` when complete + +**If using separate backend repo (VPS):** +1. Architecture defined in `igny8/docs/planning/modules/{module}/` (ARCHITECTURE FILES) +2. Backend code developed in `igny8-backend/` (`module/{module-name}` branch) - SSH access only +3. Frontend code developed in `igny8/frontend/` (`module/{module-name}` branch) +4. Integration testing in `develop` branches +5. Merge to `main` when complete + ### 8.3 Repository Exclusions (.gitignore) ``` @@ -613,11 +781,11 @@ docker-data/ | Task | Files | Status | Priority | |------|-------|--------|-----------| -| **Extend Content Model** | `domain/content/models.py` | TODO | HIGH | -| **Create Service Layer** | `domain/*/services/` | TODO | HIGH | +| **Extend Content Model** | `business-modules/content/models.py` | TODO | HIGH | +| **Create Service Layer** | `business-modules/*/services/` | TODO | HIGH | | **Refactor ViewSets** | `modules/*/views.py` | TODO | HIGH | -| **Implement Automation Models** | `domain/automation/models.py` | TODO | HIGH | -| **Implement Automation Service** | `domain/automation/services/` | TODO | HIGH | +| **Implement Automation Models** | `business-modules/automation/models.py` | TODO | HIGH | +| **Implement Automation Service** | `business-modules/automation/services/` | TODO | HIGH | | **Implement Automation API** | `modules/automation/` | TODO | HIGH | | **Implement Automation UI** | `frontend/src/pages/Automation/` | TODO | HIGH | | **Implement Schedules UI** | `frontend/src/pages/Schedules.tsx` | TODO | HIGH | @@ -627,8 +795,8 @@ docker-data/ | Task | Files | Dependencies | Priority | |------|-------|--------------|----------| | **Create Site Builder Container** | `docker-compose.app.yml` | Phase 0 | HIGH | -| **Site Builder Models** | `domain/site_building/models.py` | Phase 0 | HIGH | -| **Structure Generation Service** | `domain/site_building/services/` | Phase 0 | HIGH | +| **Site Builder Models** | `business-modules/site_building/models.py` | Phase 0 | HIGH | +| **Structure Generation Service** | `business-modules/site_building/services/` | Phase 0 | HIGH | | **Structure Generation AI Function** | `infrastructure/ai/functions/generate_site_structure.py` | Phase 0 | HIGH | | **Site Builder API** | `modules/site_builder/` | Phase 0 | HIGH | | **Site Builder Frontend** | `site-builder/src/` | Phase 0 | HIGH | @@ -637,12 +805,12 @@ docker-data/ | Task | Files | Dependencies | Priority | |------|-------|--------------|----------| -| **Linker Models** | `domain/linking/models.py` | Phase 0 | MEDIUM | -| **Linker Service** | `domain/linking/services/` | Phase 0 | MEDIUM | +| **Linker Models** | `business-modules/linking/models.py` | Phase 0 | MEDIUM | +| **Linker Service** | `business-modules/linking/services/` | Phase 0 | MEDIUM | | **Linker API** | `modules/linker/` | Phase 0 | MEDIUM | | **Linker UI** | `frontend/src/pages/Linker/` | Phase 0 | MEDIUM | -| **Optimizer Models** | `domain/optimization/models.py` | Phase 0 | MEDIUM | -| **Optimizer Service** | `domain/optimization/services/` | Phase 0 | MEDIUM | +| **Optimizer Models** | `business-modules/optimization/models.py` | Phase 0 | MEDIUM | +| **Optimizer Service** | `business-modules/optimization/services/` | Phase 0 | MEDIUM | | **Optimizer AI Function** | `infrastructure/ai/functions/optimize_content.py` | Phase 0 | MEDIUM | | **Optimizer API** | `modules/optimizer/` | Phase 0 | MEDIUM | | **Optimizer UI** | `frontend/src/pages/Optimizer/` | Phase 0 | MEDIUM | @@ -653,22 +821,22 @@ docker-data/ |------|-------|--------------|----------| | **Create Sites Container** | `docker-compose.app.yml` | Phase 1 | MEDIUM | | **Sites Renderer Frontend** | `sites/src/` | Phase 1 | MEDIUM | -| **Publisher Service** | `domain/publishing/services/` | Phase 0 | MEDIUM | -| **Sites Renderer Adapter** | `domain/publishing/services/adapters/` | Phase 1 | MEDIUM | +| **Publisher Service** | `business-modules/publishing/services/` | Phase 0 | MEDIUM | +| **Sites Renderer Adapter** | `business-modules/publishing/services/adapters/` | Phase 1 | MEDIUM | | **Publisher API** | `modules/publisher/` | Phase 0 | MEDIUM | -| **Deployment Service** | `domain/publishing/services/deployment_service.py` | Phase 1 | MEDIUM | +| **Deployment Service** | `business-modules/publishing/services/deployment_service.py` | Phase 1 | MEDIUM | ### 9.5 Phase 4: Universal Content Types | Task | Files | Dependencies | Priority | |------|-------|--------------|----------| -| **Extend Content Model** | `domain/content/models.py` | Phase 0 | LOW | +| **Extend Content Model** | `business-modules/content/models.py` | Phase 0 | LOW | | **Product Content Prompts** | `infrastructure/ai/prompts.py` | Phase 0 | LOW | | **Service Page Prompts** | `infrastructure/ai/prompts.py` | Phase 0 | LOW | | **Taxonomy Prompts** | `infrastructure/ai/prompts.py` | Phase 0 | LOW | -| **Content Type Support in Writer** | `domain/content/services/` | Phase 0 | LOW | -| **Content Type Support in Linker** | `domain/linking/services/` | Phase 2 | LOW | -| **Content Type Support in Optimizer** | `domain/optimization/services/` | Phase 2 | LOW | +| **Content Type Support in Writer** | `business-modules/content/services/` | Phase 0 | LOW | +| **Content Type Support in Linker** | `business-modules/linking/services/` | Phase 2 | LOW | +| **Content Type Support in Optimizer** | `business-modules/optimization/services/` | Phase 2 | LOW | --- @@ -718,11 +886,11 @@ docker-data/ | Location | Check | Implementation | |----------|-------|----------------| | **AI Engine** | Before AI call | `infrastructure/ai/engine.py` - Check credits, deduct before request | -| **Content Generation** | Before generation | `domain/content/services/content_generation_service.py` | +| **Content Generation** | Before generation | `business-modules/content/services/content_generation_service.py` | | **Image Generation** | Before generation | `infrastructure/ai/functions/generate_images.py` | -| **Linking** | Before linking | `domain/linking/services/linker_service.py` (NEW) | -| **Optimization** | Before optimization | `domain/optimization/services/optimizer_service.py` (NEW) | -| **Site Building** | Before structure gen | `domain/site_building/services/structure_generation_service.py` (NEW) | +| **Linking** | Before linking | `business-modules/linking/services/linker_service.py` (NEW) | +| **Optimization** | Before optimization | `business-modules/optimization/services/optimizer_service.py` (NEW) | +| **Site Building** | Before structure gen | `business-modules/site_building/services/structure_generation_service.py` (NEW) | ### 10.5 Credit Logging @@ -752,8 +920,8 @@ These are **NOT** business limits - they're technical constraints for request pr | Feature | Implementation | Location | |---------|----------------|----------| -| **Credit Check** | Before every AI operation | `domain/billing/services/credit_service.py` | -| **Credit Deduction** | After successful operation | `domain/billing/services/credit_service.py` | +| **Credit Check** | Before every AI operation | `business-modules/billing/services/credit_service.py` | +| **Credit Deduction** | After successful operation | `business-modules/billing/services/credit_service.py` | | **Credit Top-up** | On-demand purchase | `modules/billing/views.py` | | **Monthly Replenishment** | Celery Beat task | `infrastructure/messaging/automation_tasks.py` | | **Low Credit Warning** | When < 10% remaining | Frontend + Email notification | @@ -861,7 +1029,7 @@ publisher_service.publish( ### 11.6 Site Integration Service ```python -# domain/integration/services/integration_service.py +# business-modules/integration/services/integration_service.py class IntegrationService: def create_integration(self, site, platform, config, credentials): """Create new site integration""" @@ -913,12 +1081,12 @@ Content/Site Publishing Flow: | Component | File | Purpose | |-----------|------|---------| -| **SiteIntegration Model** | `domain/integration/models.py` | Store integration configs | -| **IntegrationService** | `domain/integration/services/integration_service.py` | Manage integrations | -| **SyncService** | `domain/integration/services/sync_service.py` | Handle two-way sync | -| **WordPressAdapter** | `domain/publishing/services/adapters/wordpress_adapter.py` | WordPress publishing | -| **SitesRendererAdapter** | `domain/publishing/services/adapters/sites_renderer_adapter.py` | IGNY8 Sites deployment | -| **ShopifyAdapter** | `domain/publishing/services/adapters/shopify_adapter.py` | Shopify publishing (future) | +| **SiteIntegration Model** | `business-modules/integration/models.py` | Store integration configs | +| **IntegrationService** | `business-modules/integration/services/integration_service.py` | Manage integrations | +| **SyncService** | `business-modules/integration/services/sync_service.py` | Handle two-way sync | +| **WordPressAdapter** | `business-modules/publishing/services/adapters/wordpress_adapter.py` | WordPress publishing | +| **SitesRendererAdapter** | `business-modules/publishing/services/adapters/sites_renderer_adapter.py` | IGNY8 Sites deployment | +| **ShopifyAdapter** | `business-modules/publishing/services/adapters/shopify_adapter.py` | Shopify publishing (future) | | **Integration API** | `modules/integration/views.py` | CRUD for integrations | | **Integration UI** | `frontend/src/pages/Settings/Integrations.tsx` | Manage integrations | @@ -960,7 +1128,7 @@ Site Builder → Publishing Flow: ### Key Architectural Decisions -1. **Domain-Driven Structure**: Organize by business domains, not technical layers +1. **Business Module Structure**: Organize by business modules, not technical layers 2. **Service Layer**: All business logic in services, ViewSets are thin 3. **Unified Content Model**: Extend existing `Content` model, don't create duplicates 4. **Automation First**: All functions can be automated via Celery + AutomationRule diff --git a/docs/planning/IGNY8-IMPLEMENTATION-PLAN.md b/docs/planning/IGNY8-IMPLEMENTATION-PLAN.md new file mode 100644 index 00000000..8e596ee0 --- /dev/null +++ b/docs/planning/IGNY8-IMPLEMENTATION-PLAN.md @@ -0,0 +1,1152 @@ +# IGNY8 IMPLEMENTATION PLAN +**Preserving Existing Functionality + Phase 2 Features** + +**Last Updated:** 2025-01-XX +**Purpose:** Step-by-step implementation plan that keeps everything working while adding Phase 2 features. + +--- + +## TABLE OF CONTENTS + +1. [Implementation Strategy](#1-implementation-strategy) +2. [Phase 0: Foundation & Credit System](#phase-0-foundation--credit-system) +3. [Phase 1: Service Layer Refactoring](#phase-1-service-layer-refactoring) +4. [Phase 2: Automation System](#phase-2-automation-system) +5. [Phase 3: Site Builder](#phase-3-site-builder) + - [3.0 Sites Folder Access & File Management](#30-sites-folder-access--file-management) +6. [Phase 4: Linker & Optimizer](#phase-4-linker--optimizer) +7. [Phase 5: Sites Renderer](#phase-5-sites-renderer) +8. [Phase 6: Site Integration & Multi-Destination Publishing](#phase-6-site-integration--multi-destination-publishing) +9. [Phase 7: UI Components & Module Settings](#phase-7-ui-components--module-settings) +10. [Phase 8: Universal Content Types](#phase-8-universal-content-types) +11. [Testing & Validation](#testing--validation) + +--- + +## 1. IMPLEMENTATION STRATEGY + +### 1.1 Core Principles + +| Principle | Implementation | +|-----------|----------------| +| **Backward Compatibility** | All existing APIs, workflows, and features continue working | +| **Incremental Changes** | Add new features without breaking existing ones | +| **Feature Flags** | Use flags to enable/disable new features during rollout | +| **Parallel Development** | New features developed alongside existing code | +| **Gradual Migration** | Move to new architecture gradually, not all at once | + +### 1.2 What's Currently Working (PRESERVE) + +| Module | Features | Status | +|--------|----------|--------| +| **Planner** | Keywords, Clusters, Ideas | ✅ Working | +| **Writer** | Tasks, Content Generation, Images | ✅ Working | +| **Thinker** | Prompts, Author Profiles, Strategies | ✅ Working | +| **Billing** | Credits, Transactions, Usage Logs | ✅ Working | +| **System** | Settings, Integration Settings | ✅ Working | +| **Auth** | Multi-tenancy, Users, Sites, Sectors | ✅ Working | +| **AI Framework** | 5 AI functions (cluster, ideas, content, prompts, images) | ✅ Working | +| **WordPress Publishing** | Content publishing to WordPress | ✅ Working | + +### 1.3 What Needs to Be Built (PHASE 2) + +| Feature | Status | Priority | +|---------|--------|----------| +| **Credit-Only System** | 🔨 Build | HIGH | +| **Automation System** | 🔨 Build | HIGH | +| **Service Layer** | 🔨 Build | HIGH | +| **Site Builder** | 🔨 Build | HIGH | +| **Linker** | 🔨 Build | MEDIUM | +| **Optimizer** | 🔨 Build | MEDIUM | +| **Sites Renderer** | 🔨 Build | MEDIUM | +| **Site Integration** | 🔨 Build | MEDIUM | +| **Universal Content Types** | 🔨 Build | LOW | + +--- + +## PHASE 0: FOUNDATION & CREDIT SYSTEM + +**Goal**: Migrate to credit-only model while preserving all existing functionality. + +### 0.0 Module Settings System (Enable/Disable Modules) + +| Task | Files | Dependencies | Risk | +|------|-------|--------------|------| +| **Add Module Enable/Disable** | `domain/system/models.py` | EXISTING (ModuleSettings) | LOW - Extend existing | +| **Module Settings API** | `modules/system/views.py` | EXISTING | LOW - Extend existing | +| **Module Settings UI** | `frontend/src/pages/Settings/Modules.tsx` | EXISTING (placeholder) | LOW - Implement UI | +| **Frontend Module Loader** | `frontend/src/config/modules.config.ts` | None | MEDIUM - Conditional loading | +| **Route Guard** | `frontend/src/components/common/ModuleGuard.tsx` | None | LOW - New component | + +**Module Enable/Disable Logic**: +- Each module has `enabled` flag in ModuleSettings +- Frontend checks module status before loading routes +- Disabled modules don't appear in sidebar +- Disabled modules don't load code (lazy loading check) + +**Modules to Control**: +- Planner +- Writer +- Thinker +- Automation +- Site Builder (NEW) +- Linker (NEW) +- Optimizer (NEW) +- Publisher (NEW) + +### 0.1 Credit System Updates + +| Task | Files | Dependencies | Risk | +|------|-------|--------------|------| +| **Remove Plan Limit Fields** | `core/auth/models.py` | None | LOW - Add migration to set defaults | +| **Update Plan Model** | `core/auth/models.py` | None | LOW - Keep only monthly_credits, support_level | +| **Update CreditService** | `domain/billing/services/credit_service.py` | None | MEDIUM - Add credit cost constants | +| **Add Credit Costs** | `domain/billing/constants.py` | None | LOW - Define credit costs per operation | +| **Update AI Engine** | `infrastructure/ai/engine.py` | CreditService | MEDIUM - Check credits before AI calls | +| **Update Content Generation** | `domain/content/services/` | CreditService | MEDIUM - Check credits before generation | +| **Update Image Generation** | `infrastructure/ai/functions/generate_images.py` | CreditService | MEDIUM - Check credits before generation | +| **Remove Limit Checks** | All services | None | MEDIUM - Remove all plan limit validations | +| **Update Usage Logging** | `domain/billing/models.py` | None | LOW - Ensure all operations log credits | +| **Update Frontend Limits UI** | `frontend/src/pages/Billing/` | Backend API | LOW - Replace limits with credit display | + +**Credit Cost Constants**: +```python +# domain/billing/constants.py +CREDIT_COSTS = { + 'clustering': 10, + 'idea_generation': 15, + 'content_generation': 1, # per 100 words + 'image_prompt_extraction': 2, + 'image_generation': 5, + 'linking': 8, # NEW + 'optimization': 1, # per 200 words, NEW + 'site_structure_generation': 50, # NEW + 'site_page_generation': 20, # NEW +} +``` + +### 0.2 Operational Limits (Keep) + +| Limit | Location | Implementation | +|-------|----------|----------------| +| **50 keywords per request** | `modules/planner/views.py` | Request validation | +| **6 images per request** | `modules/writer/views.py` | Request validation | + +### 0.3 Database Migrations + +| Migration | Purpose | Risk | +|-----------|---------|------| +| **Remove limit fields from Plan** | Clean up unused fields | LOW - Add defaults first | +| **Add credit cost tracking** | Enhance CreditUsageLog | LOW - Additive only | +| **Monthly credit replenishment** | Celery Beat task | LOW - New feature | + +### 0.4 Testing + +- ✅ All existing features work with credit checks +- ✅ Credit deduction happens correctly +- ✅ Insufficient credits show clear error +- ✅ Usage logging tracks all operations +- ✅ Frontend shows credit balance, not limits + +**Timeline**: 1-2 weeks + +--- + +## PHASE 1: SERVICE LAYER REFACTORING + +**Goal**: Extract business logic from ViewSets into services, preserving all existing functionality. + +### 1.1 Create Domain Structure + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Create domain/ folder** | `backend/igny8_core/domain/` | None | +| **Move Content models** | `domain/content/models.py` | Phase 0 | +| **Move Planning models** | `domain/planning/models.py` | Phase 0 | +| **Create ContentService** | `domain/content/services/content_generation_service.py` | Existing Writer logic | +| **Create PlanningService** | `domain/planning/services/clustering_service.py` | Existing Planner logic | +| **Create IdeasService** | `domain/planning/services/ideas_service.py` | Existing Planner logic | + +### 1.2 Refactor ViewSets (Keep APIs Working) + +| Module | Current | New | Risk | +|--------|---------|-----|------| +| **Planner ViewSets** | Business logic in views | Delegate to services | LOW - Internal refactor | +| **Writer ViewSets** | Business logic in views | Delegate to services | LOW - Internal refactor | +| **Billing ViewSets** | Already uses CreditService | Keep as-is | NONE | + +**Strategy**: +- ViewSets become thin wrappers +- All business logic moves to services +- APIs remain unchanged (backward compatible) + +### 1.3 Testing + +- ✅ All existing API endpoints work identically +- ✅ Response formats unchanged +- ✅ No breaking changes for frontend +- ✅ Services are testable independently + +**Timeline**: 2-3 weeks + +--- + +## PHASE 2: AUTOMATION SYSTEM + +**Goal**: Implement automation rules and scheduled tasks. + +### 2.1 Automation Models + +| Task | Files | Dependencies | +|------|-------|--------------| +| **AutomationRule Model** | `domain/automation/models.py` | Phase 1 | +| **ScheduledTask Model** | `domain/automation/models.py` | Phase 1 | +| **Automation Migrations** | `domain/automation/migrations/` | Phase 1 | + +### 2.2 Automation Service + +| Task | Files | Dependencies | +|------|-------|--------------| +| **AutomationService** | `domain/automation/services/automation_service.py` | Phase 1 services | +| **Rule Execution Engine** | `domain/automation/services/rule_engine.py` | Phase 1 services | +| **Condition Evaluator** | `domain/automation/services/condition_evaluator.py` | None | +| **Action Executor** | `domain/automation/services/action_executor.py` | Phase 1 services | + +### 2.3 Celery Beat Tasks + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Scheduled Automation Task** | `infrastructure/messaging/automation_tasks.py` | AutomationService | +| **Monthly Credit Replenishment** | `infrastructure/messaging/automation_tasks.py` | CreditService | +| **Celery Beat Configuration** | `backend/igny8_core/celery.py` | None | + +### 2.4 Automation API + +| Task | Files | Dependencies | +|------|-------|--------------| +| **AutomationRule ViewSet** | `modules/automation/views.py` | AutomationService | +| **ScheduledTask ViewSet** | `modules/automation/views.py` | AutomationService | +| **Automation URLs** | `modules/automation/urls.py` | None | + +### 2.5 Automation UI + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Automation Dashboard** | `frontend/src/pages/Automation/Dashboard.tsx` | EXISTING (placeholder) | +| **Rules Management** | `frontend/src/pages/Automation/Rules.tsx` | NEW | +| **Schedules Page** | `frontend/src/pages/Schedules.tsx` | EXISTING (placeholder) | +| **Automation API Client** | `frontend/src/services/automation.api.ts` | NEW | + +### 2.6 Testing + +- ✅ Automation rules execute correctly +- ✅ Scheduled tasks run on time +- ✅ Credit replenishment works monthly +- ✅ UI shows automation status + +**Timeline**: 2-3 weeks + +--- + +## PHASE 3: SITE BUILDER + +**Goal**: Build Site Builder for creating sites via wizard. + +### 3.0 Sites Folder Access & File Management + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Site File Management Service** | `domain/site_building/services/file_management_service.py` | Phase 1 | +| **User Site Access Check** | `domain/site_building/services/file_management_service.py` | EXISTING (SiteUserAccess) | +| **File Upload API** | `modules/site_builder/views.py` | File Management Service | +| **File Browser UI** | `site-builder/src/components/files/FileBrowser.tsx` | NEW | +| **Storage Quota Check** | `infrastructure/storage/file_storage.py` | Phase 1 | + +**Sites Folder Structure**: +``` +/data/app/sites-data/ +└── clients/ + └── {site_id}/ + └── v{version}/ + ├── site.json + ├── pages/ + └── assets/ # User-managed files + ├── images/ + ├── documents/ + └── media/ +``` + +**User Access Rules**: +- Owner/Admin: Full access to all account sites +- Editor: Access to granted sites (via SiteUserAccess) +- Viewer: Read-only access to granted sites +- File operations scoped to user's accessible sites only + +### 3.1 Site Builder Models + +| Task | Files | Dependencies | +|------|-------|--------------| +| **SiteBlueprint Model** | `domain/site_building/models.py` | Phase 1 | +| **PageBlueprint Model** | `domain/site_building/models.py` | Phase 1 | +| **Site Builder Migrations** | `domain/site_building/migrations/` | Phase 1 | + +### 3.2 Site Structure Generation + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Structure Generation AI Function** | `infrastructure/ai/functions/generate_site_structure.py` | Existing AI framework | +| **Structure Generation Service** | `domain/site_building/services/structure_generation_service.py` | Phase 1, AI framework | +| **Site Structure Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system | + +### 3.3 Site Builder API + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Site Builder ViewSet** | `modules/site_builder/views.py` | Structure Generation Service | +| **Site Builder URLs** | `modules/site_builder/urls.py` | None | +| **Site Builder Serializers** | `modules/site_builder/serializers.py` | None | + +### 3.4 Site Builder Frontend (New Container) + +**User-Friendly Name**: "Website Builder" or "Site Creator" + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Create Site Builder Container** | `docker-compose.app.yml` | None | +| **Wizard Steps** | `site-builder/src/pages/wizard/` | NEW | +| **Preview Canvas** | `site-builder/src/pages/preview/` | NEW | +| **Site Builder State** | `site-builder/src/state/builderStore.ts` | NEW | +| **Site Builder API Client** | `site-builder/src/api/builder.api.ts` | NEW | +| **Layout Selection** | `site-builder/src/components/layouts/` | NEW | +| **Template Library** | `site-builder/src/components/templates/` | NEW | +| **Block Components** | `site-builder/src/components/blocks/` | NEW (uses shared library) | + +### 3.7 Global Component Library + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Create Shared Component Library** | `frontend/src/components/shared/` | None | +| **Block Components** | `frontend/src/components/shared/blocks/` | NEW | +| **Layout Components** | `frontend/src/components/shared/layouts/` | NEW | +| **Template Components** | `frontend/src/components/shared/templates/` | NEW | +| **Component Documentation** | `frontend/src/components/shared/README.md` | None | + +**Component Library Structure**: +``` +frontend/src/components/shared/ +├── blocks/ +│ ├── Hero.tsx +│ ├── Features.tsx +│ ├── Services.tsx +│ ├── Products.tsx +│ ├── Testimonials.tsx +│ ├── ContactForm.tsx +│ └── ... +├── layouts/ +│ ├── DefaultLayout.tsx +│ ├── MinimalLayout.tsx +│ ├── MagazineLayout.tsx +│ ├── EcommerceLayout.tsx +│ └── ... +└── templates/ + ├── BlogTemplate.tsx + ├── BusinessTemplate.tsx + ├── PortfolioTemplate.tsx + └── ... +``` + +**Usage**: Site Builder, Sites Renderer, and Main App all use same components (no duplicates) + +### 3.5 Page Generation (Reuse Content Service) + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Extend ContentService** | `domain/content/services/content_generation_service.py` | Phase 1 | +| **Add Site Page Type** | `domain/content/models.py` | Phase 1 | +| **Page Generation Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system | + +### 3.6 Testing + +- ✅ Site Builder wizard works end-to-end +- ✅ Structure generation creates valid blueprints +- ✅ Preview renders correctly +- ✅ Page generation reuses existing content service + +**Timeline**: 3-4 weeks + +--- + +## PHASE 4: LINKER & OPTIMIZER + +**Goal**: Add linking and optimization as post-processing stages with multiple entry points. + +### 4.0 Content Workflow & Entry Points + +**Content Sources**: +1. **IGNY8 Generated** - Content created via Writer module +2. **WordPress Synced** - Content synced from WordPress via plugin +3. **3rd Party Synced** - Content synced from external sources (Shopify, custom APIs) + +**Content Storage Strategy**: +- All content stored in unified `Content` model +- `source` field: `'igny8'`, `'wordpress'`, `'shopify'`, `'custom'` +- `sync_status` field: `'native'`, `'imported'`, `'synced'` +- Same location, filtered by source/sync_status + +**Workflow Entry Points**: + +``` +Entry Point 1: Writer → Linker → Optimizer → Publish +Entry Point 2: WordPress Sync → Optimizer → Publish +Entry Point 3: 3rd Party Sync → Optimizer → Publish +Entry Point 4: Manual Selection → Linker/Optimizer +``` + +### 4.0.1 Writer → Linker Handover + +| Step | Action | Implementation | +|------|--------|---------------| +| **1. Content Generated** | Writer generates content | `ContentService.generate()` | +| **2. Content Saved** | Content saved to DB | `Content` model with `source='igny8'` | +| **3. Linker Trigger** | Auto-trigger or manual | `LinkerService.process(content_id)` | +| **4. Links Injected** | Links added to content | `Content.internal_links` updated | +| **5. Content Updated** | Content version incremented | `Content.linker_version++` | +| **6. Optimizer Ready** | Content ready for optimization | Status: `'linked'` or `'ready_for_optimization'` | + +**Auto-Trigger Options**: +- Immediate (after content generation) +- Scheduled (via AutomationRule) +- Manual (user clicks "Add Links") + +### 4.0.2 Optimizer Entry Points + +| Entry Point | Source | Trigger | Content Status | +|-------------|--------|---------|----------------| +| **Writer → Optimizer** | IGNY8 generated | After linking or direct | `source='igny8'`, `status='draft'` | +| **WordPress Sync → Optimizer** | WordPress plugin | After sync or manual | `source='wordpress'`, `sync_status='synced'` | +| **3rd Party Sync → Optimizer** | External API | After sync or manual | `source='shopify'/'custom'`, `sync_status='imported'` | +| **Manual Selection** | Any content | User selects content | Any source, any status | + +**Optimizer Processing**: +- All entry points use same `OptimizerService` +- Content source doesn't matter - optimizer works on any content +- Results stored in `OptimizationTask` model +- Original content preserved, optimized version stored separately + +### 4.1 Content Model Extensions (For Synced Content) + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Add source field** | `domain/content/models.py` | Phase 1 | +| **Add sync_status field** | `domain/content/models.py` | Phase 1 | +| **Add external_id field** | `domain/content/models.py` | Phase 1 | +| **Add sync_metadata field** | `domain/content/models.py` | Phase 1 | +| **Content Migrations** | `domain/content/migrations/` | Phase 1 | + +**Content Model Extensions**: +```python +class Content(SiteSectorBaseModel): + # Existing fields... + + # NEW: Source tracking + source = models.CharField( + max_length=50, + choices=[ + ('igny8', 'IGNY8 Generated'), + ('wordpress', 'WordPress Synced'), + ('shopify', 'Shopify Synced'), + ('custom', 'Custom API Synced'), + ], + default='igny8' + ) + + # NEW: Sync status + sync_status = models.CharField( + max_length=50, + choices=[ + ('native', 'Native IGNY8 Content'), + ('imported', 'Imported from External'), + ('synced', 'Synced from External'), + ], + default='native' + ) + + # NEW: External reference + external_id = models.CharField(max_length=255, blank=True, null=True) + external_url = models.URLField(blank=True, null=True) + sync_metadata = models.JSONField(default=dict) # Platform-specific metadata +``` + +### 4.2 Linker Models + +| Task | Files | Dependencies | +|------|-------|--------------| +| **InternalLink Model** | `domain/linking/models.py` | Phase 1 | +| **LinkGraph Model** | `domain/linking/models.py` | Phase 1 | +| **Linker Migrations** | `domain/linking/migrations/` | Phase 1 | + +### 4.3 Linker Service + +| Task | Files | Dependencies | +|------|-------|--------------| +| **LinkerService** | `domain/linking/services/linker_service.py` | Phase 1, ContentService | +| **Link Candidate Engine** | `domain/linking/services/candidate_engine.py` | Phase 1 | +| **Link Injection Engine** | `domain/linking/services/injection_engine.py` | Phase 1 | + +### 4.4 Content Sync Service (For WordPress/3rd Party) + +| Task | Files | Dependencies | +|------|-------|--------------| +| **ContentSyncService** | `domain/integration/services/content_sync_service.py` | Phase 1, Phase 6 | +| **WordPress Content Sync** | `domain/integration/services/wordpress_sync.py` | Phase 6 | +| **3rd Party Content Sync** | `domain/integration/services/external_sync.py` | Phase 6 | +| **Content Import Logic** | `domain/integration/services/import_service.py` | Phase 1 | + +**Sync Service Flow**: +```python +class ContentSyncService: + def sync_from_wordpress(self, site, sync_type='full'): + """Sync content from WordPress""" + # Fetch posts from WordPress + # Create Content records with source='wordpress', sync_status='synced' + # Store external_id, external_url + # Content ready for Optimizer + + def sync_from_shopify(self, site, sync_type='full'): + """Sync content from Shopify""" + # Fetch products/pages from Shopify + # Create Content records with source='shopify', sync_status='synced' + # Store external_id, external_url + # Content ready for Optimizer + + def import_content(self, content_data, source, external_id): + """Import content from any source""" + # Create Content record + # Mark with source and sync_status='imported' + # Content ready for Optimizer +``` + +### 4.5 Optimizer Models + +| Task | Files | Dependencies | +|------|-------|--------------| +| **OptimizationTask Model** | `domain/optimization/models.py` | Phase 1 | +| **OptimizationScores Model** | `domain/optimization/models.py` | Phase 1 | +| **Optimizer Migrations** | `domain/optimization/migrations/` | Phase 1 | + +### 4.6 Optimizer Service (Multiple Entry Points) + +| Task | Files | Dependencies | +|------|-------|--------------| +| **OptimizerService** | `domain/optimization/services/optimizer_service.py` | Phase 1, ContentService | +| **Content Analyzer** | `domain/optimization/services/analyzer.py` | Phase 1 | +| **Entry Point Handler** | `domain/optimization/services/entry_handler.py` | Phase 1 | +| **Optimization AI Function** | `infrastructure/ai/functions/optimize_content.py` | Existing AI framework | +| **Optimization Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system | + +**Optimizer Entry Point Handler**: +```python +class OptimizerService: + def optimize_from_writer(self, content_id): + """Entry Point 1: Writer → Optimizer""" + content = Content.objects.get(id=content_id, source='igny8') + return self.optimize(content) + + def optimize_from_wordpress_sync(self, content_id): + """Entry Point 2: WordPress Sync → Optimizer""" + content = Content.objects.get(id=content_id, source='wordpress') + return self.optimize(content) + + def optimize_from_external_sync(self, content_id): + """Entry Point 3: External Sync → Optimizer""" + content = Content.objects.get(id=content_id, source__in=['shopify', 'custom']) + return self.optimize(content) + + def optimize_manual(self, content_id): + """Entry Point 4: Manual Selection → Optimizer""" + content = Content.objects.get(id=content_id) + return self.optimize(content) # Works for any source + + def optimize(self, content): + """Unified optimization logic""" + # Works the same regardless of source + # Analyzes content + # Generates optimized version + # Stores in OptimizationTask + # Updates content.optimizer_version +``` + +| Task | Files | Dependencies | +|------|-------|--------------| +| **OptimizerService** | `domain/optimization/services/optimizer_service.py` | Phase 1, ContentService | +| **Content Analyzer** | `domain/optimization/services/analyzer.py` | Phase 1 | +| **Optimization AI Function** | `infrastructure/ai/functions/optimize_content.py` | Existing AI framework | +| **Optimization Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system | + +### 4.7 Content Pipeline Service + +| Task | Files | Dependencies | +|------|-------|--------------| +| **ContentPipelineService** | `domain/content/services/content_pipeline_service.py` | LinkerService, OptimizerService | +| **Pipeline Orchestration** | `domain/content/services/pipeline_service.py` | Phase 1 services | +| **Workflow State Machine** | `domain/content/services/workflow_state.py` | Phase 1 services | + +**Pipeline Workflow States**: +``` +Content States: +- 'draft' → Generated, not processed +- 'linked' → Links added, ready for optimization +- 'optimized' → Optimized, ready for review +- 'review' → Ready for publishing +- 'published' → Published to destination(s) +``` + +**Pipeline Orchestration**: +```python +class ContentPipelineService: + def process_writer_content(self, content_id, stages=['linking', 'optimization']): + """Writer → Linker → Optimizer pipeline""" + content = Content.objects.get(id=content_id, source='igny8') + + if 'linking' in stages: + content = self.link_content(content) + + if 'optimization' in stages: + content = self.optimize_content(content) + + return content + + def process_synced_content(self, content_id, stages=['optimization']): + """Synced Content → Optimizer (skip linking if needed)""" + content = Content.objects.get(id=content_id) + + if 'optimization' in stages: + content = self.optimize_content(content) + + return content +``` + +### 4.8 Linker & Optimizer APIs + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Linker ViewSet** | `modules/linker/views.py` | LinkerService | +| **Optimizer ViewSet** | `modules/optimizer/views.py` | OptimizerService | +| **Linker URLs** | `modules/linker/urls.py` | None | +| **Optimizer URLs** | `modules/optimizer/urls.py` | None | + +### 4.9 Linker & Optimizer UI + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Linker Dashboard** | `frontend/src/pages/Linker/Dashboard.tsx` | NEW | +| **Optimizer Dashboard** | `frontend/src/pages/Optimizer/Dashboard.tsx` | NEW | +| **Linker API Client** | `frontend/src/services/linker.api.ts` | NEW | +| **Optimizer API Client** | `frontend/src/services/optimizer.api.ts` | NEW | +| **Content Selection UI** | `frontend/src/components/optimizer/ContentSelector.tsx` | NEW | +| **Entry Point Handler UI** | `frontend/src/components/optimizer/EntryPointHandler.tsx` | NEW | +| **Source Badge Component** | `frontend/src/components/content/SourceBadge.tsx` | NEW | + +**Optimizer UI Features**: +- Show content source (IGNY8, WordPress, Shopify badge) +- Show sync status (Native, Synced, Imported badge) +- Entry point selection (from Writer, from Sync, Manual) +- Content list with source filters +- "Send to Optimizer" button (works for any source) + +### 4.10 Content Filtering & Display + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Content Filter Component** | `frontend/src/components/content/ContentFilter.tsx` | NEW | +| **Source Filter** | `frontend/src/components/content/SourceFilter.tsx` | NEW | +| **Sync Status Filter** | `frontend/src/components/content/SyncStatusFilter.tsx` | NEW | +| **Content List with Filters** | `frontend/src/pages/Writer/Content.tsx` | EXISTING (update) | + +**Content Filtering**: +- Filter by source: IGNY8, WordPress, Shopify, Custom +- Filter by sync_status: Native, Imported, Synced +- Filter by optimization status: Not optimized, Optimized, Needs optimization +- Filter by linking status: Not linked, Linked, Needs linking + +### 4.11 Testing + +- ✅ Writer → Linker handover works +- ✅ Linker finds appropriate link candidates +- ✅ Links inject correctly into content +- ✅ Optimizer works from Writer entry point +- ✅ Optimizer works from WordPress sync entry point +- ✅ Optimizer works from 3rd party sync entry point +- ✅ Optimizer works from manual selection +- ✅ Synced content stored correctly with source flags +- ✅ Content filtering works (by source, sync_status) +- ✅ Pipeline orchestrates correctly +- ✅ All entry points use same optimization logic + +**Timeline**: 4-5 weeks (extended for multiple entry points) + +--- + +## PHASE 5: SITES RENDERER + +**Goal**: Build Sites renderer for hosting public sites. + +### 5.1 Sites Renderer Container + +**User-Friendly Name**: "My Websites" or "Published Sites" + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Create Sites Container** | `docker-compose.app.yml` | None | +| **Sites Renderer Frontend** | `sites/src/` | NEW | +| **Block Components** | `sites/src/components/blocks/` | NEW (imports from shared) | +| **Layout Components** | `sites/src/components/layouts/` | NEW (imports from shared) | +| **Template Components** | `sites/src/components/templates/` | NEW (imports from shared) | +| **Site Definition Loader** | `sites/src/loaders/loadSiteDefinition.ts` | NEW | +| **Multiple Layout Support** | `sites/src/utils/layoutResolver.ts` | NEW | +| **Template System** | `sites/src/utils/templateEngine.ts` | NEW | +| **CMS Styling** | `sites/src/styles/cms.css` | NEW | + +### 5.6 Multiple Layout Options + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Layout Configuration** | `domain/site_building/models.py` | Phase 3 | +| **Layout Selector UI** | `site-builder/src/components/layouts/LayoutSelector.tsx` | Phase 3 | +| **Layout Renderer** | `sites/src/utils/layoutRenderer.ts` | Phase 5 | +| **Layout Preview** | `site-builder/src/components/preview/LayoutPreview.tsx` | Phase 3 | + +**Layout Types**: +- Default (Standard) +- Minimal (Clean, simple) +- Magazine (Editorial, content-focused) +- Ecommerce (Product-focused) +- Portfolio (Showcase) +- Blog (Content-first) +- Corporate (Business) + +### 5.2 Publisher Service + +| Task | Files | Dependencies | +|------|-------|--------------| +| **PublisherService** | `domain/publishing/services/publisher_service.py` | Phase 1 | +| **SitesRendererAdapter** | `domain/publishing/services/adapters/sites_renderer_adapter.py` | Phase 3 | +| **DeploymentService** | `domain/publishing/services/deployment_service.py` | Phase 3 | + +### 5.3 Publishing Models + +| Task | Files | Dependencies | +|------|-------|--------------| +| **PublishingRecord Model** | `domain/publishing/models.py` | Phase 1 | +| **DeploymentRecord Model** | `domain/publishing/models.py` | Phase 3 | +| **Publishing Migrations** | `domain/publishing/migrations/` | Phase 1 | + +### 5.4 Publisher API + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Publisher ViewSet** | `modules/publisher/views.py` | PublisherService | +| **Publisher URLs** | `modules/publisher/urls.py` | None | + +### 5.5 Testing + +- ✅ Sites renderer loads site definitions +- ✅ Blocks render correctly +- ✅ Deployment works end-to-end +- ✅ Sites are accessible publicly + +**Timeline**: 2-3 weeks + +--- + +## PHASE 6: SITE INTEGRATION & MULTI-DESTINATION PUBLISHING + +**Goal**: Support multiple publishing destinations (WordPress, Sites, Shopify). + +### 6.1 Site Integration Models + +| Task | Files | Dependencies | +|------|-------|--------------| +| **SiteIntegration Model** | `domain/integration/models.py` | Phase 1 | +| **Integration Migrations** | `domain/integration/migrations/` | Phase 1 | + +### 6.2 Integration Service + +| Task | Files | Dependencies | +|------|-------|--------------| +| **IntegrationService** | `domain/integration/services/integration_service.py` | Phase 1 | +| **SyncService** | `domain/integration/services/sync_service.py` | Phase 1 | + +### 6.3 Publishing Adapters + +| Task | Files | Dependencies | +|------|-------|--------------| +| **BaseAdapter** | `domain/publishing/services/adapters/base_adapter.py` | Phase 5 | +| **WordPressAdapter** | `domain/publishing/services/adapters/wordpress_adapter.py` | EXISTING (refactor) | +| **SitesRendererAdapter** | `domain/publishing/services/adapters/sites_renderer_adapter.py` | Phase 5 | +| **ShopifyAdapter** | `domain/publishing/services/adapters/shopify_adapter.py` | Phase 5 (future) | + +### 6.4 Multi-Destination Publishing + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Extend PublisherService** | `domain/publishing/services/publisher_service.py` | Phase 5 | +| **Multi-destination Support** | `domain/publishing/services/publisher_service.py` | Phase 5 | +| **Update PublishingRecord** | `domain/publishing/models.py` | Phase 5 | + +### 6.5 Site Model Extensions + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Add site_type field** | `core/auth/models.py` | None | +| **Add hosting_type field** | `core/auth/models.py` | None | +| **Add integrations relationship** | `core/auth/models.py` | Phase 6.1 | +| **Migration** | `core/auth/migrations/` | None | + +### 6.6 Integration API + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Integration ViewSet** | `modules/integration/views.py` | IntegrationService | +| **Integration URLs** | `modules/integration/urls.py` | None | + +### 6.7 Integration UI (Update Existing) + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Update Integration Settings** | `frontend/src/pages/Settings/Integration.tsx` | EXISTING (update) | +| **Site Integration Management** | `frontend/src/pages/Settings/Integration.tsx` | EXISTING (update) | +| **Integration API Client** | `frontend/src/services/integration.api.ts` | NEW | +| **Multi-Platform Support** | `frontend/src/components/integration/PlatformSelector.tsx` | NEW | +| **Integration Status** | `frontend/src/components/integration/IntegrationStatus.tsx` | NEW | + +**Update Existing Integration Page**: +- Add SiteIntegration model support +- Show multiple integrations per site +- Add WordPress, Shopify, Custom API options +- Two-way sync toggle +- Connection testing + +### 6.8 Publishing Settings UI + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Publishing Settings Page** | `frontend/src/pages/Settings/Publishing.tsx` | NEW | +| **Destination Management** | `frontend/src/pages/Settings/Publishing.tsx` | Phase 6 | +| **Publishing Rules** | `frontend/src/components/publishing/PublishingRules.tsx` | NEW | +| **Auto-Publish Settings** | `frontend/src/components/publishing/AutoPublishSettings.tsx` | NEW | +| **Publishing API Client** | `frontend/src/services/publisher.api.ts` | NEW | + +### 6.9 Individual Site Management (CMS) + +**User-Friendly Name**: "Site Manager" or "Content Manager" + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Site Management Dashboard** | `frontend/src/pages/Sites/Manage.tsx` | NEW | +| **Site Content Editor** | `frontend/src/pages/Sites/Editor.tsx` | NEW | +| **Post Editor** | `frontend/src/pages/Sites/PostEditor.tsx` | NEW | +| **Page Manager** | `frontend/src/pages/Sites/PageManager.tsx` | NEW | +| **Add/Remove Pages** | `frontend/src/components/sites/PageManager.tsx` | NEW | +| **Content List** | `frontend/src/components/sites/ContentList.tsx` | NEW | +| **Site Settings** | `frontend/src/pages/Sites/Settings.tsx` | NEW | +| **Site API Client** | `frontend/src/services/sites.api.ts` | NEW | + +**Site Management Features**: +- View all pages/posts for a site +- Add new pages +- Remove pages +- Edit page content +- Manage page order +- Change page templates +- Update site settings +- Preview site + +### 6.11 WordPress Plugin Connection (API Only) + +| Task | Files | Dependencies | +|------|-------|--------------| +| **WordPress Sync Endpoints** | `modules/integration/views.py` | IntegrationService | +| **Two-way Sync Logic** | `domain/integration/services/sync_service.py` | Phase 6.2 | +| **WordPress Webhook Handler** | `modules/integration/views.py` | Phase 6.2 | + +**Note**: WordPress plugin itself is built separately. Only API endpoints for plugin connection are built here. + +### 6.9 Testing + +- ✅ Site integrations work correctly +- ✅ Multi-destination publishing works +- ✅ WordPress sync works (when plugin connected) +- ✅ Two-way sync functions properly + +**Timeline**: 2-3 weeks + +--- + +## PHASE 7: UI COMPONENTS & MODULE SETTINGS + +**Goal**: Build comprehensive UI system with shared components, module settings, and site management. + +### 7.1 Global Component Library + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Component Library Structure** | `frontend/src/components/shared/` | None | +| **Block Components** | `frontend/src/components/shared/blocks/` | None | +| **Layout Components** | `frontend/src/components/shared/layouts/` | None | +| **Template Components** | `frontend/src/components/shared/templates/` | None | +| **Component Documentation** | `frontend/src/components/shared/README.md` | None | +| **Component Storybook** | `frontend/.storybook/` | Optional | +| **Component Tests** | `frontend/src/components/shared/**/*.test.tsx` | None | + +**Component Standards**: +- All components use TypeScript +- All components have props interfaces +- All components are responsive +- All components support dark mode +- All components are accessible (ARIA) +- No duplicate components + +### 7.2 Module Settings UI + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Module Settings Page** | `frontend/src/pages/Settings/Modules.tsx` | EXISTING (implement) | +| **Module Toggle Component** | `frontend/src/components/settings/ModuleToggle.tsx` | NEW | +| **Module Status Indicator** | `frontend/src/components/settings/ModuleStatus.tsx` | NEW | +| **Module Configuration** | `frontend/src/components/settings/ModuleConfig.tsx` | NEW | +| **Module Settings API** | `frontend/src/services/modules.api.ts` | NEW | + +**Module Settings Features**: +- Enable/disable modules per account +- Module-specific configuration +- Module status display +- Module usage statistics +- Module dependencies check + +### 7.3 Frontend Module Loader + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Module Config** | `frontend/src/config/modules.config.ts` | Phase 0 | +| **Module Guard** | `frontend/src/components/common/ModuleGuard.tsx` | Phase 0 | +| **Conditional Route Loading** | `frontend/src/App.tsx` | Phase 0 | +| **Sidebar Module Filter** | `frontend/src/layout/AppSidebar.tsx` | Phase 0 | + +**Module Loading Logic**: +```typescript +// Check if module is enabled before loading +const isModuleEnabled = (moduleName: string) => { + const settings = useSettingsStore(state => state.moduleSettings); + return settings[moduleName]?.enabled ?? true; // Default enabled +}; + +// Conditional route loading +{isModuleEnabled('site_builder') && ( + } /> +)} +``` + +### 7.4 Site Management UI + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Site List View** | `frontend/src/pages/Sites/List.tsx` | NEW | +| **Site Dashboard** | `frontend/src/pages/Sites/Dashboard.tsx` | NEW | +| **Site Content Manager** | `frontend/src/pages/Sites/Content.tsx` | NEW | +| **Post Editor** | `frontend/src/pages/Sites/PostEditor.tsx` | NEW | +| **Page Manager** | `frontend/src/pages/Sites/Pages.tsx` | NEW | +| **Site Settings** | `frontend/src/pages/Sites/Settings.tsx` | NEW | +| **Site Preview** | `frontend/src/pages/Sites/Preview.tsx` | NEW | + +**Site Management Features**: +- List all sites +- Site dashboard with stats +- Content management (posts, pages) +- Add/remove pages +- Edit content +- Change templates +- Update site settings +- Preview site + +### 7.5 Layout & Template System + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Layout Selector** | `frontend/src/components/sites/LayoutSelector.tsx` | NEW | +| **Template Library** | `frontend/src/components/sites/TemplateLibrary.tsx` | NEW | +| **Layout Preview** | `frontend/src/components/sites/LayoutPreview.tsx` | NEW | +| **Template Customizer** | `frontend/src/components/sites/TemplateCustomizer.tsx` | NEW | +| **Style Editor** | `frontend/src/components/sites/StyleEditor.tsx` | NEW | + +**Layout Options**: +- Default Layout +- Minimal Layout +- Magazine Layout +- Ecommerce Layout +- Portfolio Layout +- Blog Layout +- Corporate Layout + +**Template System**: +- Pre-built templates +- Custom templates +- Template customization +- Style presets +- Color schemes +- Typography options + +### 7.6 CMS Styling System + +| Task | Files | Dependencies | +|------|-------|--------------| +| **CMS Theme System** | `frontend/src/styles/cms/` | NEW | +| **Style Presets** | `frontend/src/styles/cms/presets.ts` | NEW | +| **Color Schemes** | `frontend/src/styles/cms/colors.ts` | NEW | +| **Typography System** | `frontend/src/styles/cms/typography.ts` | NEW | +| **Component Styles** | `frontend/src/styles/cms/components.ts` | NEW | + +**CMS Features**: +- Theme customization +- Color palette management +- Typography settings +- Component styling +- Responsive breakpoints +- Dark/light mode + +### 7.7 Testing + +- ✅ All components render correctly +- ✅ Module settings enable/disable modules +- ✅ Disabled modules don't load +- ✅ Site management works end-to-end +- ✅ Layout system works +- ✅ Template system works +- ✅ No duplicate components + +**Timeline**: 3-4 weeks + +--- + +## PHASE 8: UNIVERSAL CONTENT TYPES + +**Goal**: Extend content system to support products, services, taxonomies. + +### 8.1 Content Model Extensions + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Add entity_type field** | `domain/content/models.py` | Phase 1 | +| **Add json_blocks field** | `domain/content/models.py` | Phase 1 | +| **Add structure_data field** | `domain/content/models.py` | Phase 1 | +| **Content Migrations** | `domain/content/migrations/` | Phase 1 | + +### 8.2 Content Type Prompts + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Product Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system | +| **Service Page Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system | +| **Taxonomy Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system | + +### 8.3 Content Service Extensions + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Product Content Generation** | `domain/content/services/content_generation_service.py` | Phase 1 | +| **Service Page Generation** | `domain/content/services/content_generation_service.py` | Phase 1 | +| **Taxonomy Generation** | `domain/content/services/content_generation_service.py` | Phase 1 | + +### 8.4 Linker & Optimizer Extensions + +| Task | Files | Dependencies | +|------|-------|--------------| +| **Product Linking** | `domain/linking/services/linker_service.py` | Phase 4 | +| **Taxonomy Linking** | `domain/linking/services/linker_service.py` | Phase 4 | +| **Product Optimization** | `domain/optimization/services/optimizer_service.py` | Phase 4 | +| **Taxonomy Optimization** | `domain/optimization/services/optimizer_service.py` | Phase 4 | + +### 8.5 Testing + +- ✅ Product content generates correctly +- ✅ Service pages work +- ✅ Taxonomy pages work +- ✅ Linking works for all types +- ✅ Optimization works for all types + +**Timeline**: 2-3 weeks + +--- + +## TESTING & VALIDATION + +### Testing Strategy + +| Phase | Testing Focus | Validation | +|-------|--------------|------------| +| **Phase 0** | Credit system, existing features | All existing features work with credits | +| **Phase 1** | Service layer, API compatibility | APIs unchanged, services work | +| **Phase 2** | Automation execution | Rules execute, schedules work | +| **Phase 3** | Site Builder end-to-end | Sites build and deploy | +| **Phase 4** | Linker/Optimizer quality | Content improves, links work | +| **Phase 5** | Sites renderer | Sites render correctly | +| **Phase 6** | Multi-destination publishing | Content publishes to all destinations | +| **Phase 7** | UI components & module settings | Components work, modules can be disabled, site management works | +| **Phase 8** | Universal content types | All content types work | + +### Backward Compatibility Checklist + +- ✅ All existing API endpoints work +- ✅ All existing workflows function +- ✅ Frontend continues working +- ✅ Database migrations are safe +- ✅ No data loss during migration + +### Rollout Strategy + +1. **Feature Flags**: Enable new features gradually +2. **Beta Testing**: Test with select accounts first +3. **Gradual Rollout**: Enable for all users after validation +4. **Monitoring**: Track errors, performance, usage + +--- + +## SUMMARY + +### Implementation Timeline + +| Phase | Duration | Dependencies | +|-------|----------|--------------| +| **Phase 0: Credit System** | 1-2 weeks | None | +| **Phase 1: Service Layer** | 2-3 weeks | Phase 0 | +| **Phase 2: Automation** | 2-3 weeks | Phase 1 | +| **Phase 3: Site Builder** | 3-4 weeks | Phase 1, Phase 2 | +| **Phase 4: Linker/Optimizer** | 3-4 weeks | Phase 1 | +| **Phase 5: Sites Renderer** | 2-3 weeks | Phase 3 | +| **Phase 6: Site Integration** | 2-3 weeks | Phase 5 | +| **Phase 7: UI Components & Module Settings** | 3-4 weeks | Phase 0, Phase 3, Phase 5 | +| **Phase 8: Universal Types** | 2-3 weeks | Phase 4 | + +**Total Estimated Time**: 20-29 weeks (5-7 months) + +### Key Success Criteria + +- ✅ All existing features continue working +- ✅ Credit system is universal and consistent +- ✅ Automation system is functional +- ✅ Site Builder creates and deploys sites ("Website Builder" in UI) +- ✅ Sites Renderer hosts sites ("My Websites" in UI) +- ✅ Linker and Optimizer improve content +- ✅ Multi-destination publishing works +- ✅ Module settings enable/disable modules (modules don't load when disabled) +- ✅ Global component library (no duplicates, shared across all apps) +- ✅ Multiple layout options for sites (7 layout types) +- ✅ Site management UI (CMS) for content editing, page management +- ✅ Site Integration UI updated with new SiteIntegration model +- ✅ Publishing Settings UI functional +- ✅ All content types supported + +--- + +**END OF DOCUMENT** + diff --git a/Igny8-phase-2-plan.md b/docs/planning/Igny8-phase-2-plan.md similarity index 100% rename from Igny8-phase-2-plan.md rename to docs/planning/Igny8-phase-2-plan.md diff --git a/sample-usage-limits-credit-system b/docs/planning/sample-usage-limits-credit-system similarity index 100% rename from sample-usage-limits-credit-system rename to docs/planning/sample-usage-limits-credit-system diff --git a/docs/refactor/README.md b/docs/refactor/README.md new file mode 100644 index 00000000..fccef8e6 --- /dev/null +++ b/docs/refactor/README.md @@ -0,0 +1,89 @@ +# REFACTORING DOCUMENTATION + +**Purpose**: This directory contains refactoring plans, migration guides, and architectural refactoring documentation. + +--- + +## Directory Structure + +``` +docs/refactor/ +├── README.md # This file +├── routes/ # Route refactoring plans +├── folder-structure/ # Folder structure refactoring plans +└── migrations/ # Migration guides for refactoring +``` + +--- + +## Refactoring Plans + +### Current Refactoring Status + +**Phase 0: Foundation & Credit System** +- [ ] Credit-only model migration +- [ ] Plan model simplification +- [ ] Module settings system + +**Phase 1: Service Layer Refactoring** +- [ ] Domain-driven structure +- [ ] Service layer implementation +- [ ] Model migrations + +**Phase 2: Automation System** +- [ ] AutomationRule model +- [ ] ScheduledTask model +- [ ] Celery integration + +**Phase 3: Site Builder** +- [ ] Site Builder models +- [ ] File management service +- [ ] Sites folder access + +**Phase 4: Linker & Optimizer** +- [ ] Content model extensions +- [ ] Multiple entry points +- [ ] Workflow implementation + +**Phase 5: Sites Renderer** +- [ ] Sites container +- [ ] Layout system +- [ ] Template system + +**Phase 6: Site Integration** +- [ ] SiteIntegration model +- [ ] Multi-destination publishing +- [ ] Integration adapters + +**Phase 7: UI Components** +- [ ] Global component library +- [ ] Module settings UI +- [ ] Site management UI + +**Phase 8: Universal Content Types** +- [ ] Content type extensions +- [ ] Taxonomy support +- [ ] Product/Service pages + +--- + +## Route Refactoring + +See `routes/` directory for route refactoring plans. + +--- + +## Folder Structure Refactoring + +See `folder-structure/` directory for folder structure refactoring plans. + +--- + +## Migration Guides + +See `migrations/` directory for step-by-step migration guides. + +--- + +**Last Updated**: 2025-01-XX + diff --git a/docs/refactor/folder-structure/README.md b/docs/refactor/folder-structure/README.md new file mode 100644 index 00000000..f1bd09b6 --- /dev/null +++ b/docs/refactor/folder-structure/README.md @@ -0,0 +1,64 @@ +# FOLDER STRUCTURE REFACTORING PLANS + +**Purpose**: Documentation for folder structure refactoring and reorganization. + +--- + +## Current Structure + +``` +backend/igny8_core/ +├── modules/ # Feature modules +├── ai/ # AI framework +├── api/ # API base classes +└── middleware/ # Custom middleware +``` + +--- + +## Planned Structure (Domain-Driven) + +``` +backend/igny8_core/ +├── core/ # Core models (Account, User, Site, Sector) +├── domain/ # Domain-specific code +│ ├── content/ # Content domain +│ ├── planning/ # Planning domain +│ ├── linking/ # Linking domain +│ ├── optimization/# Optimization domain +│ ├── site_building/# Site building domain +│ ├── integration/ # Integration domain +│ └── billing/ # Billing domain +├── infrastructure/ # Infrastructure code +│ ├── ai/ # AI framework +│ ├── storage/ # Storage services +│ └── queue/ # Queue management +├── modules/ # Module ViewSets (thin layer) +├── shared/ # Shared utilities +└── api/ # API base classes +``` + +--- + +## Migration Strategy + +1. Create new domain folders +2. Move models to domain folders +3. Create service layer in domain folders +4. Update imports incrementally +5. Keep modules folder for ViewSets only + +--- + +## File Organization Rules + +- **Models**: `domain/{domain}/models.py` +- **Services**: `domain/{domain}/services/` +- **Serializers**: `modules/{module}/serializers.py` +- **ViewSets**: `modules/{module}/views.py` +- **URLs**: `modules/{module}/urls.py` + +--- + +**Last Updated**: 2025-01-XX + diff --git a/docs/refactor/migrations/README.md b/docs/refactor/migrations/README.md new file mode 100644 index 00000000..1bd22fbb --- /dev/null +++ b/docs/refactor/migrations/README.md @@ -0,0 +1,52 @@ +# MIGRATION GUIDES + +**Purpose**: Step-by-step migration guides for refactoring. + +--- + +## Migration Guides + +### Phase 0: Credit System Migration +- [ ] Remove plan limit fields +- [ ] Update Plan model +- [ ] Update CreditService +- [ ] Update AI Engine +- [ ] Update frontend + +### Phase 1: Service Layer Migration +- [ ] Create domain folders +- [ ] Move models +- [ ] Create services +- [ ] Update ViewSets +- [ ] Update imports + +### Phase 2: Content Model Extensions +- [ ] Add source field +- [ ] Add sync_status field +- [ ] Add external_id fields +- [ ] Create migrations +- [ ] Update serializers + +### Phase 3: New Module Integration +- [ ] Automation module +- [ ] Linker module +- [ ] Optimizer module +- [ ] Site Builder module +- [ ] Integration module + +--- + +## Migration Checklist + +For each migration: +- [ ] Create migration files +- [ ] Test migrations +- [ ] Update code references +- [ ] Update tests +- [ ] Update documentation +- [ ] Deploy incrementally + +--- + +**Last Updated**: 2025-01-XX + diff --git a/docs/refactor/routes/README.md b/docs/refactor/routes/README.md new file mode 100644 index 00000000..f08b6721 --- /dev/null +++ b/docs/refactor/routes/README.md @@ -0,0 +1,56 @@ +# ROUTE REFACTORING PLANS + +**Purpose**: Documentation for API route refactoring and reorganization. + +--- + +## Current Route Structure + +### Backend Routes +- `/api/v1/planner/` - Planner module routes +- `/api/v1/writer/` - Writer module routes +- `/api/v1/thinker/` - Thinker module routes +- `/api/v1/system/` - System module routes +- `/api/v1/billing/` - Billing module routes +- `/api/v1/auth/` - Authentication routes + +### Frontend Routes +- `/planner/*` - Planner pages +- `/writer/*` - Writer pages +- `/thinker/*` - Thinker pages +- `/settings/*` - Settings pages +- `/billing/*` - Billing pages + +--- + +## Planned Route Changes + +### Phase 1: Service Layer Routes +- New routes for service-based endpoints +- Domain-specific route organization + +### Phase 2: New Module Routes +- `/api/v1/automation/` - Automation routes +- `/api/v1/linker/` - Linker routes +- `/api/v1/optimizer/` - Optimizer routes +- `/api/v1/site-builder/` - Site Builder routes +- `/api/v1/integration/` - Integration routes + +### Phase 3: Frontend Route Updates +- New module pages +- Route guards for module access +- Conditional route loading + +--- + +## Migration Strategy + +1. Add new routes alongside existing routes +2. Gradually migrate endpoints to new structure +3. Maintain backward compatibility +4. Update frontend routes incrementally + +--- + +**Last Updated**: 2025-01-XX +