salman/igny8
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
4561f73afb96d122cbe33e5ff5dbc27854f6ff3e
igny8/IGNY8-IMPLEMENTATION-PLAN.md
IGNY8 VPS (Salman) 4561f73afb Enhance architecture plan with new site integration model and credit-based usage system 
- Introduced a new Site Integration API structure to support multiple platforms (WordPress, Shopify).
- Added credit-based usage system, removing previous plan limits and implementing a credit cost structure for various operations.
- Updated Site model to include new fields for integration and hosting types.
- Enhanced publishing workflow to support multi-destination publishing and integration adapters.
2025-11-16 16:31:24 +00:00

592 lines
22 KiB
Markdown
Raw Blame History

# IGNY8 IMPLEMENTATION PLAN
**Preserving Existing Functionality + Phase 2 Features**
**Last Updated:** 2025-01-XX
**Purpose:** Step-by-step implementation plan that keeps everything working while adding Phase 2 features.
---
## TABLE OF CONTENTS
1. [Implementation Strategy](#1-implementation-strategy)
2. [Phase 0: Foundation & Credit System](#phase-0-foundation--credit-system)
3. [Phase 1: Service Layer Refactoring](#phase-1-service-layer-refactoring)
4. [Phase 2: Automation System](#phase-2-automation-system)
5. [Phase 3: Site Builder](#phase-3-site-builder)
6. [Phase 4: Linker & Optimizer](#phase-4-linker--optimizer)
7. [Phase 5: Sites Renderer](#phase-5-sites-renderer)
8. [Phase 6: Site Integration & Multi-Destination Publishing](#phase-6-site-integration--multi-destination-publishing)
9. [Phase 7: Universal Content Types](#phase-7-universal-content-types)
10. [Testing & Validation](#testing--validation)
---
## 1. IMPLEMENTATION STRATEGY
### 1.1 Core Principles
| Principle | Implementation |
|-----------|----------------|
| **Backward Compatibility** | All existing APIs, workflows, and features continue working |
| **Incremental Changes** | Add new features without breaking existing ones |
| **Feature Flags** | Use flags to enable/disable new features during rollout |
| **Parallel Development** | New features developed alongside existing code |
| **Gradual Migration** | Move to new architecture gradually, not all at once |
### 1.2 What's Currently Working (PRESERVE)
| Module | Features | Status |
|--------|----------|--------|
| **Planner** | Keywords, Clusters, Ideas | ✅ Working |
| **Writer** | Tasks, Content Generation, Images | ✅ Working |
| **Thinker** | Prompts, Author Profiles, Strategies | ✅ Working |
| **Billing** | Credits, Transactions, Usage Logs | ✅ Working |
| **System** | Settings, Integration Settings | ✅ Working |
| **Auth** | Multi-tenancy, Users, Sites, Sectors | ✅ Working |
| **AI Framework** | 5 AI functions (cluster, ideas, content, prompts, images) | ✅ Working |
| **WordPress Publishing** | Content publishing to WordPress | ✅ Working |
### 1.3 What Needs to Be Built (PHASE 2)
| Feature | Status | Priority |
|---------|--------|----------|
| **Credit-Only System** | 🔨 Build | HIGH |
| **Automation System** | 🔨 Build | HIGH |
| **Service Layer** | 🔨 Build | HIGH |
| **Site Builder** | 🔨 Build | HIGH |
| **Linker** | 🔨 Build | MEDIUM |
| **Optimizer** | 🔨 Build | MEDIUM |
| **Sites Renderer** | 🔨 Build | MEDIUM |
| **Site Integration** | 🔨 Build | MEDIUM |
| **Universal Content Types** | 🔨 Build | LOW |
---
## PHASE 0: FOUNDATION & CREDIT SYSTEM
**Goal**: Migrate to credit-only model while preserving all existing functionality.
### 0.1 Credit System Updates
| Task | Files | Dependencies | Risk |
|------|-------|--------------|------|
| **Remove Plan Limit Fields** | `core/auth/models.py` | None | LOW - Add migration to set defaults |
| **Update Plan Model** | `core/auth/models.py` | None | LOW - Keep only monthly_credits, support_level |
| **Update CreditService** | `domain/billing/services/credit_service.py` | None | MEDIUM - Add credit cost constants |
| **Add Credit Costs** | `domain/billing/constants.py` | None | LOW - Define credit costs per operation |
| **Update AI Engine** | `infrastructure/ai/engine.py` | CreditService | MEDIUM - Check credits before AI calls |
| **Update Content Generation** | `domain/content/services/` | CreditService | MEDIUM - Check credits before generation |
| **Update Image Generation** | `infrastructure/ai/functions/generate_images.py` | CreditService | MEDIUM - Check credits before generation |
| **Remove Limit Checks** | All services | None | MEDIUM - Remove all plan limit validations |
| **Update Usage Logging** | `domain/billing/models.py` | None | LOW - Ensure all operations log credits |
| **Update Frontend Limits UI** | `frontend/src/pages/Billing/` | Backend API | LOW - Replace limits with credit display |
**Credit Cost Constants**:
```python
# domain/billing/constants.py
CREDIT_COSTS = {
'clustering': 10,
'idea_generation': 15,
'content_generation': 1, # per 100 words
'image_prompt_extraction': 2,
'image_generation': 5,
'linking': 8, # NEW
'optimization': 1, # per 200 words, NEW
'site_structure_generation': 50, # NEW
'site_page_generation': 20, # NEW
}
```
### 0.2 Operational Limits (Keep)
| Limit | Location | Implementation |
|-------|----------|----------------|
| **50 keywords per request** | `modules/planner/views.py` | Request validation |
| **6 images per request** | `modules/writer/views.py` | Request validation |
### 0.3 Database Migrations
| Migration | Purpose | Risk |
|-----------|---------|------|
| **Remove limit fields from Plan** | Clean up unused fields | LOW - Add defaults first |
| **Add credit cost tracking** | Enhance CreditUsageLog | LOW - Additive only |
| **Monthly credit replenishment** | Celery Beat task | LOW - New feature |
### 0.4 Testing
- ✅ All existing features work with credit checks
- ✅ Credit deduction happens correctly
- ✅ Insufficient credits show clear error
- ✅ Usage logging tracks all operations
- ✅ Frontend shows credit balance, not limits
**Timeline**: 1-2 weeks
---
## PHASE 1: SERVICE LAYER REFACTORING
**Goal**: Extract business logic from ViewSets into services, preserving all existing functionality.
### 1.1 Create Domain Structure
| Task | Files | Dependencies |
|------|-------|--------------|
| **Create domain/ folder** | `backend/igny8_core/domain/` | None |
| **Move Content models** | `domain/content/models.py` | Phase 0 |
| **Move Planning models** | `domain/planning/models.py` | Phase 0 |
| **Create ContentService** | `domain/content/services/content_generation_service.py` | Existing Writer logic |
| **Create PlanningService** | `domain/planning/services/clustering_service.py` | Existing Planner logic |
| **Create IdeasService** | `domain/planning/services/ideas_service.py` | Existing Planner logic |
### 1.2 Refactor ViewSets (Keep APIs Working)
| Module | Current | New | Risk |
|--------|---------|-----|------|
| **Planner ViewSets** | Business logic in views | Delegate to services | LOW - Internal refactor |
| **Writer ViewSets** | Business logic in views | Delegate to services | LOW - Internal refactor |
| **Billing ViewSets** | Already uses CreditService | Keep as-is | NONE |
**Strategy**:
- ViewSets become thin wrappers
- All business logic moves to services
- APIs remain unchanged (backward compatible)
### 1.3 Testing
- ✅ All existing API endpoints work identically
- ✅ Response formats unchanged
- ✅ No breaking changes for frontend
- ✅ Services are testable independently
**Timeline**: 2-3 weeks
---
## PHASE 2: AUTOMATION SYSTEM
**Goal**: Implement automation rules and scheduled tasks.
### 2.1 Automation Models
| Task | Files | Dependencies |
|------|-------|--------------|
| **AutomationRule Model** | `domain/automation/models.py` | Phase 1 |
| **ScheduledTask Model** | `domain/automation/models.py` | Phase 1 |
| **Automation Migrations** | `domain/automation/migrations/` | Phase 1 |
### 2.2 Automation Service
| Task | Files | Dependencies |
|------|-------|--------------|
| **AutomationService** | `domain/automation/services/automation_service.py` | Phase 1 services |
| **Rule Execution Engine** | `domain/automation/services/rule_engine.py` | Phase 1 services |
| **Condition Evaluator** | `domain/automation/services/condition_evaluator.py` | None |
| **Action Executor** | `domain/automation/services/action_executor.py` | Phase 1 services |
### 2.3 Celery Beat Tasks
| Task | Files | Dependencies |
|------|-------|--------------|
| **Scheduled Automation Task** | `infrastructure/messaging/automation_tasks.py` | AutomationService |
| **Monthly Credit Replenishment** | `infrastructure/messaging/automation_tasks.py` | CreditService |
| **Celery Beat Configuration** | `backend/igny8_core/celery.py` | None |
### 2.4 Automation API
| Task | Files | Dependencies |
|------|-------|--------------|
| **AutomationRule ViewSet** | `modules/automation/views.py` | AutomationService |
| **ScheduledTask ViewSet** | `modules/automation/views.py` | AutomationService |
| **Automation URLs** | `modules/automation/urls.py` | None |
### 2.5 Automation UI
| Task | Files | Dependencies |
|------|-------|--------------|
| **Automation Dashboard** | `frontend/src/pages/Automation/Dashboard.tsx` | EXISTING (placeholder) |
| **Rules Management** | `frontend/src/pages/Automation/Rules.tsx` | NEW |
| **Schedules Page** | `frontend/src/pages/Schedules.tsx` | EXISTING (placeholder) |
| **Automation API Client** | `frontend/src/services/automation.api.ts` | NEW |
### 2.6 Testing
- ✅ Automation rules execute correctly
- ✅ Scheduled tasks run on time
- ✅ Credit replenishment works monthly
- ✅ UI shows automation status
**Timeline**: 2-3 weeks
---
## PHASE 3: SITE BUILDER
**Goal**: Build Site Builder for creating sites via wizard.
### 3.1 Site Builder Models
| Task | Files | Dependencies |
|------|-------|--------------|
| **SiteBlueprint Model** | `domain/site_building/models.py` | Phase 1 |
| **PageBlueprint Model** | `domain/site_building/models.py` | Phase 1 |
| **Site Builder Migrations** | `domain/site_building/migrations/` | Phase 1 |
### 3.2 Site Structure Generation
| Task | Files | Dependencies |
|------|-------|--------------|
| **Structure Generation AI Function** | `infrastructure/ai/functions/generate_site_structure.py` | Existing AI framework |
| **Structure Generation Service** | `domain/site_building/services/structure_generation_service.py` | Phase 1, AI framework |
| **Site Structure Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system |
### 3.3 Site Builder API
| Task | Files | Dependencies |
|------|-------|--------------|
| **Site Builder ViewSet** | `modules/site_builder/views.py` | Structure Generation Service |
| **Site Builder URLs** | `modules/site_builder/urls.py` | None |
| **Site Builder Serializers** | `modules/site_builder/serializers.py` | None |
### 3.4 Site Builder Frontend (New Container)
| Task | Files | Dependencies |
|------|-------|--------------|
| **Create Site Builder Container** | `docker-compose.app.yml` | None |
| **Wizard Steps** | `site-builder/src/pages/wizard/` | NEW |
| **Preview Canvas** | `site-builder/src/pages/preview/` | NEW |
| **Site Builder State** | `site-builder/src/state/builderStore.ts` | NEW |
| **Site Builder API Client** | `site-builder/src/api/builder.api.ts` | NEW |
### 3.5 Page Generation (Reuse Content Service)
| Task | Files | Dependencies |
|------|-------|--------------|
| **Extend ContentService** | `domain/content/services/content_generation_service.py` | Phase 1 |
| **Add Site Page Type** | `domain/content/models.py` | Phase 1 |
| **Page Generation Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system |
### 3.6 Testing
- ✅ Site Builder wizard works end-to-end
- ✅ Structure generation creates valid blueprints
- ✅ Preview renders correctly
- ✅ Page generation reuses existing content service
**Timeline**: 3-4 weeks
---
## PHASE 4: LINKER & OPTIMIZER
**Goal**: Add linking and optimization as post-processing stages.
### 4.1 Linker Models
| Task | Files | Dependencies |
|------|-------|--------------|
| **InternalLink Model** | `domain/linking/models.py` | Phase 1 |
| **LinkGraph Model** | `domain/linking/models.py` | Phase 1 |
| **Linker Migrations** | `domain/linking/migrations/` | Phase 1 |
### 4.2 Linker Service
| Task | Files | Dependencies |
|------|-------|--------------|
| **LinkerService** | `domain/linking/services/linker_service.py` | Phase 1, ContentService |
| **Link Candidate Engine** | `domain/linking/services/candidate_engine.py` | Phase 1 |
| **Link Injection Engine** | `domain/linking/services/injection_engine.py` | Phase 1 |
### 4.3 Optimizer Models
| Task | Files | Dependencies |
|------|-------|--------------|
| **OptimizationTask Model** | `domain/optimization/models.py` | Phase 1 |
| **OptimizationScores Model** | `domain/optimization/models.py` | Phase 1 |
| **Optimizer Migrations** | `domain/optimization/migrations/` | Phase 1 |
### 4.4 Optimizer Service
| Task | Files | Dependencies |
|------|-------|--------------|
| **OptimizerService** | `domain/optimization/services/optimizer_service.py` | Phase 1, ContentService |
| **Content Analyzer** | `domain/optimization/services/analyzer.py` | Phase 1 |
| **Optimization AI Function** | `infrastructure/ai/functions/optimize_content.py` | Existing AI framework |
| **Optimization Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system |
### 4.5 Content Pipeline Service
| Task | Files | Dependencies |
|------|-------|--------------|
| **ContentPipelineService** | `domain/content/services/content_pipeline_service.py` | LinkerService, OptimizerService |
| **Pipeline Orchestration** | `domain/content/services/pipeline_service.py` | Phase 1 services |
### 4.6 Linker & Optimizer APIs
| Task | Files | Dependencies |
|------|-------|--------------|
| **Linker ViewSet** | `modules/linker/views.py` | LinkerService |
| **Optimizer ViewSet** | `modules/optimizer/views.py` | OptimizerService |
| **Linker URLs** | `modules/linker/urls.py` | None |
| **Optimizer URLs** | `modules/optimizer/urls.py` | None |
### 4.7 Linker & Optimizer UI
| Task | Files | Dependencies |
|------|-------|--------------|
| **Linker Dashboard** | `frontend/src/pages/Linker/Dashboard.tsx` | NEW |
| **Optimizer Dashboard** | `frontend/src/pages/Optimizer/Dashboard.tsx` | NEW |
| **Linker API Client** | `frontend/src/services/linker.api.ts` | NEW |
| **Optimizer API Client** | `frontend/src/services/optimizer.api.ts` | NEW |
### 4.8 Testing
- ✅ Linker finds appropriate link candidates
- ✅ Links inject correctly into content
- ✅ Optimizer improves content quality
- ✅ Pipeline orchestrates correctly
**Timeline**: 3-4 weeks
---
## PHASE 5: SITES RENDERER
**Goal**: Build Sites renderer for hosting public sites.
### 5.1 Sites Renderer Container
| Task | Files | Dependencies |
|------|-------|--------------|
| **Create Sites Container** | `docker-compose.app.yml` | None |
| **Sites Renderer Frontend** | `sites/src/` | NEW |
| **Block Component Library** | `sites/src/components/Blocks/` | NEW |
| **Site Definition Loader** | `sites/src/loaders/loadSiteDefinition.ts` | NEW |
### 5.2 Publisher Service
| Task | Files | Dependencies |
|------|-------|--------------|
| **PublisherService** | `domain/publishing/services/publisher_service.py` | Phase 1 |
| **SitesRendererAdapter** | `domain/publishing/services/adapters/sites_renderer_adapter.py` | Phase 3 |
| **DeploymentService** | `domain/publishing/services/deployment_service.py` | Phase 3 |
### 5.3 Publishing Models
| Task | Files | Dependencies |
|------|-------|--------------|
| **PublishingRecord Model** | `domain/publishing/models.py` | Phase 1 |
| **DeploymentRecord Model** | `domain/publishing/models.py` | Phase 3 |
| **Publishing Migrations** | `domain/publishing/migrations/` | Phase 1 |
### 5.4 Publisher API
| Task | Files | Dependencies |
|------|-------|--------------|
| **Publisher ViewSet** | `modules/publisher/views.py` | PublisherService |
| **Publisher URLs** | `modules/publisher/urls.py` | None |
### 5.5 Testing
- ✅ Sites renderer loads site definitions
- ✅ Blocks render correctly
- ✅ Deployment works end-to-end
- ✅ Sites are accessible publicly
**Timeline**: 2-3 weeks
---
## PHASE 6: SITE INTEGRATION & MULTI-DESTINATION PUBLISHING
**Goal**: Support multiple publishing destinations (WordPress, Sites, Shopify).
### 6.1 Site Integration Models
| Task | Files | Dependencies |
|------|-------|--------------|
| **SiteIntegration Model** | `domain/integration/models.py` | Phase 1 |
| **Integration Migrations** | `domain/integration/migrations/` | Phase 1 |
### 6.2 Integration Service
| Task | Files | Dependencies |
|------|-------|--------------|
| **IntegrationService** | `domain/integration/services/integration_service.py` | Phase 1 |
| **SyncService** | `domain/integration/services/sync_service.py` | Phase 1 |
### 6.3 Publishing Adapters
| Task | Files | Dependencies |
|------|-------|--------------|
| **BaseAdapter** | `domain/publishing/services/adapters/base_adapter.py` | Phase 5 |
| **WordPressAdapter** | `domain/publishing/services/adapters/wordpress_adapter.py` | EXISTING (refactor) |
| **SitesRendererAdapter** | `domain/publishing/services/adapters/sites_renderer_adapter.py` | Phase 5 |
| **ShopifyAdapter** | `domain/publishing/services/adapters/shopify_adapter.py` | Phase 5 (future) |
### 6.4 Multi-Destination Publishing
| Task | Files | Dependencies |
|------|-------|--------------|
| **Extend PublisherService** | `domain/publishing/services/publisher_service.py` | Phase 5 |
| **Multi-destination Support** | `domain/publishing/services/publisher_service.py` | Phase 5 |
| **Update PublishingRecord** | `domain/publishing/models.py` | Phase 5 |
### 6.5 Site Model Extensions
| Task | Files | Dependencies |
|------|-------|--------------|
| **Add site_type field** | `core/auth/models.py` | None |
| **Add hosting_type field** | `core/auth/models.py` | None |
| **Add integrations relationship** | `core/auth/models.py` | Phase 6.1 |
| **Migration** | `core/auth/migrations/` | None |
### 6.6 Integration API
| Task | Files | Dependencies |
|------|-------|--------------|
| **Integration ViewSet** | `modules/integration/views.py` | IntegrationService |
| **Integration URLs** | `modules/integration/urls.py` | None |
### 6.7 Integration UI
| Task | Files | Dependencies |
|------|-------|--------------|
| **Integration Settings** | `frontend/src/pages/Settings/Integrations.tsx` | NEW |
| **Integration API Client** | `frontend/src/services/integration.api.ts` | NEW |
### 6.8 WordPress Plugin Connection (API Only)
| Task | Files | Dependencies |
|------|-------|--------------|
| **WordPress Sync Endpoints** | `modules/integration/views.py` | IntegrationService |
| **Two-way Sync Logic** | `domain/integration/services/sync_service.py` | Phase 6.2 |
| **WordPress Webhook Handler** | `modules/integration/views.py` | Phase 6.2 |
**Note**: WordPress plugin itself is built separately. Only API endpoints for plugin connection are built here.
### 6.9 Testing
- ✅ Site integrations work correctly
- ✅ Multi-destination publishing works
- ✅ WordPress sync works (when plugin connected)
- ✅ Two-way sync functions properly
**Timeline**: 2-3 weeks
---
## PHASE 7: UNIVERSAL CONTENT TYPES
**Goal**: Extend content system to support products, services, taxonomies.
### 7.1 Content Model Extensions
| Task | Files | Dependencies |
|------|-------|--------------|
| **Add entity_type field** | `domain/content/models.py` | Phase 1 |
| **Add json_blocks field** | `domain/content/models.py` | Phase 1 |
| **Add structure_data field** | `domain/content/models.py` | Phase 1 |
| **Content Migrations** | `domain/content/migrations/` | Phase 1 |
### 7.2 Content Type Prompts
| Task | Files | Dependencies |
|------|-------|--------------|
| **Product Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system |
| **Service Page Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system |
| **Taxonomy Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system |
### 7.3 Content Service Extensions
| Task | Files | Dependencies |
|------|-------|--------------|
| **Product Content Generation** | `domain/content/services/content_generation_service.py` | Phase 1 |
| **Service Page Generation** | `domain/content/services/content_generation_service.py` | Phase 1 |
| **Taxonomy Generation** | `domain/content/services/content_generation_service.py` | Phase 1 |
### 7.4 Linker & Optimizer Extensions
| Task | Files | Dependencies |
|------|-------|--------------|
| **Product Linking** | `domain/linking/services/linker_service.py` | Phase 4 |
| **Taxonomy Linking** | `domain/linking/services/linker_service.py` | Phase 4 |
| **Product Optimization** | `domain/optimization/services/optimizer_service.py` | Phase 4 |
| **Taxonomy Optimization** | `domain/optimization/services/optimizer_service.py` | Phase 4 |
### 7.5 Testing
- ✅ Product content generates correctly
- ✅ Service pages work
- ✅ Taxonomy pages work
- ✅ Linking works for all types
- ✅ Optimization works for all types
**Timeline**: 2-3 weeks
---
## TESTING & VALIDATION
### Testing Strategy
| Phase | Testing Focus | Validation |
|-------|--------------|------------|
| **Phase 0** | Credit system, existing features | All existing features work with credits |
| **Phase 1** | Service layer, API compatibility | APIs unchanged, services work |
| **Phase 2** | Automation execution | Rules execute, schedules work |
| **Phase 3** | Site Builder end-to-end | Sites build and deploy |
| **Phase 4** | Linker/Optimizer quality | Content improves, links work |
| **Phase 5** | Sites renderer | Sites render correctly |
| **Phase 6** | Multi-destination publishing | Content publishes to all destinations |
| **Phase 7** | Universal content types | All content types work |
### Backward Compatibility Checklist
- ✅ All existing API endpoints work
- ✅ All existing workflows function
- ✅ Frontend continues working
- ✅ Database migrations are safe
- ✅ No data loss during migration
### Rollout Strategy
1. **Feature Flags**: Enable new features gradually
2. **Beta Testing**: Test with select accounts first
3. **Gradual Rollout**: Enable for all users after validation
4. **Monitoring**: Track errors, performance, usage
---
## SUMMARY
### Implementation Timeline
| Phase | Duration | Dependencies |
|-------|----------|--------------|
| **Phase 0: Credit System** | 1-2 weeks | None |
| **Phase 1: Service Layer** | 2-3 weeks | Phase 0 |
| **Phase 2: Automation** | 2-3 weeks | Phase 1 |
| **Phase 3: Site Builder** | 3-4 weeks | Phase 1, Phase 2 |
| **Phase 4: Linker/Optimizer** | 3-4 weeks | Phase 1 |
| **Phase 5: Sites Renderer** | 2-3 weeks | Phase 3 |
| **Phase 6: Site Integration** | 2-3 weeks | Phase 5 |
| **Phase 7: Universal Types** | 2-3 weeks | Phase 4 |
**Total Estimated Time**: 17-25 weeks (4-6 months)
### Key Success Criteria
- ✅ All existing features continue working
- ✅ Credit system is universal and consistent
- ✅ Automation system is functional
- ✅ Site Builder creates and deploys sites
- ✅ Linker and Optimizer improve content
- ✅ Multi-destination publishing works
- ✅ All content types supported
---
**END OF DOCUMENT**
Reference in New Issue View Git Blame Copy Permalink