Enhance architecture plan with new site integration model and credit-based usage system

- Introduced a new Site Integration API structure to support multiple platforms (WordPress, Shopify). - Added credit-based usage system, removing previous plan limits and implementing a credit cost structure for various operations. - Updated Site model to include new fields for integration and hosting types. - Enhanced publishing workflow to support multi-destination publishing and integration adapters.
2025-11-16 16:31:24 +00:00
parent bca5229c61
commit 4561f73afb
3 changed files with 982 additions and 29 deletions
--- a/IGNY8-HOLISTIC-ARCHITECTURE-PLAN.md
+++ b/IGNY8-HOLISTIC-ARCHITECTURE-PLAN.md
@@ -187,6 +187,10 @@ backend/igny8_core/
 │   │   ├── views.py
 │   │   └── urls.py
 │   │
+│   ├── integration/               # Site Integration API (NEW)
+│   │   ├── views.py
+│   │   └── urls.py
+│   │
 │   ├── automation/                # Automation API (NEW)
 │   │   ├── views.py
 │   │   ├── serializers.py
@@ -216,7 +220,8 @@ backend/igny8_core/
 │   │
 │   ├── external/                  # External integrations
 │   │   ├── wordpress.py           # WordPress client
-│   │   └── sites_renderer.py      # Sites renderer client (NEW)
+│   │   ├── sites_renderer.py      # Sites renderer client (NEW)
+│   │   └── shopify.py             # Shopify client (Future)
 │   │
 │   ├── storage/                   # Storage abstractions
 │   │   └── file_storage.py
@@ -250,10 +255,11 @@ backend/igny8_core/
 | `ContentIdeas` | `modules/planner/models.py` | `domain/planning/models.py` | Add: `entity_type` support |
 | `InternalLinks` | - | `domain/linking/models.py` | NEW: `source_id`, `target_id`, `anchor`, `position`, `link_type` |
 | `OptimizationTask` | - | `domain/optimization/models.py` | NEW: `content_id`, `type`, `target_keyword`, `scores_before`, `scores_after`, `html_before`, `html_after` |
-| `SiteBlueprint` | - | `domain/site_building/models.py` | NEW: `tenant`, `site`, `config_json`, `structure_json`, `status` |
+| `SiteBlueprint` | - | `domain/site_building/models.py` | NEW: `tenant`, `site`, `config_json`, `structure_json`, `status`, `hosting_type` |
 | `PageBlueprint` | - | `domain/site_building/models.py` | NEW: `site_blueprint`, `slug`, `type`, `blocks_json`, `status` |
-| `PublishingRecord` | - | `domain/publishing/models.py` | NEW: `content_id`, `destination`, `status`, `external_id`, `published_at` |
-| `DeploymentRecord` | - | `domain/publishing/models.py` | NEW: `site_blueprint`, `version`, `status`, `build_url`, `deployed_at` |
+| `SiteIntegration` | - | `domain/integration/models.py` | NEW: `site`, `platform`, `platform_type`, `config_json`, `credentials`, `is_active`, `sync_enabled` |
+| `PublishingRecord` | - | `domain/publishing/models.py` | NEW: `content_id`, `destination`, `destination_type`, `status`, `external_id`, `published_at`, `sync_status` |
+| `DeploymentRecord` | - | `domain/publishing/models.py` | NEW: `site_blueprint`, `version`, `status`, `build_url`, `deployed_at`, `deployment_type` |
 | `AutomationRule` | - | `domain/automation/models.py` | NEW: `name`, `trigger`, `conditions`, `actions`, `schedule`, `is_active` |
 | `ScheduledTask` | - | `domain/automation/models.py` | NEW: `automation_rule`, `scheduled_at`, `status`, `executed_at` |

@@ -666,35 +672,94 @@ docker-data/

 ---

-## 10. LIMITS & BOUNDARIES
+## 10. CREDIT-BASED USAGE SYSTEM

-### 10.1 Plan Limits (Existing)
+### 10.1 Core Principle

-| Limit | Model Field | Enforcement Location | Per |
-|-------|-------------|---------------------|-----|
-| **Max Keywords** | `Plan.max_keywords` | `domain/planning/services/clustering_service.py` | Account |
-| **Max Clusters** | `Plan.max_clusters` | `domain/planning/services/clustering_service.py` | Account |
-| **Max Content Ideas** | `Plan.max_content_ideas` | `domain/planning/services/ideas_service.py` | Account |
-| **Daily Content Tasks** | `Plan.daily_content_tasks` | `domain/content/services/content_generation_service.py` | Site |
-| **Daily AI Requests** | `Plan.daily_ai_requests` | `infrastructure/ai/engine.py` | Site |
-| **Monthly Word Count** | `Plan.monthly_word_count_limit` | `domain/content/services/content_generation_service.py` | Site |
-| **Daily Image Generation** | `Plan.daily_image_generation_limit` | `infrastructure/ai/functions/generate_images.py` | Site |
-| **Monthly Image Count** | `Plan.monthly_image_count` | `infrastructure/ai/functions/generate_images.py` | Site |
-| **Max Sites** | `Plan.max_sites` | `core/auth/models.py` (Account.save) | Account |
-| **Max Users** | `Plan.max_users` | `core/auth/models.py` (Account.save) | Account |
+**Credit-Only Model**: All features are unlimited. Only credits restrict usage.

-### 10.2 New Limits for Phase 2
+- ✅ All features unlocked for paid users
+- ✅ No plan-based limits (keywords, clusters, tasks, etc.)
+- ✅ Only credits control usage
+- ✅ Operational limits only (50 keywords/request, 6 images/request)

-| Limit | Model Field | Enforcement Location | Per |
-|-------|-------------|---------------------|-----|
-| **Max Sites Built** | `Plan.max_sites_built` (NEW) | `domain/site_building/services/` | Account |
-| **Daily Site Generations** | `Plan.daily_site_generations` (NEW) | `domain/site_building/services/` | Site |
-| **Storage Quota** | `Plan.storage_quota_gb` (NEW) | `infrastructure/storage/file_storage.py` | Site |
-| **Max Automation Rules** | `Plan.max_automation_rules` (NEW) | `domain/automation/services/` | Account |
-| **Daily Automation Executions** | `Plan.daily_automation_executions` (NEW) | `domain/automation/services/` | Site |
-| **Max Concurrent Tasks** | `Plan.max_concurrent_tasks` (NEW) | Celery queue config | Site |
+### 10.2 Plan Model (Simplified)

-### 10.3 Admin Boundaries
+| Field | Type | Purpose |
+|-------|------|---------|
+| `monthly_credits` | Integer | Credits added per month |
+| `support_level` | CharField | Support tier (basic, priority, enterprise) |
+| `billing_cycle` | CharField | 'monthly' or 'annual' |
+| `price` | Decimal | Subscription price |
+| `features` | JSONField | Feature flags (for future use) |
+
+**REMOVED FIELDS** (no longer needed):
+- `max_keywords`, `max_clusters`, `max_content_ideas`
+- `daily_content_tasks`, `monthly_word_count_limit`
+- `daily_image_generation_limit`, `monthly_image_count`
+- All other usage limits
+
+### 10.3 Credit Cost Structure
+
+| Operation | Credit Cost | Type | Notes |
+|-----------|-------------|------|-------|
+| **Clustering** | 10 credits | Fixed | Per clustering request |
+| **Idea Generation** | 15 credits | Fixed | Per cluster → ideas request |
+| **Content Generation** | 1 credit per 100 words | Variable | Based on word count |
+| **Image Generation** | 5 credits | Fixed | Per image (user sees batch) |
+| **Image Prompt Extraction** | 2 credits | Fixed | Per content piece |
+| **Linking** | 8 credits | Fixed | Per content piece (NEW) |
+| **Optimization** | 1 credit per 200 words | Variable | Per content piece (NEW) |
+| **Site Structure Generation** | 50 credits | Fixed | Per site blueprint (NEW) |
+| **Site Page Generation** | 20 credits | Fixed | Per page (NEW) |
+
+### 10.4 Credit Enforcement
+
+| Location | Check | Implementation |
+|----------|-------|----------------|
+| **AI Engine** | Before AI call | `infrastructure/ai/engine.py` - Check credits, deduct before request |
+| **Content Generation** | Before generation | `domain/content/services/content_generation_service.py` |
+| **Image Generation** | Before generation | `infrastructure/ai/functions/generate_images.py` |
+| **Linking** | Before linking | `domain/linking/services/linker_service.py` (NEW) |
+| **Optimization** | Before optimization | `domain/optimization/services/optimizer_service.py` (NEW) |
+| **Site Building** | Before structure gen | `domain/site_building/services/structure_generation_service.py` (NEW) |
+
+### 10.5 Credit Logging
+
+| Log Type | Fields | Purpose |
+|----------|--------|---------|
+| **CreditUsageLog** | `account`, `operation_type`, `credits_used`, `related_object_type`, `related_object_id`, `metadata` | Track all credit consumption |
+| **CreditTransaction** | `account`, `transaction_type`, `amount`, `description` | Track credit additions/deductions |
+
+**Usage Tracking**:
+- Total credits used (account level)
+- Credits per operation type (clustering, content, images, etc.)
+- Credits per site (site level breakdown)
+- Credits per user (user level breakdown)
+- Historical usage trends
+
+### 10.6 Operational Limits (Technical Constraints)
+
+| Limit | Value | Type | Reason |
+|-------|-------|-------|--------|
+| **Keywords per request** | 50 | Technical | API payload size, processing time |
+| **Images per request** | 6 | Technical | Queue management (user sees as batch) |
+| **Images per AI call** | 1 | Technical | Image API limitation (internal) |
+
+These are **NOT** business limits - they're technical constraints for request processing.
+
+### 10.7 Credit System Features
+
+| Feature | Implementation | Location |
+|---------|----------------|----------|
+| **Credit Check** | Before every AI operation | `domain/billing/services/credit_service.py` |
+| **Credit Deduction** | After successful operation | `domain/billing/services/credit_service.py` |
+| **Credit Top-up** | On-demand purchase | `modules/billing/views.py` |
+| **Monthly Replenishment** | Celery Beat task | `infrastructure/messaging/automation_tasks.py` |
+| **Low Credit Warning** | When < 10% remaining | Frontend + Email notification |
+| **Usage Dashboard** | Show credits by operation type | `frontend/src/pages/Billing/Usage.tsx` |
+
+### 10.8 Admin Boundaries

 | Boundary | Implementation | Location |
 |----------|----------------|----------|
@@ -705,7 +770,7 @@ docker-data/
 | **Credit Management** | Admin-only actions | `modules/billing/views.py` |
 | **Automation Management** | Role-based (Owner/Admin) | `modules/automation/views.py` |

-### 10.4 Multi-Tenancy Boundaries
+### 10.9 Multi-Tenancy Boundaries

 | Boundary | Implementation | Location |
 |----------|----------------|----------|
@@ -718,6 +783,179 @@ docker-data/

 ---

+## 11. SITE INTEGRATION & MULTI-DESTINATION PUBLISHING
+
+### 11.1 Site Integration Model
+
+| Field | Type | Purpose |
+|-------|------|---------|
+| `site` | ForeignKey(Site) | Links to Site model |
+| `platform` | CharField | Platform name ('wordpress', 'shopify', 'custom') |
+| `platform_type` | CharField | Type: 'cms', 'ecommerce', 'custom_api' |
+| `config_json` | JSONField | Platform-specific configuration |
+| `credentials` | EncryptedField | Encrypted API keys, tokens, passwords |
+| `is_active` | Boolean | Enable/disable integration |
+| `sync_enabled` | Boolean | Enable two-way sync |
+| `last_sync_at` | DateTime | Last successful sync |
+| `sync_status` | CharField | 'success', 'failed', 'pending' |
+
+**Purpose**: 
+- Sites can be NEW (built via Site Builder) or EXISTING (already integrated)
+- One site can have multiple integrations (WordPress + Shopify)
+- Integration credentials stored securely
+- Two-way sync support
+
+### 11.2 Site Types & Hosting
+
+| Site Type | Description | Integration | Hosting |
+|-----------|-------------|-------------|---------|
+| **NEW Site (IGNY8 Built)** | Created via Site Builder | None initially | IGNY8 Sites renderer |
+| **EXISTING WordPress** | Already has WordPress site | WordPress integration | WordPress hosting |
+| **EXISTING Shopify** | Already has Shopify store | Shopify integration | Shopify hosting |
+| **EXISTING Custom** | Custom platform/API | Custom integration | External hosting |
+| **Hybrid** | Built in IGNY8, also integrated | Multiple integrations | IGNY8 + External |
+
+### 11.3 Site Builder Structure
+
+```
+Site Builder Flow:
+1. User creates SiteBlueprint (via wizard)
+2. SiteBlueprint.hosting_type = 'igny8_sites' | 'wordpress' | 'shopify' | 'multi'
+3. If 'wordpress' or 'shopify':
+   - User must configure SiteIntegration first
+   - Site Builder generates content compatible with platform
+4. If 'igny8_sites':
+   - Site deployed to Sites renderer
+5. If 'multi':
+   - Site can publish to multiple destinations
+```
+
+### 11.4 Multi-Destination Publishing
+
+| Content Type | Can Publish To | Implementation |
+|--------------|----------------|----------------|
+| **Blog Posts** | WordPress, IGNY8 Sites, Shopify Blog | PublisherService routes to adapters |
+| **Pages** | WordPress, IGNY8 Sites, Shopify Pages | PublisherService routes to adapters |
+| **Products** | Shopify, WooCommerce (WordPress), IGNY8 Sites | Product adapter per platform |
+| **Categories** | WordPress, Shopify Collections, IGNY8 Sites | Taxonomy adapter per platform |
+| **Site Blueprint** | IGNY8 Sites, WordPress (via plugin), Shopify (future) | DeploymentService routes to adapters |
+
+**Publishing Flow:**
+```python
+# Single content can publish to multiple destinations
+publisher_service.publish(
+    content=content,
+    destinations=['wordpress', 'igny8_sites', 'shopify']  # Multiple destinations
+)
+```
+
+### 11.5 Integration Adapters
+
+| Adapter | Platform | Purpose | Implementation |
+|---------|----------|---------|-----------------|
+| **WordPressAdapter** | WordPress | Publish content, sync status | Uses WordPress REST API |
+| **SitesRendererAdapter** | IGNY8 Sites | Deploy site blueprints | Uploads to Sites container |
+| **ShopifyAdapter** | Shopify | Publish products, pages, blog | Uses Shopify Admin API |
+| **BaseAdapter** | Abstract | Common interface | All adapters inherit |
+
+### 11.6 Site Integration Service
+
+```python
+# domain/integration/services/integration_service.py
+class IntegrationService:
+    def create_integration(self, site, platform, config, credentials):
+        """Create new site integration"""
+    
+    def test_integration(self, integration):
+        """Test integration connection"""
+    
+    def sync_site_data(self, integration, sync_type='full'):
+        """Sync site data from external platform"""
+        # Full: All posts/products
+        # Incremental: Only changes since last sync
+    
+    def get_available_platforms(self):
+        """Get list of supported platforms"""
+        return ['wordpress', 'shopify', 'custom_api']
+```
+
+### 11.7 Site Model Extensions
+
+| Field | Type | Purpose |
+|-------|------|---------|
+| `site_type` | CharField | 'new', 'existing', 'hybrid' |
+| `hosting_type` | CharField | 'igny8_sites', 'wordpress', 'shopify', 'custom', 'multi' |
+| `integrations` | ManyToMany(SiteIntegration) | Multiple integrations per site |
+
+**Current Site Model** (needs extension):
+- Already has: `wp_url`, `wp_username`, `wp_app_password` (WordPress specific)
+- **NEW**: Move to `SiteIntegration` model for flexibility
+- **NEW**: Add `site_type` and `hosting_type` fields
+
+### 11.8 Publishing Workflow
+
+```
+Content/Site Publishing Flow:
+
+1. User selects content/site to publish
+2. System checks Site.integrations (available destinations)
+3. User selects destinations (can be multiple)
+4. PublisherService routes to appropriate adapters:
+   - WordPressAdapter.publish()
+   - SitesRendererAdapter.deploy()
+   - ShopifyAdapter.publish()
+5. PublishingRecord created for each destination
+6. Status tracked per destination
+7. Two-way sync enabled if integration.sync_enabled = True
+```
+
+### 11.9 Implementation Files
+
+| Component | File | Purpose |
+|-----------|------|---------|
+| **SiteIntegration Model** | `domain/integration/models.py` | Store integration configs |
+| **IntegrationService** | `domain/integration/services/integration_service.py` | Manage integrations |
+| **SyncService** | `domain/integration/services/sync_service.py` | Handle two-way sync |
+| **WordPressAdapter** | `domain/publishing/services/adapters/wordpress_adapter.py` | WordPress publishing |
+| **SitesRendererAdapter** | `domain/publishing/services/adapters/sites_renderer_adapter.py` | IGNY8 Sites deployment |
+| **ShopifyAdapter** | `domain/publishing/services/adapters/shopify_adapter.py` | Shopify publishing (future) |
+| **Integration API** | `modules/integration/views.py` | CRUD for integrations |
+| **Integration UI** | `frontend/src/pages/Settings/Integrations.tsx` | Manage integrations |
+
+### 11.10 Site Builder Integration Flow
+
+```
+Site Builder → Publishing Flow:
+
+1. User builds site via Site Builder wizard
+2. SiteBlueprint created with hosting_type
+3. If hosting_type = 'wordpress' or 'shopify':
+   a. User must configure SiteIntegration
+   b. Site Builder generates platform-compatible structure
+4. User clicks "Deploy"
+5. DeploymentService:
+   - If 'igny8_sites': Deploy to Sites renderer
+   - If 'wordpress': Publish via WordPressAdapter
+   - If 'shopify': Publish via ShopifyAdapter
+   - If 'multi': Publish to all configured destinations
+6. DeploymentRecord created
+7. Site available on selected platforms
+```
+
+### 11.11 Content Type Support Per Platform
+
+| Content Type | WordPress | IGNY8 Sites | Shopify | Notes |
+|--------------|-----------|-------------|---------|-------|
+| **Blog Posts** | ✅ | ✅ | ✅ (Blog) | All platforms support |
+| **Pages** | ✅ | ✅ | ✅ (Pages) | All platforms support |
+| **Products** | ✅ (WooCommerce) | ✅ | ✅ | Product adapter needed |
+| **Product Categories** | ✅ (WooCommerce) | ✅ | ✅ (Collections) | Taxonomy mapping |
+| **Product Attributes** | ✅ (WooCommerce) | ✅ | ✅ (Variants) | Attribute mapping |
+| **Service Pages** | ✅ | ✅ | ❌ | Custom content type |
+| **Taxonomy Terms** | ✅ | ✅ | ❌ | Blog-specific |
+
+---
+
 ## SUMMARY

 ### Key Architectural Decisions
--- a/IGNY8-IMPLEMENTATION-PLAN.md
+++ b/IGNY8-IMPLEMENTATION-PLAN.md
@@ -0,0 +1,591 @@
+# IGNY8 IMPLEMENTATION PLAN
+**Preserving Existing Functionality + Phase 2 Features**
+
+**Last Updated:** 2025-01-XX  
+**Purpose:** Step-by-step implementation plan that keeps everything working while adding Phase 2 features.
+
+---
+
+## TABLE OF CONTENTS
+
+1. [Implementation Strategy](#1-implementation-strategy)
+2. [Phase 0: Foundation & Credit System](#phase-0-foundation--credit-system)
+3. [Phase 1: Service Layer Refactoring](#phase-1-service-layer-refactoring)
+4. [Phase 2: Automation System](#phase-2-automation-system)
+5. [Phase 3: Site Builder](#phase-3-site-builder)
+6. [Phase 4: Linker & Optimizer](#phase-4-linker--optimizer)
+7. [Phase 5: Sites Renderer](#phase-5-sites-renderer)
+8. [Phase 6: Site Integration & Multi-Destination Publishing](#phase-6-site-integration--multi-destination-publishing)
+9. [Phase 7: Universal Content Types](#phase-7-universal-content-types)
+10. [Testing & Validation](#testing--validation)
+
+---
+
+## 1. IMPLEMENTATION STRATEGY
+
+### 1.1 Core Principles
+
+| Principle | Implementation |
+|-----------|----------------|
+| **Backward Compatibility** | All existing APIs, workflows, and features continue working |
+| **Incremental Changes** | Add new features without breaking existing ones |
+| **Feature Flags** | Use flags to enable/disable new features during rollout |
+| **Parallel Development** | New features developed alongside existing code |
+| **Gradual Migration** | Move to new architecture gradually, not all at once |
+
+### 1.2 What's Currently Working (PRESERVE)
+
+| Module | Features | Status |
+|--------|----------|--------|
+| **Planner** | Keywords, Clusters, Ideas | ✅ Working |
+| **Writer** | Tasks, Content Generation, Images | ✅ Working |
+| **Thinker** | Prompts, Author Profiles, Strategies | ✅ Working |
+| **Billing** | Credits, Transactions, Usage Logs | ✅ Working |
+| **System** | Settings, Integration Settings | ✅ Working |
+| **Auth** | Multi-tenancy, Users, Sites, Sectors | ✅ Working |
+| **AI Framework** | 5 AI functions (cluster, ideas, content, prompts, images) | ✅ Working |
+| **WordPress Publishing** | Content publishing to WordPress | ✅ Working |
+
+### 1.3 What Needs to Be Built (PHASE 2)
+
+| Feature | Status | Priority |
+|---------|--------|----------|
+| **Credit-Only System** | 🔨 Build | HIGH |
+| **Automation System** | 🔨 Build | HIGH |
+| **Service Layer** | 🔨 Build | HIGH |
+| **Site Builder** | 🔨 Build | HIGH |
+| **Linker** | 🔨 Build | MEDIUM |
+| **Optimizer** | 🔨 Build | MEDIUM |
+| **Sites Renderer** | 🔨 Build | MEDIUM |
+| **Site Integration** | 🔨 Build | MEDIUM |
+| **Universal Content Types** | 🔨 Build | LOW |
+
+---
+
+## PHASE 0: FOUNDATION & CREDIT SYSTEM
+
+**Goal**: Migrate to credit-only model while preserving all existing functionality.
+
+### 0.1 Credit System Updates
+
+| Task | Files | Dependencies | Risk |
+|------|-------|--------------|------|
+| **Remove Plan Limit Fields** | `core/auth/models.py` | None | LOW - Add migration to set defaults |
+| **Update Plan Model** | `core/auth/models.py` | None | LOW - Keep only monthly_credits, support_level |
+| **Update CreditService** | `domain/billing/services/credit_service.py` | None | MEDIUM - Add credit cost constants |
+| **Add Credit Costs** | `domain/billing/constants.py` | None | LOW - Define credit costs per operation |
+| **Update AI Engine** | `infrastructure/ai/engine.py` | CreditService | MEDIUM - Check credits before AI calls |
+| **Update Content Generation** | `domain/content/services/` | CreditService | MEDIUM - Check credits before generation |
+| **Update Image Generation** | `infrastructure/ai/functions/generate_images.py` | CreditService | MEDIUM - Check credits before generation |
+| **Remove Limit Checks** | All services | None | MEDIUM - Remove all plan limit validations |
+| **Update Usage Logging** | `domain/billing/models.py` | None | LOW - Ensure all operations log credits |
+| **Update Frontend Limits UI** | `frontend/src/pages/Billing/` | Backend API | LOW - Replace limits with credit display |
+
+**Credit Cost Constants**:
+```python
+# domain/billing/constants.py
+CREDIT_COSTS = {
+    'clustering': 10,
+    'idea_generation': 15,
+    'content_generation': 1,  # per 100 words
+    'image_prompt_extraction': 2,
+    'image_generation': 5,
+    'linking': 8,  # NEW
+    'optimization': 1,  # per 200 words, NEW
+    'site_structure_generation': 50,  # NEW
+    'site_page_generation': 20,  # NEW
+}
+```
+
+### 0.2 Operational Limits (Keep)
+
+| Limit | Location | Implementation |
+|-------|----------|----------------|
+| **50 keywords per request** | `modules/planner/views.py` | Request validation |
+| **6 images per request** | `modules/writer/views.py` | Request validation |
+
+### 0.3 Database Migrations
+
+| Migration | Purpose | Risk |
+|-----------|---------|------|
+| **Remove limit fields from Plan** | Clean up unused fields | LOW - Add defaults first |
+| **Add credit cost tracking** | Enhance CreditUsageLog | LOW - Additive only |
+| **Monthly credit replenishment** | Celery Beat task | LOW - New feature |
+
+### 0.4 Testing
+
+- ✅ All existing features work with credit checks
+- ✅ Credit deduction happens correctly
+- ✅ Insufficient credits show clear error
+- ✅ Usage logging tracks all operations
+- ✅ Frontend shows credit balance, not limits
+
+**Timeline**: 1-2 weeks
+
+---
+
+## PHASE 1: SERVICE LAYER REFACTORING
+
+**Goal**: Extract business logic from ViewSets into services, preserving all existing functionality.
+
+### 1.1 Create Domain Structure
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **Create domain/ folder** | `backend/igny8_core/domain/` | None |
+| **Move Content models** | `domain/content/models.py` | Phase 0 |
+| **Move Planning models** | `domain/planning/models.py` | Phase 0 |
+| **Create ContentService** | `domain/content/services/content_generation_service.py` | Existing Writer logic |
+| **Create PlanningService** | `domain/planning/services/clustering_service.py` | Existing Planner logic |
+| **Create IdeasService** | `domain/planning/services/ideas_service.py` | Existing Planner logic |
+
+### 1.2 Refactor ViewSets (Keep APIs Working)
+
+| Module | Current | New | Risk |
+|--------|---------|-----|------|
+| **Planner ViewSets** | Business logic in views | Delegate to services | LOW - Internal refactor |
+| **Writer ViewSets** | Business logic in views | Delegate to services | LOW - Internal refactor |
+| **Billing ViewSets** | Already uses CreditService | Keep as-is | NONE |
+
+**Strategy**: 
+- ViewSets become thin wrappers
+- All business logic moves to services
+- APIs remain unchanged (backward compatible)
+
+### 1.3 Testing
+
+- ✅ All existing API endpoints work identically
+- ✅ Response formats unchanged
+- ✅ No breaking changes for frontend
+- ✅ Services are testable independently
+
+**Timeline**: 2-3 weeks
+
+---
+
+## PHASE 2: AUTOMATION SYSTEM
+
+**Goal**: Implement automation rules and scheduled tasks.
+
+### 2.1 Automation Models
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **AutomationRule Model** | `domain/automation/models.py` | Phase 1 |
+| **ScheduledTask Model** | `domain/automation/models.py` | Phase 1 |
+| **Automation Migrations** | `domain/automation/migrations/` | Phase 1 |
+
+### 2.2 Automation Service
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **AutomationService** | `domain/automation/services/automation_service.py` | Phase 1 services |
+| **Rule Execution Engine** | `domain/automation/services/rule_engine.py` | Phase 1 services |
+| **Condition Evaluator** | `domain/automation/services/condition_evaluator.py` | None |
+| **Action Executor** | `domain/automation/services/action_executor.py` | Phase 1 services |
+
+### 2.3 Celery Beat Tasks
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **Scheduled Automation Task** | `infrastructure/messaging/automation_tasks.py` | AutomationService |
+| **Monthly Credit Replenishment** | `infrastructure/messaging/automation_tasks.py` | CreditService |
+| **Celery Beat Configuration** | `backend/igny8_core/celery.py` | None |
+
+### 2.4 Automation API
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **AutomationRule ViewSet** | `modules/automation/views.py` | AutomationService |
+| **ScheduledTask ViewSet** | `modules/automation/views.py` | AutomationService |
+| **Automation URLs** | `modules/automation/urls.py` | None |
+
+### 2.5 Automation UI
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **Automation Dashboard** | `frontend/src/pages/Automation/Dashboard.tsx` | EXISTING (placeholder) |
+| **Rules Management** | `frontend/src/pages/Automation/Rules.tsx` | NEW |
+| **Schedules Page** | `frontend/src/pages/Schedules.tsx` | EXISTING (placeholder) |
+| **Automation API Client** | `frontend/src/services/automation.api.ts` | NEW |
+
+### 2.6 Testing
+
+- ✅ Automation rules execute correctly
+- ✅ Scheduled tasks run on time
+- ✅ Credit replenishment works monthly
+- ✅ UI shows automation status
+
+**Timeline**: 2-3 weeks
+
+---
+
+## PHASE 3: SITE BUILDER
+
+**Goal**: Build Site Builder for creating sites via wizard.
+
+### 3.1 Site Builder Models
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **SiteBlueprint Model** | `domain/site_building/models.py` | Phase 1 |
+| **PageBlueprint Model** | `domain/site_building/models.py` | Phase 1 |
+| **Site Builder Migrations** | `domain/site_building/migrations/` | Phase 1 |
+
+### 3.2 Site Structure Generation
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **Structure Generation AI Function** | `infrastructure/ai/functions/generate_site_structure.py` | Existing AI framework |
+| **Structure Generation Service** | `domain/site_building/services/structure_generation_service.py` | Phase 1, AI framework |
+| **Site Structure Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system |
+
+### 3.3 Site Builder API
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **Site Builder ViewSet** | `modules/site_builder/views.py` | Structure Generation Service |
+| **Site Builder URLs** | `modules/site_builder/urls.py` | None |
+| **Site Builder Serializers** | `modules/site_builder/serializers.py` | None |
+
+### 3.4 Site Builder Frontend (New Container)
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **Create Site Builder Container** | `docker-compose.app.yml` | None |
+| **Wizard Steps** | `site-builder/src/pages/wizard/` | NEW |
+| **Preview Canvas** | `site-builder/src/pages/preview/` | NEW |
+| **Site Builder State** | `site-builder/src/state/builderStore.ts` | NEW |
+| **Site Builder API Client** | `site-builder/src/api/builder.api.ts` | NEW |
+
+### 3.5 Page Generation (Reuse Content Service)
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **Extend ContentService** | `domain/content/services/content_generation_service.py` | Phase 1 |
+| **Add Site Page Type** | `domain/content/models.py` | Phase 1 |
+| **Page Generation Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system |
+
+### 3.6 Testing
+
+- ✅ Site Builder wizard works end-to-end
+- ✅ Structure generation creates valid blueprints
+- ✅ Preview renders correctly
+- ✅ Page generation reuses existing content service
+
+**Timeline**: 3-4 weeks
+
+---
+
+## PHASE 4: LINKER & OPTIMIZER
+
+**Goal**: Add linking and optimization as post-processing stages.
+
+### 4.1 Linker Models
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **InternalLink Model** | `domain/linking/models.py` | Phase 1 |
+| **LinkGraph Model** | `domain/linking/models.py` | Phase 1 |
+| **Linker Migrations** | `domain/linking/migrations/` | Phase 1 |
+
+### 4.2 Linker Service
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **LinkerService** | `domain/linking/services/linker_service.py` | Phase 1, ContentService |
+| **Link Candidate Engine** | `domain/linking/services/candidate_engine.py` | Phase 1 |
+| **Link Injection Engine** | `domain/linking/services/injection_engine.py` | Phase 1 |
+
+### 4.3 Optimizer Models
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **OptimizationTask Model** | `domain/optimization/models.py` | Phase 1 |
+| **OptimizationScores Model** | `domain/optimization/models.py` | Phase 1 |
+| **Optimizer Migrations** | `domain/optimization/migrations/` | Phase 1 |
+
+### 4.4 Optimizer Service
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **OptimizerService** | `domain/optimization/services/optimizer_service.py` | Phase 1, ContentService |
+| **Content Analyzer** | `domain/optimization/services/analyzer.py` | Phase 1 |
+| **Optimization AI Function** | `infrastructure/ai/functions/optimize_content.py` | Existing AI framework |
+| **Optimization Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system |
+
+### 4.5 Content Pipeline Service
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **ContentPipelineService** | `domain/content/services/content_pipeline_service.py` | LinkerService, OptimizerService |
+| **Pipeline Orchestration** | `domain/content/services/pipeline_service.py` | Phase 1 services |
+
+### 4.6 Linker & Optimizer APIs
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **Linker ViewSet** | `modules/linker/views.py` | LinkerService |
+| **Optimizer ViewSet** | `modules/optimizer/views.py` | OptimizerService |
+| **Linker URLs** | `modules/linker/urls.py` | None |
+| **Optimizer URLs** | `modules/optimizer/urls.py` | None |
+
+### 4.7 Linker & Optimizer UI
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **Linker Dashboard** | `frontend/src/pages/Linker/Dashboard.tsx` | NEW |
+| **Optimizer Dashboard** | `frontend/src/pages/Optimizer/Dashboard.tsx` | NEW |
+| **Linker API Client** | `frontend/src/services/linker.api.ts` | NEW |
+| **Optimizer API Client** | `frontend/src/services/optimizer.api.ts` | NEW |
+
+### 4.8 Testing
+
+- ✅ Linker finds appropriate link candidates
+- ✅ Links inject correctly into content
+- ✅ Optimizer improves content quality
+- ✅ Pipeline orchestrates correctly
+
+**Timeline**: 3-4 weeks
+
+---
+
+## PHASE 5: SITES RENDERER
+
+**Goal**: Build Sites renderer for hosting public sites.
+
+### 5.1 Sites Renderer Container
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **Create Sites Container** | `docker-compose.app.yml` | None |
+| **Sites Renderer Frontend** | `sites/src/` | NEW |
+| **Block Component Library** | `sites/src/components/Blocks/` | NEW |
+| **Site Definition Loader** | `sites/src/loaders/loadSiteDefinition.ts` | NEW |
+
+### 5.2 Publisher Service
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **PublisherService** | `domain/publishing/services/publisher_service.py` | Phase 1 |
+| **SitesRendererAdapter** | `domain/publishing/services/adapters/sites_renderer_adapter.py` | Phase 3 |
+| **DeploymentService** | `domain/publishing/services/deployment_service.py` | Phase 3 |
+
+### 5.3 Publishing Models
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **PublishingRecord Model** | `domain/publishing/models.py` | Phase 1 |
+| **DeploymentRecord Model** | `domain/publishing/models.py` | Phase 3 |
+| **Publishing Migrations** | `domain/publishing/migrations/` | Phase 1 |
+
+### 5.4 Publisher API
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **Publisher ViewSet** | `modules/publisher/views.py` | PublisherService |
+| **Publisher URLs** | `modules/publisher/urls.py` | None |
+
+### 5.5 Testing
+
+- ✅ Sites renderer loads site definitions
+- ✅ Blocks render correctly
+- ✅ Deployment works end-to-end
+- ✅ Sites are accessible publicly
+
+**Timeline**: 2-3 weeks
+
+---
+
+## PHASE 6: SITE INTEGRATION & MULTI-DESTINATION PUBLISHING
+
+**Goal**: Support multiple publishing destinations (WordPress, Sites, Shopify).
+
+### 6.1 Site Integration Models
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **SiteIntegration Model** | `domain/integration/models.py` | Phase 1 |
+| **Integration Migrations** | `domain/integration/migrations/` | Phase 1 |
+
+### 6.2 Integration Service
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **IntegrationService** | `domain/integration/services/integration_service.py` | Phase 1 |
+| **SyncService** | `domain/integration/services/sync_service.py` | Phase 1 |
+
+### 6.3 Publishing Adapters
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **BaseAdapter** | `domain/publishing/services/adapters/base_adapter.py` | Phase 5 |
+| **WordPressAdapter** | `domain/publishing/services/adapters/wordpress_adapter.py` | EXISTING (refactor) |
+| **SitesRendererAdapter** | `domain/publishing/services/adapters/sites_renderer_adapter.py` | Phase 5 |
+| **ShopifyAdapter** | `domain/publishing/services/adapters/shopify_adapter.py` | Phase 5 (future) |
+
+### 6.4 Multi-Destination Publishing
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **Extend PublisherService** | `domain/publishing/services/publisher_service.py` | Phase 5 |
+| **Multi-destination Support** | `domain/publishing/services/publisher_service.py` | Phase 5 |
+| **Update PublishingRecord** | `domain/publishing/models.py` | Phase 5 |
+
+### 6.5 Site Model Extensions
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **Add site_type field** | `core/auth/models.py` | None |
+| **Add hosting_type field** | `core/auth/models.py` | None |
+| **Add integrations relationship** | `core/auth/models.py` | Phase 6.1 |
+| **Migration** | `core/auth/migrations/` | None |
+
+### 6.6 Integration API
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **Integration ViewSet** | `modules/integration/views.py` | IntegrationService |
+| **Integration URLs** | `modules/integration/urls.py` | None |
+
+### 6.7 Integration UI
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **Integration Settings** | `frontend/src/pages/Settings/Integrations.tsx` | NEW |
+| **Integration API Client** | `frontend/src/services/integration.api.ts` | NEW |
+
+### 6.8 WordPress Plugin Connection (API Only)
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **WordPress Sync Endpoints** | `modules/integration/views.py` | IntegrationService |
+| **Two-way Sync Logic** | `domain/integration/services/sync_service.py` | Phase 6.2 |
+| **WordPress Webhook Handler** | `modules/integration/views.py` | Phase 6.2 |
+
+**Note**: WordPress plugin itself is built separately. Only API endpoints for plugin connection are built here.
+
+### 6.9 Testing
+
+- ✅ Site integrations work correctly
+- ✅ Multi-destination publishing works
+- ✅ WordPress sync works (when plugin connected)
+- ✅ Two-way sync functions properly
+
+**Timeline**: 2-3 weeks
+
+---
+
+## PHASE 7: UNIVERSAL CONTENT TYPES
+
+**Goal**: Extend content system to support products, services, taxonomies.
+
+### 7.1 Content Model Extensions
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **Add entity_type field** | `domain/content/models.py` | Phase 1 |
+| **Add json_blocks field** | `domain/content/models.py` | Phase 1 |
+| **Add structure_data field** | `domain/content/models.py` | Phase 1 |
+| **Content Migrations** | `domain/content/migrations/` | Phase 1 |
+
+### 7.2 Content Type Prompts
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **Product Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system |
+| **Service Page Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system |
+| **Taxonomy Prompts** | `infrastructure/ai/prompts.py` | Existing prompt system |
+
+### 7.3 Content Service Extensions
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **Product Content Generation** | `domain/content/services/content_generation_service.py` | Phase 1 |
+| **Service Page Generation** | `domain/content/services/content_generation_service.py` | Phase 1 |
+| **Taxonomy Generation** | `domain/content/services/content_generation_service.py` | Phase 1 |
+
+### 7.4 Linker & Optimizer Extensions
+
+| Task | Files | Dependencies |
+|------|-------|--------------|
+| **Product Linking** | `domain/linking/services/linker_service.py` | Phase 4 |
+| **Taxonomy Linking** | `domain/linking/services/linker_service.py` | Phase 4 |
+| **Product Optimization** | `domain/optimization/services/optimizer_service.py` | Phase 4 |
+| **Taxonomy Optimization** | `domain/optimization/services/optimizer_service.py` | Phase 4 |
+
+### 7.5 Testing
+
+- ✅ Product content generates correctly
+- ✅ Service pages work
+- ✅ Taxonomy pages work
+- ✅ Linking works for all types
+- ✅ Optimization works for all types
+
+**Timeline**: 2-3 weeks
+
+---
+
+## TESTING & VALIDATION
+
+### Testing Strategy
+
+| Phase | Testing Focus | Validation |
+|-------|--------------|------------|
+| **Phase 0** | Credit system, existing features | All existing features work with credits |
+| **Phase 1** | Service layer, API compatibility | APIs unchanged, services work |
+| **Phase 2** | Automation execution | Rules execute, schedules work |
+| **Phase 3** | Site Builder end-to-end | Sites build and deploy |
+| **Phase 4** | Linker/Optimizer quality | Content improves, links work |
+| **Phase 5** | Sites renderer | Sites render correctly |
+| **Phase 6** | Multi-destination publishing | Content publishes to all destinations |
+| **Phase 7** | Universal content types | All content types work |
+
+### Backward Compatibility Checklist
+
+- ✅ All existing API endpoints work
+- ✅ All existing workflows function
+- ✅ Frontend continues working
+- ✅ Database migrations are safe
+- ✅ No data loss during migration
+
+### Rollout Strategy
+
+1. **Feature Flags**: Enable new features gradually
+2. **Beta Testing**: Test with select accounts first
+3. **Gradual Rollout**: Enable for all users after validation
+4. **Monitoring**: Track errors, performance, usage
+
+---
+
+## SUMMARY
+
+### Implementation Timeline
+
+| Phase | Duration | Dependencies |
+|-------|----------|--------------|
+| **Phase 0: Credit System** | 1-2 weeks | None |
+| **Phase 1: Service Layer** | 2-3 weeks | Phase 0 |
+| **Phase 2: Automation** | 2-3 weeks | Phase 1 |
+| **Phase 3: Site Builder** | 3-4 weeks | Phase 1, Phase 2 |
+| **Phase 4: Linker/Optimizer** | 3-4 weeks | Phase 1 |
+| **Phase 5: Sites Renderer** | 2-3 weeks | Phase 3 |
+| **Phase 6: Site Integration** | 2-3 weeks | Phase 5 |
+| **Phase 7: Universal Types** | 2-3 weeks | Phase 4 |
+
+**Total Estimated Time**: 17-25 weeks (4-6 months)
+
+### Key Success Criteria
+
+- ✅ All existing features continue working
+- ✅ Credit system is universal and consistent
+- ✅ Automation system is functional
+- ✅ Site Builder creates and deploys sites
+- ✅ Linker and Optimizer improve content
+- ✅ Multi-destination publishing works
+- ✅ All content types supported
+
+---
+
+**END OF DOCUMENT**
+
--- a/124
+++ b/124
@@ -0,0 +1,124 @@
+# Unified Credit-Based Usage System — Instructions for Cursor
+
+## Objective
+Transition IGNY8 from a limit-based subscription model to a fully credit-driven usage model.  
+Every feature is unlocked for all paid users, and only credits determine how much the platform can be used.
+
+---
+
+## 1. Core Principle
+Remove all numerical limits tied to subscription plans.  
+Examples of limits that must no longer exist:
+
+- Number of keywords allowed  
+- Number of clusters  
+- Number of content ideas  
+- Daily or monthly content tasks  
+- Monthly word count  
+- Image generation limits  
+- Site limits  
+- User limits  
+- Any plan-based feature restrictions  
+
+The only limiter in the entire system should be **credits**.
+
+---
+
+## 2. Purpose of Subscription Plans
+Plans should no longer define usage capacity.  
+Plans only define:
+
+- Monthly credits added to the account  
+- Support and onboarding level  
+- Billing cycle (monthly or yearly)
+
+Every paid user gets full access to all features without restrictions.
+
+---
+
+## 3. Credit Economy
+Every system action should consume a defined number of credits.  
+The values can be adjusted later, but the mechanism must be universal.
+
+Example conceptual structure:
+
+- One clustering request consumes a small number of credits  
+- One idea generation request consumes more credits  
+- Content generation consumes credits based on word count  
+- Image generation consumes credits per image  
+- Optimization consumes credits per thousand words  
+- Linking operations consume a fixed number of credits  
+
+Credits should be deducted at the moment the action is initiated.
+
+---
+
+## 4. System Behavior
+Cursor should apply these global rules:
+
+- If the account has enough credits, the action proceeds  
+- If credits are insufficient, the system shows a clear warning and does not run the action  
+- All features remain available, but actions require credits  
+- Credit consumption must be tracked and logged for transparency  
+- Credit usage must be consistent across all modules  
+
+This ensures that usage is pay-as-you-go within the subscription credit budget.
+
+---
+
+## 5. Removal of Old Logic
+All of the following must be removed from the system:
+
+- Any checks for keyword count  
+- Any maximum allowed clusters  
+- Any caps on ideas  
+- Any daily task limits  
+- Any monthly generation limits  
+- Any image generation limits  
+- Any limits based on the user’s plan  
+- Any feature locking based on plan tier  
+
+The platform should feel “unlimited,” with only credit balance controlling usage.
+
+---
+
+## 6. Frontend Adjustments
+On the interface:
+
+- Remove UI elements showing limits  
+- Replace them with a simple display of remaining credits  
+- Show estimated credit cost before performing any action  
+- If credits are insufficient, display a clear prompt to buy more credits  
+- All features must appear unlocked and available  
+
+The user should understand that only credits restrict their usage, not the plan.
+
+---
+
+## 7. Billing and Top-ups
+Plans supply monthly credits, but users should be able to purchase extra credits anytime.  
+The system must support:
+
+- Monthly credit replenishment  
+- On-demand credit top-ups  
+- Tracking of credit usage history  
+- Notifications when credits run low  
+
+This ensures consistent monetization and scalability.
+
+---
+
+## 8. Guiding Principle for Cursor
+Cursor must treat **credits as the universal currency of usage**, and remove every other form of operational restriction.
+
+The product becomes:
+
+- Simpler for users  
+- Simpler for engineering  
+- Easier to maintain  
+- Scalable in pricing  
+- Fully usage-based  
+
+All future modules (Optimizer, Linker, etc.) must also follow the same credit model.
+
+---