igny8/docs/planning/phases/PHASE-0-FOUNDATION-CREDIT-SYSTEM.md

# PHASE 0: FOUNDATION & CREDIT SYSTEM
**Detailed Implementation Plan**

**Goal**: Migrate to credit-only model while preserving all existing functionality.

**Timeline**: 1-2 weeks  
**Priority**: HIGH  
**Dependencies**: None

---

## TABLE OF CONTENTS

1. [Overview](#overview)
2. [Module Settings System](#module-settings-system)
3. [Credit System Updates](#credit-system-updates)
4. [Operational Limits](#operational-limits)
5. [Database Migrations](#database-migrations)
6. [Testing & Validation](#testing--validation)
7. [Implementation Checklist](#implementation-checklist)

---

## OVERVIEW

### Objectives
- ✅ Migrate from plan-based limits to credit-only system
- ✅ Implement module enable/disable functionality
- ✅ Add credit cost tracking for all operations
- ✅ Preserve all existing functionality
- ✅ Update frontend to show credits instead of limits

### Key Principles
- **Backward Compatibility**: All existing APIs continue working
- **No Breaking Changes**: Frontend continues working without changes
- **Gradual Migration**: Add credit checks without removing existing code initially
- **Credit-Only Model**: Remove all plan limit fields, keep only credits

---

## MODULE SETTINGS SYSTEM

### 0.0 Module Settings System (Enable/Disable Modules)

**Purpose**: Allow accounts to enable/disable modules per account.

#### Backend Implementation

| Task | File | Current State | Implementation |
|------|------|---------------|----------------|
| **Extend ModuleSettings Model** | `business/system/models.py` | EXISTING (ModuleSettings) | Add `enabled` boolean field per module |
| **Module Settings API** | `modules/system/views.py` | EXISTING | Extend ViewSet to handle enable/disable |
| **Module Settings Serializer** | `modules/system/serializers.py` | EXISTING | Add enabled field to serializer |

**ModuleSettings Model Extension**:
```python
# business/system/models.py (or core/system/models.py if exists)
class ModuleSettings(AccountBaseModel):
    # Existing fields...
    
    # NEW: Module enable/disable flags
    planner_enabled = models.BooleanField(default=True)
    writer_enabled = models.BooleanField(default=True)
    thinker_enabled = models.BooleanField(default=True)
    automation_enabled = models.BooleanField(default=True)
    site_builder_enabled = models.BooleanField(default=True)
    linker_enabled = models.BooleanField(default=True)
    optimizer_enabled = models.BooleanField(default=True)
    publisher_enabled = models.BooleanField(default=True)
```

**Modules to Control**:
- Planner
- Writer
- Thinker
- Automation
- Site Builder (NEW)
- Linker (NEW)
- Optimizer (NEW)
- Publisher (NEW)

#### Frontend Implementation

| Task | File | Current State | Implementation |
|------|------|---------------|----------------|
| **Module Settings UI** | `frontend/src/pages/Settings/Modules.tsx` | EXISTING (placeholder) | Implement toggle UI for each module |
| **Frontend Module Loader** | `frontend/src/config/modules.config.ts` | NEW | Define module config with enabled checks |
| **Route Guard** | `frontend/src/components/common/ModuleGuard.tsx` | NEW | Component to check module status before rendering |
| **Sidebar Filter** | `frontend/src/layout/AppSidebar.tsx` | EXISTING | Filter out disabled modules from sidebar |

**Module Enable/Disable Logic**:
- Each module has `enabled` flag in ModuleSettings
- Frontend checks module status before loading routes
- Disabled modules don't appear in sidebar
- Disabled modules don't load code (lazy loading check)

**Module Config Example**:
```typescript
// frontend/src/config/modules.config.ts
export const MODULES = {
  planner: {
    name: 'Planner',
    route: '/planner',
    enabled: true, // Checked from API
  },
  writer: {
    name: 'Writer',
    route: '/writer',
    enabled: true,
  },
  // ... other modules
};
```

**Route Guard Example**:
```typescript
// frontend/src/components/common/ModuleGuard.tsx
const ModuleGuard = ({ module, children }) => {
  const { moduleSettings } = useSettingsStore();
  const isEnabled = moduleSettings[module]?.enabled ?? true;
  
  if (!isEnabled) {
    return <Navigate to="/settings/modules" />;
  }
  
  return children;
};
```

---

## CREDIT SYSTEM UPDATES

### 0.1 Credit System Updates

**Purpose**: Migrate from plan-based limits to credit-only system.

#### Plan Model Updates

| Task | File | Current State | Implementation |
|------|------|---------------|----------------|
| **Remove Plan Limit Fields** | `core/auth/models.py` | EXISTING | Remove all limit fields, add migration |
| **Update Plan Model** | `core/auth/models.py` | EXISTING | Keep only `monthly_credits`, `support_level`, `billing_cycle`, `price` |

**Plan Model (Simplified)**:
```python
# core/auth/models.py
class Plan(models.Model):
    name = models.CharField(max_length=255)
    monthly_credits = models.IntegerField(default=0)  # KEEP
    support_level = models.CharField(max_length=50)  # KEEP
    billing_cycle = models.CharField(max_length=20)  # KEEP
    price = models.DecimalField(max_digits=10, decimal_places=2)  # KEEP
    features = models.JSONField(default=dict)  # KEEP (for future use)
    
    # REMOVE: All limit fields
    # - max_keywords
    # - max_clusters
    # - max_content_ideas
    # - daily_content_tasks
    # - monthly_word_count_limit
    # - daily_image_generation_limit
    # - monthly_image_count
    # - etc.
```

**Migration Strategy**:
1. Create migration to add defaults for removed fields (if needed)
2. Create migration to remove limit fields
3. Ensure existing accounts have credit balances set

#### Credit Cost Constants

| Task | File | Current State | Implementation |
|------|------|---------------|----------------|
| **Add Credit Costs** | `business/billing/constants.py` | NEW | Define credit costs per operation |

**Credit Cost Constants**:
```python
# business/billing/constants.py
CREDIT_COSTS = {
    'clustering': 10,                    # Per clustering request
    'idea_generation': 15,               # Per cluster → ideas request
    'content_generation': 1,              # Per 100 words
    'image_prompt_extraction': 2,        # Per content piece
    'image_generation': 5,               # Per image
    'linking': 8,                        # Per content piece (NEW)
    'optimization': 1,                   # Per 200 words (NEW)
    'site_structure_generation': 50,     # Per site blueprint (NEW)
    'site_page_generation': 20,          # Per page (NEW)
}
```

#### CreditService Updates

| Task | File | Current State | Implementation |
|------|------|---------------|----------------|
| **Update CreditService** | `business/billing/services/credit_service.py` | EXISTING | Add credit cost constants, update methods |

**CreditService Methods**:
```python
# business/billing/services/credit_service.py
class CreditService:
    def check_credits(self, account, operation_type, amount=None):
        """Check if account has sufficient credits"""
        required = self.get_credit_cost(operation_type, amount)
        if account.credits < required:
            raise InsufficientCreditsError(f"Need {required} credits, have {account.credits}")
        return True
    
    def deduct_credits(self, account, operation_type, amount=None):
        """Deduct credits after operation"""
        cost = self.get_credit_cost(operation_type, amount)
        account.credits -= cost
        account.save()
        # Log usage
        CreditUsageLog.objects.create(...)
    
    def get_credit_cost(self, operation_type, amount=None):
        """Get credit cost for operation"""
        base_cost = CREDIT_COSTS.get(operation_type, 0)
        if operation_type == 'content_generation' and amount:
            return base_cost * (amount / 100)  # Per 100 words
        if operation_type == 'optimization' and amount:
            return base_cost * (amount / 200)  # Per 200 words
        return base_cost
```

#### AI Engine Updates

| Task | File | Current State | Implementation |
|------|------|---------------|----------------|
| **Update AI Engine** | `infrastructure/ai/engine.py` | EXISTING | Check credits before AI calls |

**AI Engine Credit Check**:
```python
# infrastructure/ai/engine.py
class AIEngine:
    def execute(self, function, payload, account):
        # Check credits BEFORE AI call
        operation_type = function.get_operation_type()
        estimated_cost = function.get_estimated_cost(payload)
        
        credit_service.check_credits(account, operation_type, estimated_cost)
        
        # Execute AI function
        result = function.execute(payload)
        
        # Deduct credits AFTER successful execution
        credit_service.deduct_credits(account, operation_type, actual_cost)
        
        return result
```

#### Content Generation Updates

| Task | File | Current State | Implementation |
|------|------|---------------|----------------|
| **Update Content Generation** | `business/content/services/content_generation_service.py` | NEW (Phase 1) | Check credits before generation |

**Content Generation Credit Check**:
```python
# business/content/services/content_generation_service.py
class ContentGenerationService:
    def generate_content(self, task, account):
        # Check credits before generation
        estimated_words = task.estimated_word_count or 1000
        credit_service.check_credits(account, 'content_generation', estimated_words)
        
        # Generate content
        content = self._generate(task)
        
        # Deduct credits after generation
        actual_words = content.word_count
        credit_service.deduct_credits(account, 'content_generation', actual_words)
        
        return content
```

#### Image Generation Updates

| Task | File | Current State | Implementation |
|------|------|---------------|----------------|
| **Update Image Generation** | `infrastructure/ai/functions/generate_images.py` | EXISTING | Check credits before generation |

**Image Generation Credit Check**:
```python
# infrastructure/ai/functions/generate_images.py
class GenerateImagesFunction(BaseAIFunction):
    def execute(self, payload, account):
        image_ids = payload['image_ids']
        
        # Check credits before generation
        credit_service.check_credits(account, 'image_generation', len(image_ids))
        
        # Generate images
        results = self._generate_images(image_ids)
        
        # Deduct credits after generation
        credit_service.deduct_credits(account, 'image_generation', len(results))
        
        return results
```

#### Remove Limit Checks

| Task | File | Current State | Implementation |
|------|------|---------------|----------------|
| **Remove Limit Checks** | All services | EXISTING | Remove all plan limit validations |

**Files to Update**:
- `modules/planner/views.py` - Remove keyword/cluster limit checks
- `modules/writer/views.py` - Remove task/content limit checks
- `infrastructure/ai/engine.py` - Remove plan limit checks
- All ViewSets - Remove limit validation

**Before (Remove)**:
```python
# OLD: Check plan limits
if account.plan.max_keywords and keywords_count > account.plan.max_keywords:
    raise ValidationError("Exceeds plan limit")
```

**After (Credit Only)**:
```python
# NEW: Check credits only
credit_service.check_credits(account, 'clustering', keyword_count)
```

#### Usage Logging Updates

| Task | File | Current State | Implementation |
|------|------|---------------|----------------|
| **Update Usage Logging** | `business/billing/models.py` | EXISTING | Ensure all operations log credits |

**CreditUsageLog Model**:
```python
# business/billing/models.py
class CreditUsageLog(AccountBaseModel):
    account = models.ForeignKey(Account, on_delete=models.CASCADE)
    operation_type = models.CharField(max_length=50)
    credits_used = models.IntegerField()
    related_object_type = models.CharField(max_length=50, blank=True)
    related_object_id = models.IntegerField(null=True, blank=True)
    metadata = models.JSONField(default=dict)
    created_at = models.DateTimeField(auto_now_add=True)
```

#### Frontend Updates

| Task | File | Current State | Implementation |
|------|------|---------------|----------------|
| **Update Frontend Limits UI** | `frontend/src/pages/Billing/` | EXISTING | Replace limits display with credit display |

**Frontend Changes**:
- Remove plan limit displays
- Show credit balance prominently
- Show credit costs per operation
- Show usage history by operation type

---

## OPERATIONAL LIMITS

### 0.2 Operational Limits (Keep)

**Purpose**: Technical constraints, not business limits.

| Limit | Value | Location | Implementation | Reason |
|-------|-------|----------|----------------|--------|
| **Keywords per request** | 50 | `modules/planner/views.py` | Request validation | API payload size, processing time |
| **Images per request** | 6 | `modules/writer/views.py` | Request validation | Queue management (user sees as batch) |
| **Images per AI call** | 1 | `infrastructure/ai/functions/generate_images.py` | Internal | Image API limitation |

**Note**: These are **NOT** business limits - they're technical constraints for request processing.

---

## DATABASE MIGRATIONS

### 0.3 Database Migrations

| Migration | Purpose | Risk | Implementation |
|-----------|---------|------|----------------|
| **Remove limit fields from Plan** | Clean up unused fields | LOW - Add defaults first | Create migration to remove fields |
| **Add credit cost tracking** | Enhance CreditUsageLog | LOW - Additive only | Add fields to CreditUsageLog |
| **Monthly credit replenishment** | Celery Beat task | LOW - New feature | Add scheduled task |

**Migration 1: Remove Plan Limit Fields**:
```python
# core/auth/migrations/XXXX_remove_plan_limits.py
class Migration(migrations.Migration):
    operations = [
        migrations.RemoveField(model_name='plan', name='max_keywords'),
        migrations.RemoveField(model_name='plan', name='max_clusters'),
        # ... remove all limit fields
    ]
```

**Migration 2: Add Credit Cost Tracking**:
```python
# business/billing/migrations/XXXX_add_credit_tracking.py
class Migration(migrations.Migration):
    operations = [
        migrations.AddField(
            model_name='creditusagelog',
            name='related_object_type',
            field=models.CharField(max_length=50, blank=True),
        ),
        migrations.AddField(
            model_name='creditusagelog',
            name='related_object_id',
            field=models.IntegerField(null=True, blank=True),
        ),
        migrations.AddField(
            model_name='creditusagelog',
            name='metadata',
            field=models.JSONField(default=dict),
        ),
    ]
```

**Migration 3: Monthly Credit Replenishment**:
- Add Celery Beat task (see Automation section)

---

## TESTING & VALIDATION

### 0.4 Testing

**Test Cases**:

1. **Credit System Tests**:
   - ✅ All existing features work with credit checks
   - ✅ Credit deduction happens correctly
   - ✅ Insufficient credits show clear error
   - ✅ Usage logging tracks all operations
   - ✅ Frontend shows credit balance, not limits

2. **Module Settings Tests**:
   - ✅ Disabled modules don't appear in sidebar
   - ✅ Disabled modules don't load routes
   - ✅ Disabled modules return 403/404 appropriately
   - ✅ Module settings persist correctly

3. **Backward Compatibility Tests**:
   - ✅ All existing API endpoints work
   - ✅ All existing workflows function
   - ✅ Frontend continues working
   - ✅ No data loss during migration

**Test Files to Create**:
- `backend/tests/test_credit_system.py`
- `backend/tests/test_module_settings.py`
- `frontend/src/__tests__/ModuleGuard.test.tsx`

---

## IMPLEMENTATION CHECKLIST

### Backend Tasks

- [ ] Create `business/billing/constants.py` with credit costs
- [ ] Update `CreditService` with credit cost methods
- [ ] Update `Plan` model - remove limit fields
- [ ] Create migration to remove plan limit fields
- [ ] Update `AIEngine` to check credits before AI calls
- [ ] Update content generation to check credits
- [ ] Update image generation to check credits
- [ ] Remove all plan limit checks from ViewSets
- [ ] Update `CreditUsageLog` model with tracking fields
- [ ] Create migration for credit tracking
- [ ] Extend `ModuleSettings` model with enabled flags
- [ ] Update module settings API
- [ ] Add monthly credit replenishment Celery Beat task

### Frontend Tasks

- [ ] Implement `frontend/src/pages/Settings/Modules.tsx`
- [ ] Create `frontend/src/config/modules.config.ts`
- [ ] Create `frontend/src/components/common/ModuleGuard.tsx`
- [ ] Update `frontend/src/App.tsx` with conditional route loading
- [ ] Update `frontend/src/layout/AppSidebar.tsx` to filter disabled modules
- [ ] Update `frontend/src/pages/Billing/` to show credits instead of limits
- [ ] Update billing UI to show credit costs per operation

### Testing Tasks

- [ ] Test credit deduction for all operations
- [ ] Test insufficient credits error handling
- [ ] Test module enable/disable functionality
- [ ] Test disabled modules don't load
- [ ] Test backward compatibility
- [ ] Test migration safety

---

## RISK ASSESSMENT

| Risk | Level | Mitigation |
|------|-------|------------|
| **Breaking existing functionality** | MEDIUM | Extensive testing, gradual rollout |
| **Credit calculation errors** | MEDIUM | Unit tests for credit calculations |
| **Migration data loss** | LOW | Backup before migration, test on staging |
| **Frontend breaking changes** | LOW | Backward compatible API changes |

---

## SUCCESS CRITERIA

- ✅ All existing features work with credit checks
- ✅ Credit deduction happens correctly for all operations
- ✅ Insufficient credits show clear error messages
- ✅ Usage logging tracks all operations
- ✅ Frontend shows credit balance, not limits
- ✅ Module settings enable/disable modules correctly
- ✅ Disabled modules don't appear in UI
- ✅ No breaking changes for existing users

---

**END OF PHASE 0 DOCUMENT**