reorg-docs

2026-01-01 05:40:42 +00:00
parent d16e5e1a4b
commit dd63403e94
11 changed files with 0 additions and 411 deletions
--- a/docs/plans/implemented/MASTER-IMPLEMENTATION-PLAN.md
+++ b/docs/plans/implemented/MASTER-IMPLEMENTATION-PLAN.md
@@ -0,0 +1,868 @@
+# IGNY8 Master Implementation Plan
+
+**Created:** December 29, 2025
+**Last Updated:** December 29, 2025
+**Status:** Ready for Implementation
+**Prepared by:** Architecture & Implementation Analysis
+
+---
+
+## Executive Summary
+
+This document provides a comprehensive, actionable implementation plan for all pending IGNY8 fixes and enhancements. It is organized by priority, with clear dependencies, file locations, and implementation details derived from deep codebase analysis.
+
+---
+
+## System Architecture Overview
+
+### Tech Stack
+| Layer | Technology |
+|-------|------------|
+| Backend | Django 4.x + DRF + Celery + Redis |
+| Frontend | React 19 + TypeScript + Vite + Zustand |
+| Database | PostgreSQL 14+ |
+| AI Integration | OpenAI (GPT-4/4o), DALL-E, Runware |
+| Task Queue | Celery + Redis broker |
+
+### Key Architectural Patterns
+- **Multi-tenant isolation** via `AccountBaseModel` with automatic `account` scoping
+- **7-stage automation pipeline** orchestrated by `AutomationService`
+- **Token-based billing** with credit system (`CreditService`, `CreditCostConfig`)
+- **Singleton global settings** (`GlobalIntegrationSettings.pk=1`, `BillingConfiguration.pk=1`)
+
+---
+
+## PHASE 1: Critical Pre-Launch Fixes
+
+### 1.1 Payment & Account System
+
+#### Task 1.1.1: Payment Method Saving Fix
+**Problem:** Individual account payment method not saving properly.
+
+**Root Cause Analysis:**
+- `AccountPaymentMethod` model exists at [backend/igny8_core/business/billing/models.py:661-694](backend/igny8_core/business/billing/models.py#L661-L694)
+- Has `unique_together = [['account', 'display_name']]` constraint
+- Issue likely in the view/serializer that handles create/update
+
+**Files to Modify:**
+1. `backend/igny8_core/modules/billing/views.py` - AccountPaymentMethodViewSet
+2. `backend/igny8_core/modules/billing/serializers.py` - Serializer validation
+3. `frontend/src/pages/Account/PlansAndBillingPage.tsx` - Form submission
+
+**Implementation Steps:**
+1. Audit the `AccountPaymentMethodViewSet` - ensure `perform_create()` sets `account` from `request.account`
+2. Check serializer validation for `display_name` uniqueness per account
+3. Verify frontend API call includes all required fields (type, display_name, is_default)
+4. Add explicit error handling for duplicate payment method names
+5. Test: Create, update, delete payment methods via UI
+
+---
+
+#### Task 1.1.2: Remove Country-Specific Payment Methods
+**Problem:** Payment methods have country-specific configuration that should be simplified to global-only.
+
+**Current State:**
+- `PaymentMethodConfig` model at [models.py:607-658](backend/igny8_core/business/billing/models.py#L607-L658) has `country_code` field
+- Frontend likely queries by country
+
+**Files to Modify:**
+1. `backend/igny8_core/business/billing/models.py` - PaymentMethodConfig
+2. `backend/igny8_core/modules/billing/views.py` - Payment method list endpoint
+3. `frontend/src/services/billing.api.ts` - Remove country filtering
+4. `frontend/src/pages/Account/PlansAndBillingPage.tsx` - Payment UI
+
+**Implementation Steps:**
+1. Update `PaymentMethodConfig` queryset to filter only `is_enabled=True` without country check
+2. Create migration to update existing records (set `country_code='*'` or `''` for global)
+3. Simplify frontend payment method selection to show all global methods
+4. Keep `country_code` field for future use but ignore in current queries
+
+---
+
+#### Task 1.1.3: Account Edit Form Fix
+**Problem:** Payment method and account-specific edit form not updating correctly.
+
+**Files to Investigate:**
+1. `frontend/src/pages/Account/AccountSettingsPage.tsx`
+2. `backend/igny8_core/modules/account/views.py`
+3. `backend/igny8_core/modules/account/serializers.py`
+
+**Implementation Steps:**
+1. Trace form submission from frontend to backend
+2. Check serializer `update()` method handles partial updates correctly
+3. Verify optimistic UI update in Zustand store after successful save
+4. Add error boundary for failed saves with clear user feedback
+
+---
+
+### 1.2 Backend Issues
+
+#### Task 1.2.1: Django Admin Keywords 500 Error
+**Problem:** Backend Django admin keywords page returns 500 error.
+
+**Files to Investigate:**
+1. `backend/igny8_core/modules/planner/admin.py` - KeywordsAdmin
+2. `backend/igny8_core/modules/planner/models.py` - Keywords model
+
+**Likely Causes:**
+- N+1 query on related fields (cluster, site, account)
+- Missing `list_select_related` or `list_prefetch_related`
+- Deleted FK reference
+
+**Implementation Steps:**
+1. Check Django logs for exact traceback
+2. Add to KeywordsAdmin:
+   ```python
+   list_select_related = ['site', 'account', 'cluster', 'seed_keyword']
+   list_display = ['keyword', 'site', 'status', ...]  # Only valid fields
+   ```
+3. Ensure all FK fields in `list_display` have `on_delete=SET_NULL` or exist
+4. Add `@admin.display(description='...')` for computed fields
+5. Test admin pagination with 1000+ keywords
+
+---
+
+#### Task 1.2.2: Delete Functions & Cascade Relationships
+**Problem:** Many pages delete function not working (images, image prompts, etc.), and when upstream records are deleted, downstream status and relationships are not properly updated.
+
+**Current State:**
+- Models use soft delete via `SoftDeleteModel` mixin
+- `deleted` field marks as soft-deleted
+- Frontend calls `DELETE /api/v1/writer/images/{id}/`
+- **Cascade issue:** When parent records deleted, child records retain stale FK references and incorrect status
+
+**Files to Investigate:**
+1. `backend/igny8_core/modules/writer/views.py` - ImagesViewSet, ImagePromptsViewSet
+2. `backend/igny8_core/modules/planner/models.py`
+3. `backend/igny8_core/modules/writer/models.py`
+4. `frontend/src/services/api.ts` - Delete API calls
+5. `frontend/src/pages/Writer/Images.tsx` - Delete handlers
+
+**Cascade Relationships to Fix:**
+```
+Cluster (delete) -> Keywords.cluster = NULL (SET_NULL) + Keywords.status = 'new'
+                 -> ContentIdeas.cluster = NULL + Ideas.status = 'new'
+
+ContentIdea (delete) -> Tasks.idea = NULL (SET_NULL) + handle orphan tasks
+
+Task (delete) -> Content.task = NULL (SET_NULL)
+
+Content (delete) -> Images (CASCADE - soft delete)
+                 -> ImagePrompts (CASCADE - soft delete)
+                 -> PublishingRecord (SET_NULL)
+```
+
+**Implementation Steps:**
+1. Verify ViewSet has `destroy()` method or inherits from `DestroyModelMixin`
+2. Check permission classes allow delete for current user role
+3. **Add cascade status updates** - when parent deleted:
+   - Create `pre_delete` or `post_delete` signal handlers
+   - Reset child record status to appropriate value (e.g., `status='new'` for orphaned keywords)
+   - Clear FK references properly
+4. Add bulk delete endpoint if needed: `POST /api/v1/writer/images/bulk-delete/`
+5. Test: Single delete, bulk delete, cascade delete with status verification
+6. Ensure UI shows warning before deleting parent records with dependencies
+
+---
+
+#### Task 1.2.3: Soft Deletion Verification
+**Problem:** Need to verify soft deletion criteria across system.
+
+**Current Pattern:**
+- `SoftDeleteModel` base class with `deleted` BooleanField
+- Managers filter `deleted=False` by default
+- Cascade handled manually or via Django signals
+
+**Files to Audit:**
+1. `backend/igny8_core/common/models.py` - Base soft delete mixin
+2. All models with soft delete: `Content`, `Images`, `Tasks`, `Keywords`, `Clusters`
+
+**Implementation Steps:**
+1. Create comprehensive soft delete audit:
+   | Model | Has SoftDelete | Manager Filters | Cascade Rule | Status Reset |
+   |-------|----------------|-----------------|--------------|--------------|
+   | Content | Yes | Yes | Images soft-deleted | N/A |
+   | Tasks | Yes | Yes | None | N/A |
+   | Images | Yes | Yes | ImagePrompts soft-deleted | N/A |
+   | Clusters | Yes | Yes | Keywords.cluster=NULL | Keywords.status='new' |
+   | ContentIdeas | Yes | Yes | Tasks.idea=NULL | Tasks orphan handling |
+2. Verify `on_delete` behavior for all FKs
+3. Add `pre_delete` signal to handle cascade soft deletes AND status resets
+4. Create admin action to purge soft-deleted records older than retention period
+
+---
+
+### 1.3 Data Integrity
+
+#### Task 1.3.1: CRUD Verification
+**Problem:** Need clear definition and verification of CRUD operations on each page.
+
+**Create CRUD Matrix:**
+
+| Page | Create | Read | Update | Delete | Notes |
+|------|--------|------|--------|--------|-------|
+| Planner/Keywords | Import | List/Detail | Status | Bulk | Import via CSV |
+| Planner/Clusters | AI Gen | List/Detail | Name, Status | Single | Created by AI |
+| Planner/Ideas | AI Gen | List/Detail | Title, Status | Single | Created by AI |
+| Writer/Tasks | From Ideas | List/Detail | All fields | Single | Manual + Auto |
+| Writer/Content | AI Gen | List/Detail | All fields | Single | With Images cascade |
+| Writer/Images | AI Gen | List/Detail | Alt, Caption | Single/Bulk | Upload + Generate |
+
+**Implementation Steps:**
+1. Create this matrix for all pages
+2. Verify each operation has working API endpoint
+3. Add frontend tests for each CRUD action
+4. Document any special behaviors (bulk operations, cascade effects)
+
+---
+
+## PHASE 2: Automation Pipeline Fixes
+
+### 2.1 Stage Card & Metrics Issues
+
+#### Task 2.1.1: Automation Credit Display & Accuracy Fix
+**Problem:** Credit display in stage cards shows incorrect values and doesn't match `/account/usage/credits` endpoint.
+
+**Root Cause Analysis:**
+- `StageCard.tsx` at [frontend/src/components/Automation/StageCard.tsx](frontend/src/components/Automation/StageCard.tsx) shows `result.credits_used`
+- `AutomationRun.stage_X_result` JSON contains `credits_used` field
+- Issue: `_get_credits_used()` in `automation_service.py` counts AITaskLog records, not actual credits
+
+**Current Implementation (Incorrect):**
+```python
+# automation_service.py:1645-1656
+def _get_credits_used(self) -> int:
+    total = AITaskLog.objects.filter(
+        account=self.account,
+        created_at__gte=self.run.started_at
+    ).aggregate(total=Count('id'))['total'] or 0  # WRONG: counts records, not credits
+    return total
+```
+
+**Reference:** `/account/usage/credits` shows accurate data - uses `CreditUsageLog`
+
+**Files to Modify:**
+1. `backend/igny8_core/business/automation/services/automation_service.py` - `_get_credits_used()`
+2. `backend/igny8_core/business/billing/models.py` - Query `CreditUsageLog`
+
+**Implementation Steps:**
+1. Fix `_get_credits_used()` to use same source as `/account/usage/credits`:
+   ```python
+   def _get_credits_used(self) -> int:
+       from igny8_core.business.billing.models import CreditUsageLog
+       from django.db.models import Sum
+       total = CreditUsageLog.objects.filter(
+           account=self.account,
+           created_at__gte=self.run.started_at
+       ).aggregate(total=Sum('credits_used'))['total'] or 0
+       return total
+   ```
+2. Track credits in `AutomationRun.total_credits_used` incrementally
+3. Add stage-specific credit tracking in stage results
+4. Verify credits displayed match `/account/usage/credits` endpoint after fix
+
+---
+
+#### Task 2.1.2: Stage 6 Image Generation & Progress Bar Fix
+**Problem:**
+1. Image generation (Stage 6) behaves differently than other AI functions
+2. Stage 6 progress bar showing wrong counts (always 0/remaining, 0%)
+
+**Current Implementation Analysis:**
+Stage 6 in [automation_service.py:1204-1416](backend/igny8_core/business/automation/services/automation_service.py#L1204-L1416):
+- Uses `process_image_generation_queue.delay()` - Celery async task
+- Other stages use `AIEngine.execute()` synchronously
+- `_wait_for_task()` polls Celery result
+
+**Key Difference:**
+- Stages 1-5: `AIEngine.execute()` -> synchronous, returns immediately
+- Stage 6: `process_image_generation_queue.delay()` -> Celery task, needs polling
+
+**Progress Bar Issue:**
+- `stage_6_result` may not have `images_total` field at stage start
+- Incremental saves happen but `images_total` not set initially
+- Frontend `getProcessedFromResult()` can't calculate progress without total
+
+**Files to Modify:**
+1. `backend/igny8_core/ai/tasks.py` - `process_image_generation_queue`
+2. `backend/igny8_core/ai/engine.py` - AIEngine vs Celery path
+3. `backend/igny8_core/business/automation/services/automation_service.py` - Stage 6 initial save
+
+**Implementation Steps:**
+1. Audit `process_image_generation_queue` task for:
+   - Proper credit deduction BEFORE generation (image credits are pre-paid)
+   - Error handling and retry logic
+   - Status update on Image model
+2. **Fix Stage 6 initial save** to include total immediately:
+   ```python
+   # In run_stage_6() - set total at start
+   self.run.stage_6_result = {
+       'images_processed': 0,
+       'images_total': total_images,  # SET IMMEDIATELY
+       'images_generated': 0,
+       'credits_used': 0,
+       'time_elapsed': '0m 0s',
+       'in_progress': True
+   }
+   self.run.save(update_fields=['stage_6_result'])
+   ```
+3. Consider refactoring Stage 6 to use same `AIEngine` pattern for consistency:
+   ```python
+   engine = AIEngine(account=self.account)
+   result = engine.execute(
+       fn=GenerateImagesFunction(),
+       payload={'ids': [image.id]}
+   )
+   ```
+4. If keeping async, ensure proper progress tracking updates stage result after each image
+
+---
+
+#### Task 2.1.3: Main Progress Bar Fix
+**Problem:** Main progress bar completes at 100% at stage 5 instead of stage 6.
+
+**Root Cause Analysis:**
+- `GlobalProgressBar.tsx` likely calculates: `current_stage / 7 * 100`
+- But Stage 7 is "Review Gate" (no processing) - should complete at Stage 6
+
+**Current Implementation in GlobalProgressBar:**
+- Uses `globalProgress.overall_percentage` from API
+- Or calculates from `stages` array
+
+**Files to Modify:**
+1. `frontend/src/components/Automation/GlobalProgressBar.tsx`
+2. `backend/igny8_core/business/automation/views.py` - `/run_progress/` endpoint
+
+**Implementation Steps:**
+1. Backend calculation should be:
+   ```python
+   # In get_run_progress() view
+   if run.status == 'completed':
+       overall_percentage = 100
+   else:
+       # Weight stages 1-6 equally, stage 7 is 0%
+       completed_stages = min(run.current_stage - 1, 6)
+       stage_weight = 100 / 6
+       within_stage_progress = # based on processed/total
+       overall_percentage = (completed_stages * stage_weight) + (within_stage_progress * stage_weight / 100)
+   ```
+2. Stage 6 completion should trigger 100%
+3. Stage 7 is display-only, no automation work
+
+---
+
+## PHASE 3: AI Provider Configuration
+
+### Task 3.1: Flexible Model Configuration System
+
+**Reference:** [flexible-model-configuration-plan.md](flexible-model-configuration-plan.md)
+
+**Current State:**
+- `MODEL_RATES` and `IMAGE_MODEL_RATES` hardcoded in [ai/constants.py](backend/igny8_core/ai/constants.py)
+- `AIModelConfig` model already exists at [billing/models.py:697-930](backend/igny8_core/business/billing/models.py#L697-L930)
+
+**Implementation Phases:**
+
+#### Phase 3.1.1: Model Registry Service
+**Files to Create:**
+1. `backend/igny8_core/ai/model_registry.py`
+
+**Implementation:**
+```python
+class ModelRegistry:
+    """Central registry for AI model configurations with caching."""
+
+    _cache = {}
+    _cache_ttl = 300
+
+    @classmethod
+    def get_model(cls, model_id: str) -> Optional[AIModelConfig]:
+        # Check cache, then DB, then fallback to constants.py
+        pass
+
+    @classmethod
+    def get_rate(cls, model_id: str, rate_type: str) -> Decimal:
+        # Get input/output rate for text or per-image for image
+        pass
+
+    @classmethod
+    def calculate_cost(cls, model_id: str, input_tokens: int, output_tokens: int) -> Decimal:
+        model = cls.get_model(model_id)
+        if model.model_type == 'text':
+            return model.get_cost_for_tokens(input_tokens, output_tokens)
+        return Decimal('0')
+```
+
+#### Phase 3.1.2: Update AICore to Use Registry
+**Files to Modify:**
+1. `backend/igny8_core/ai/ai_core.py` - Replace `MODEL_RATES` lookup
+
+**Before:**
+```python
+from igny8_core.ai.constants import MODEL_RATES
+rate = MODEL_RATES.get(model, {})
+```
+
+**After:**
+```python
+from igny8_core.ai.model_registry import ModelRegistry
+model_config = ModelRegistry.get_model(model)
+```
+
+#### Phase 3.1.3: Data Migration
+Create migration to seed `AIModelConfig` from current `constants.py`:
+
+```python
+# Migration: 0002_seed_ai_models.py
+def seed_models(apps, schema_editor):
+    AIModelConfig = apps.get_model('billing', 'AIModelConfig')
+
+    text_models = [
+        ('gpt-4o-mini', 'GPT-4o Mini', 'openai', 0.15, 0.60, True),
+        ('gpt-4o', 'GPT-4o', 'openai', 2.50, 10.00, False),
+        # ... all models from constants.py
+    ]
+
+    for model_id, name, provider, input_cost, output_cost, is_default in text_models:
+        AIModelConfig.objects.create(
+            model_name=model_id,
+            display_name=name,
+            provider=provider,
+            model_type='text',
+            input_cost_per_1m=input_cost,
+            output_cost_per_1m=output_cost,
+            is_active=True,
+            is_default=is_default,
+        )
+```
+
+#### Phase 3.1.4: Admin UI for Models
+**Files to Create/Modify:**
+1. `backend/igny8_core/modules/billing/admin.py` - AIModelConfigAdmin
+2. `frontend/src/pages/Admin/AISettings.tsx` - Model management UI
+
+---
+
+### Task 3.2: Bria Integration (Image Generation)
+
+**Current State:** Runware is already integrated and working with 1 model configured. Need to add Bria as additional image generation provider.
+
+**Files to Modify:**
+1. `backend/igny8_core/modules/system/global_settings_models.py` - Add Bria API key field
+2. `backend/igny8_core/ai/image_service.py` - Add Bria provider alongside Runware
+3. Seed `AIModelConfig` with Bria models
+
+**Implementation Steps:**
+1. Add to `GlobalIntegrationSettings`:
+   ```python
+   bria_api_key = models.CharField(max_length=255, blank=True)
+   bria_default_model = models.CharField(max_length=100, default='bria-2.3')
+   ```
+2. Create `BriaImageService` class mirroring existing Runware pattern
+3. Add Bria models to `AIModelConfig`:
+   ```python
+   ('bria-2.3', 'Bria 2.3', 'bria', 'image', cost_per_image=0.015)
+   ('bria-2.3-fast', 'Bria 2.3 Fast', 'bria', 'image', cost_per_image=0.010)
+   ```
+4. Update `process_image_generation_queue` to support provider selection (openai/runware/bria)
+5. Add UI dropdown in admin settings for selecting default image provider
+
+---
+
+### Task 3.3: Anthropic Integration (Text Generation)
+
+**Files to Modify:**
+1. `backend/igny8_core/modules/system/global_settings_models.py` - Add Anthropic API key
+2. `backend/igny8_core/ai/ai_core.py` - Add Anthropic client
+3. Seed `AIModelConfig` with Claude models
+
+**Implementation Steps:**
+1. Add to `GlobalIntegrationSettings`:
+   ```python
+   anthropic_api_key = models.CharField(max_length=255, blank=True)
+   anthropic_default_model = models.CharField(max_length=100, default='claude-3-sonnet')
+   ```
+2. Install `anthropic` Python package
+3. Add Anthropic client initialization in `AICore`:
+   ```python
+   from anthropic import Anthropic
+
+   if provider == 'anthropic':
+       client = Anthropic(api_key=settings.anthropic_api_key)
+       response = client.messages.create(model=model, messages=messages, ...)
+   ```
+4. Add Claude models to `AIModelConfig`
+
+---
+
+## PHASE 4: Design System & UI Standardization
+
+### 4.1 Sidebar & Navigation Fixes
+
+**Problems Identified:**
+- Sidebar padding too small
+- Icon sizes too small
+- Padding between dropdown menu items insufficient
+
+**Files to Modify:**
+1. `frontend/src/components/Layout/Sidebar.tsx` or equivalent
+2. `frontend/src/components/ui/navigation/*`
+3. `frontend/src/index.css` or Tailwind config
+
+**Implementation Steps:**
+1. Increase sidebar padding: `p-2` -> `p-3` or `p-4`
+2. Increase icon sizes: `size-4` -> `size-5` or `size-6`
+3. Increase dropdown menu item spacing: `py-1` -> `py-2`
+4. Apply consistent hover states and active indicators
+
+---
+
+### 4.2 Footer Widget Metrics Fix
+
+**Problem:** In Planner and Writer submodule pages, the workflow completion metrics (footer widgets) are correct on some pages but empty or missing on others.
+
+**Files to Investigate:**
+1. `frontend/src/components/common/FooterWidgets.tsx` or similar
+2. `frontend/src/pages/Planner/*.tsx` - All planner pages
+3. `frontend/src/pages/Writer/*.tsx` - All writer pages
+
+**Implementation Steps:**
+1. Audit all pages with footer widgets:
+   | Page | Footer Widget Present | Data Correct |
+   |------|----------------------|--------------|
+   | Planner/Keywords | ? | ? |
+   | Planner/Clusters | ? | ? |
+   | Planner/Ideas | ? | ? |
+   | Writer/Tasks | ? | ? |
+   | Writer/Content | ? | ? |
+   | Writer/Images | ? | ? |
+2. Ensure consistent data fetching for workflow metrics
+3. Add fallback values for empty data
+4. Verify API calls return correct counts
+
+---
+
+### 4.3 Header Metrics Verification
+
+**Problem:** Header metrics in table action rows show incorrect values on some pages (e.g., Volume on Clusters page shows 0 when it should aggregate keyword volumes).
+
+**Example Issue (from screenshot):**
+- Clusters page shows `Volume: 0`
+- Should show sum of `volume` from all keywords assigned to clusters
+
+**Files to Investigate:**
+1. `frontend/src/pages/Planner/Clusters.tsx`
+2. `frontend/src/config/pages/clusters.config.ts`
+3. `backend/igny8_core/modules/planner/views.py` - ClustersViewSet
+
+**Pages to Audit:**
+
+| Page | Metric | Expected Calculation | Current Status |
+|------|--------|---------------------|----------------|
+| Clusters | Volume | SUM(keywords.volume) for all clusters | Shows 0 - BROKEN |
+| Clusters | Keywords | COUNT(keywords) for all clusters | Verify |
+| Keywords | Volume | SUM(volume) for displayed keywords | Verify |
+| Ideas | Word Count | SUM(estimated_word_count) | Verify |
+| Content | Word Count | SUM(word_count) | Verify |
+| Images | Total | COUNT(images) | Verify |
+
+**Implementation Steps:**
+1. For Clusters Volume fix:
+   - Backend: Add `total_volume` annotation to ClustersViewSet list endpoint
+   - Or: Create dedicated metrics endpoint `/api/v1/planner/clusters/metrics/`
+   - Frontend: Fetch and display aggregated volume
+2. Audit all other header metrics
+3. Create consistent pattern for metrics calculation (backend aggregation vs frontend sum)
+4. Add loading states for metrics while data fetches
+
+---
+
+### 4.4 Typography & Spacing Standardization
+
+**Problem:** Font sizes, spacing, and element sizes are inconsistent across components.
+
+**Objective:** Create single source of truth for all typography and spacing.
+
+**Design Tokens to Standardize:**
+
+```javascript
+// tailwind.config.js additions
+module.exports = {
+  theme: {
+    extend: {
+      fontSize: {
+        'heading-1': ['2rem', { lineHeight: '2.5rem', fontWeight: '700' }],
+        'heading-2': ['1.5rem', { lineHeight: '2rem', fontWeight: '600' }],
+        'heading-3': ['1.25rem', { lineHeight: '1.75rem', fontWeight: '600' }],
+        'body-lg': ['1rem', { lineHeight: '1.5rem' }],
+        'body-md': ['0.875rem', { lineHeight: '1.25rem' }],
+        'body-sm': ['0.75rem', { lineHeight: '1rem' }],
+        'label': ['0.75rem', { lineHeight: '1rem', fontWeight: '500' }],
+      },
+      spacing: {
+        'card-padding': '1.5rem',
+        'section-gap': '2rem',
+        'input-padding': '0.75rem',
+      },
+      // Icon sizes
+      iconSize: {
+        'xs': '0.875rem',  // 14px
+        'sm': '1rem',       // 16px
+        'md': '1.25rem',    // 20px
+        'lg': '1.5rem',     // 24px
+        'xl': '2rem',       // 32px
+      }
+    }
+  }
+}
+```
+
+**Implementation Steps:**
+1. Create typography scale document
+2. Audit current usage across all components
+3. Replace hardcoded values with design tokens
+4. Update all:
+   - Page headings
+   - Section titles
+   - Card titles
+   - Table headers/cells
+   - Form labels
+   - Button text
+   - Icon sizes
+
+---
+
+### 4.5 Color Scheme Consistency & Balance
+
+**Problem:**
+1. Module colors not consistent across the system (e.g., Keywords module icon color differs from its progress bars and references)
+2. Multi-color usage is unbalanced (same color repeated in sequence, inconsistent patterns)
+
+**Color Assignments (Single Source of Truth):**
+
+| Module/Element | Primary Color | Use Cases |
+|----------------|---------------|-----------|
+| Keywords | `brand-500` (blue) | Icon, progress bar, metric cards, badges |
+| Clusters | `purple-500` | Icon, progress bar, metric cards, badges |
+| Ideas | `purple-600` | Icon, progress bar, metric cards, badges |
+| Content | `success-500` (green) | Icon, progress bar, metric cards, badges |
+| Tasks | `success-600` | Icon, progress bar, metric cards, badges |
+| Images | `purple-500` | Icon, progress bar, metric cards, badges |
+| Automation | `brand-500` | Pipeline stage cards |
+| Billing | `warning-500` (amber) | Credit displays, warnings |
+
+**Multi-Color Balance Rules:**
+1. When displaying 5+ colored elements, ensure:
+   - No more than 2 consecutive items share the same color
+   - Colors distributed evenly across the spectrum
+   - Visual hierarchy maintained (primary actions in brand color)
+
+2. Dashboard metric cards should follow pattern:
+   ```
+   [blue] [purple] [green] [amber] [purple]
+   ```
+   NOT:
+   ```
+   [blue] [blue] [purple] [purple] [purple]  // Unbalanced
+   ```
+
+**Files to Audit & Fix:**
+1. `frontend/src/pages/Dashboard/Home.tsx`
+2. `frontend/src/pages/Automation/AutomationPage.tsx`
+3. `frontend/src/pages/Planner/*.tsx`
+4. `frontend/src/pages/Writer/*.tsx`
+5. `frontend/src/pages/Account/*.tsx`
+6. All metric card and progress bar components
+
+**Implementation Steps:**
+1. Create color mapping constant:
+   ```typescript
+   // src/config/colors.config.ts
+   export const MODULE_COLORS = {
+     keywords: { bg: 'bg-brand-500', text: 'text-brand-600', border: 'border-brand-500' },
+     clusters: { bg: 'bg-purple-500', text: 'text-purple-600', border: 'border-purple-500' },
+     ideas: { bg: 'bg-purple-600', text: 'text-purple-700', border: 'border-purple-600' },
+     content: { bg: 'bg-success-500', text: 'text-success-600', border: 'border-success-500' },
+     images: { bg: 'bg-purple-500', text: 'text-purple-600', border: 'border-purple-500' },
+   };
+   ```
+2. Replace all hardcoded colors with config references
+3. Audit each dashboard/page for color balance
+4. Fix sequential same-color issues
+5. Add visual testing for color consistency
+
+---
+
+### 4.6 Component Audit & Duplicate Removal
+
+**Objective:** Inventory all UI components and remove duplicate/parallel design systems.
+
+**Audit Checklist:**
+- [ ] Button variants (primary, secondary, outline, ghost)
+- [ ] Card components
+- [ ] Form inputs (text, select, checkbox, radio)
+- [ ] Table components
+- [ ] Modal/dialog
+- [ ] Navigation components
+- [ ] Icon usage (Lucide vs custom)
+- [ ] Metric cards
+- [ ] Progress bars
+
+**Current Systems to Consolidate:**
+1. Tailwind CSS 4.0 classes (KEEP - primary)
+2. Custom CSS files (AUDIT - keep only for complex animations)
+3. Inline styles (REMOVE)
+4. CSS-in-JS (REMOVE if present)
+
+**Implementation Steps:**
+1. Create component inventory spreadsheet
+2. Identify all duplicate components (e.g., multiple Button implementations)
+3. Choose canonical version for each component type
+4. Replace all uses of deprecated versions
+5. Delete deprecated files
+6. Add lint rules to prevent future duplicates
+
+---
+
+### 4.7 Design System Verification
+
+**Objective:** Ensure only standard styles remain and prevent regression.
+
+**Verification Steps:**
+1. Visual regression testing with screenshots
+2. Component documentation/storybook review
+3. Cross-browser testing (Chrome, Firefox, Safari)
+4. Mobile responsive testing
+5. Dark mode consistency check
+
+**Success Criteria:**
+- All pages use same typography scale
+- All modules use assigned colors consistently
+- No inline styles in codebase
+- No duplicate component files
+- Sidebar/navigation properly spaced
+- Header metrics accurate on all pages
+- Footer widgets present and correct on all subpages
+
+---
+
+## PHASE 5: Cleanup (From TODOS.md)
+
+### Task 5.1: SiteBuilder Removal
+
+**Status:** Deprecated module to be removed
+
+**Search Patterns:**
+- `sitebuilder`, `site_builder`, `SiteBuilder`
+- Components/pages with "SiteBuilder" in name
+- API endpoints containing `sitebuilder`
+
+**Implementation Steps:**
+1. Search codebase for all references
+2. Remove unused imports
+3. Delete SiteBuilder-specific files
+4. Update routes/navigation
+5. Remove database tables if empty
+
+---
+
+### Task 5.2: Inactive Modules Documentation
+
+**Modules on hold for Phase 2:**
+- **Linker** - Internal linking suggestions
+- **Optimizer** - Content optimization
+
+**Implementation Steps:**
+1. Ensure modules are disabled in routes
+2. Add feature flags to prevent accidental activation
+3. Document Phase 2 activation requirements
+
+---
+
+## Implementation Priority & Dependencies
+
+### Priority Matrix
+
+| Phase | Priority | Effort | Dependencies |
+|-------|----------|--------|--------------|
+| 1.1 Payment System | CRITICAL | Medium | None |
+| 1.2 Backend Issues | CRITICAL | Medium | None |
+| 1.3 Data Integrity | HIGH | Low | 1.2 |
+| 2.1 Automation Fixes | CRITICAL | Medium | None |
+| 3 AI Providers | MEDIUM | High | None |
+| 4 Design System | HIGH | High | None |
+| 5 Cleanup | LOW | Low | None |
+
+### Recommended Execution Order
+
+```
+Week 1: Phase 1.1 (Payment) + Phase 1.2 (Backend + Cascade)
+Week 2: Phase 1.3 (CRUD) + Phase 2 (Automation Fixes)
+Week 3: Phase 3 (AI Providers)
+Week 4: Phase 4.1-4.3 (Sidebar, Footer, Header Metrics)
+Week 5: Phase 4.4-4.6 (Typography, Colors, Components)
+Week 6: Phase 4.7 (Verification) + Phase 5 (Cleanup)
+```
+
+---
+
+## Testing Strategy
+
+### Unit Tests
+- All service methods
+- Model validations
+- Credit calculations
+
+### Integration Tests
+- API endpoint flows
+- Automation pipeline stages
+- Payment webhooks
+
+### E2E Tests
+- Complete automation run
+- User payment flow
+- Content publishing
+
+### Visual Tests
+- Typography consistency
+- Color scheme accuracy
+- Responsive layouts
+- Dark mode
+
+---
+
+## Monitoring & Verification
+
+### Success Metrics
+
+| Metric | Target | Measurement |
+|--------|--------|-------------|
+| Payment success rate | >99% | Payment.status counts |
+| Automation completion | >95% | AutomationRun.status |
+| Credit accuracy | 100% | Manual audit |
+| Stage progress accuracy | 100% | UI vs DB comparison |
+| Header metrics accuracy | 100% | Visual verification |
+| Color consistency | 100% | Design audit |
+
+### Logging Requirements
+
+All critical operations should log:
+- Timestamp
+- Account ID
+- Operation type
+- Input parameters
+- Result/Error
+- Duration
+
+---
+
+## Document History
+
+| Date | Version | Author | Changes |
+|------|---------|--------|---------|
+| 2025-12-29 | 1.0 | Architecture Review | Initial comprehensive plan |
+| 2025-12-29 | 1.1 | User Feedback | Merged tasks, added UI/styling phase, fixed task descriptions |
+
+---
+
+*This plan is designed to be executable step-by-step. Each task includes specific file locations, implementation details, and verification criteria derived from actual codebase analysis.*