Files

IGNY8 VPS (Salman) 879ef6ff06 fixes related to automation and celery schedules

2026-01-18 12:22:27 +00:00

58 KiB

Raw Blame History

Settings & AI Configuration Consolidation Plan

Created: January 17, 2026
Updated: January 17, 2026
Status: FINALIZED - READY FOR IMPLEMENTATION
Priority: 🟡 HIGH - Improve UX & Reduce Confusion
Depends On: None (standalone improvement)

📋 SINGLE PLAN FILE - This document covers both backend AND frontend implementation.

Executive Summary

The IGNY8 codebase has 40+ scattered settings across 17 different models with significant redundancy. This plan consolidates them into a clean, hierarchical structure with a unified frontend experience.

Key Decisions (Finalized)

AI Models: Simple Testing vs Live toggle (no quality tiers)
Credit Budget: Enabled by default, uses all available credits
Per-Run Limits: Apply to INPUT items (keywords, clusters, etc.)
SiteIntegration Model: To be REMOVED (redundant with Site.wp_* fields)
Terminology: Use "Site" not "WordPress" throughout

Part 1: Current State Analysis

1.1 Settings Model Inventory

Scope	Model	Location	Purpose	Status
GLOBAL	`IntegrationProvider`	modules/system/models.py	API keys - ONE record per provider (openai, runware, stripe, paypal, resend)	✅ CANONICAL (SINGLE)
	`SystemAISettings`	modules/system/ai_settings.py	AI parameter defaults (temp, tokens, image style)	✅ Active
	`GlobalIntegrationSettings`	modules/system/models.py	Legacy API keys + AI defaults	⚠️ DEPRECATED
	`AIModelConfig`	business/billing/models.py	Model definitions + credit pricing	✅ Active
	`CreditCostConfig`	business/billing/models.py	Fixed credits per operation	✅ Active
	`BillingConfiguration`	business/billing/models.py	Token-credit ratios	✅ Active
	`GlobalModuleSettings`	modules/system/models.py	Platform module toggles	✅ Active
	`CreditPackage`	business/billing/models.py	Credit purchase bundles	✅ Active
	`PaymentMethodConfig`	business/billing/models.py	Payment methods per country	✅ Active
ACCOUNT	`AccountSettings`	modules/system/settings_models.py	Key-value store - MULTIPLE records per account (ai.temperature, ai.max_tokens, etc.)	✅ CANONICAL
	`ModuleEnableSettings`	modules/system/settings_models.py	Account module toggles	✅ Active
	`AISettings`	modules/system/settings_models.py	Integration configs per account (overlaps AccountSettings)	⚠️ TO REMOVE - use AccountSettings instead
	`IntegrationSettings`	business/integration/models.py	Account integration overrides	✅ Active
	`AccountPaymentMethod`	business/billing/models.py	Saved payment methods	✅ Active
SITE	`AutomationConfig`	business/automation/models.py	Automation stages, limits	✅ Active
	`PublishingSettings`	business/integration/models.py	Publishing schedule	✅ Active
	`SiteAIBudgetAllocation`	business/billing/models.py	Credit % per AI function	✅ Active
	`SiteIntegration`	business/integration/models.py	Site platform credentials	⚠️ TO BE REMOVED - Redundant with Site.wp_* fields
USER	`UserSettings`	modules/system/settings_models.py	User preferences	✅ Active

1.2 Identified Problems

Problem 1: API Keys in Multiple Places

❌ GlobalIntegrationSettings.openai_api_key
❌ GlobalIntegrationSettings.anthropic_api_key
❌ GlobalIntegrationSettings.runware_api_key
✅ IntegrationProvider.encrypted_credentials (CANONICAL)

Fix: Remove API key fields from GlobalIntegrationSettings

Problem 2: AI Parameters Fragmented

SystemAISettings.temperature ← Global default
GlobalIntegrationSettings.openai_temperature ← Duplicate
AccountSettings[ai.temperature] ← Account override
IntegrationSettings.config.temperature ← Another override

Fix: Keep only SystemAISettings + AccountSettings override

Problem 3: Model Selection Confusion

AIModelConfig.is_default = True ← Should be canonical
GlobalIntegrationSettings.default_text_model ← Duplicate
GlobalIntegrationSettings.default_image_model ← Duplicate

Fix: Use AIModelConfig.is_default only

Problem 4: Frontend "Testing vs Live" Model Not Implemented

Current: Model names exposed directly (GPT-4.1, Claude 3.5, etc.)
Needed: "Testing Model" and "Live Model" abstraction

Fix: Add model_tier field to AIModelConfig

Problem 5: Redundant AISettings Model

AISettings.integration_type + config ← Overlaps with:
- AccountSettings (ai.* keys)
- IntegrationSettings.config

Fix: Remove AISettings model, consolidate into AccountSettings

Part 2: Target Architecture

2.0 Frontend Location: CONFIRMED

Location: Site Settings > AI & Automation Tab

This is the CORRECT location for the unified settings page:

Near existing site configuration
Familiar location for users
Keeps all site-specific settings together
Rename existing "AI Settings" tab to "AI & Automation"

NOT in Setup menu - Setup is for initial configuration, not ongoing settings.

2.1 Django Admin Sidebar - Before & After

BEFORE (Current - From Screenshot)

Accounts & Users           ▼
├── Accounts
├── Users
├── Sites
├── Sectors
└── Site Access

Plans & Billing           ▼
├── Plans
├── Subscriptions
├── Invoices
├── Payments
├── Credit Packages
├── Payment Methods (Global)
└── Account Payment Methods

Credits                   ▼
├── Credit Transactions
├── Credit Usage Logs
├── Credit Cost Config          ← Rarely used
└── Billing Configuration       ← Rarely used

(AI Models scattered elsewhere)

AFTER (Proposed - Reorganized)

Dashboard & Reports        ▼
├── Dashboard
├── Revenue Report
├── Usage Report
├── Content Report
├── AI Cost Analysis              ← Margin/pricing reports here
└── Token Usage

Accounts & Users           ▼
├── Accounts
├── Users
├── Sites
├── Sectors
└── Site Access

Plans & Billing           ▼
├── Plans
├── Subscriptions
├── Invoices
├── Payments
├── Credit Packages
├── Payment Methods (Global)
└── Account Payment Methods

Credits & AI Usage        ▼          ← CONSOLIDATED LOGS
├── Credit Transactions
├── Credit Usage Log
├── AI Task Logs                     ← Moved from AI Configuration
└── Plan Limits

AI Configuration          ▼          ← SIMPLIFIED
├── AI Models (Testing/Live)         ← Main config with COST fields
├── System AI Settings               ← Default parameters
└── Integration Providers            ← API keys (OpenAI, Runware, Stripe, etc.)

Content Pipeline          ▼          ← RENAMED (was Planning + Writing)
├── Keywords
├── Clusters
├── Content Ideas
├── Tasks
├── Content
├── Images
└── Image Prompts

Publishing                ▼
├── Publishing Records
├── Deployments
└── Sync Events
    (Removed: Integrations - SiteIntegration removed)

Automation                ▼          ← NEW SECTION
├── Automation Configs
└── Automation Runs

Global Settings           ▼          ← SIMPLIFIED
├── Global AI Prompts
├── Module Settings
├── Author Profiles
└── Strategies

Account & User Settings   ▼          ← CONSOLIDATED
├── Account Settings                 ← All key-value settings
├── User Settings
└── Module Enable Settings

Resources                 ▼
├── Industries
├── Industry Sectors
└── Seed Keywords

Plugin Management         ▼
├── Plugins
├── Plugin Versions
├── Installations
└── Downloads

Email Settings            ▼
├── Email Configuration
├── Email Templates
└── Email Logs

Logs & Monitoring         ▼
├── System Health
├── API Monitor
├── Debug Console
├── Celery Tasks
└── Admin Log

Django Admin              ▼
├── Groups
├── Permissions
├── Content Types
└── Sessions

Trash                     ▼
├── ... (all trash items)

Key Reorganization Changes

Before	After	Reason
AI Task Logs in "AI Configuration"	Moved to "Credits & AI Usage"	All logs together
Credit Cost Config, Billing Config	REMOVED	Costs in AIModelConfig
Planning + Writing (separate)	"Content Pipeline" (combined)	Logical flow
SiteIntegration in Publishing	REMOVED	Redundant with Site.wp_*
Global Settings (cluttered)	Simplified + separate Automation	Cleaner
System Configuration (scattered)	"Account & User Settings"	Consolidated

2.2 Model Hierarchy (After Consolidation)

GLOBAL (Admin-Only Configuration)
├── IntegrationProvider           # API keys for all providers (OpenAI, Anthropic, Runware)
├── AIModelConfig                 # Model definitions + pricing + is_testing flag
├── SystemAISettings              # Default AI parameters (temp, tokens, image style)
└── GlobalModuleSettings          # Platform module toggles

ACCOUNT (User Account Settings)
├── AccountSettings               # Key-value store for all account config
├── ModuleEnableSettings          # Which modules user can access
└── AccountPaymentMethod          # Saved payment methods

SITE (Per-Site Configuration)
├── SiteAIAutomationSettings      # NEW: Unified automation+AI+publishing
└── Site.wp_url, Site.wp_api_key  # Integration credentials (moved from SiteIntegration)

USER
└── UserSettings                  # User preferences

REMOVED Models:

SiteIntegration → credentials now in Site model
CreditCostConfig → not actively used
BillingConfiguration → hardcode values
GlobalIntegrationSettings → replaced by IntegrationProvider
AISettings (settings_models) → use AccountSettings

2.3 AI Model Designation: Testing vs Live (Simplified)

REMOVED: Standard, Premium, Quality tiers
NEW: Simple boolean is_testing field

Field Value	Frontend Label	Description
`is_testing = True`	Testing	Cheap model for testing (100x cheaper, low quality)
`is_testing = False`	Live	Production model

Constraint: Only ONE model can be is_testing=True per model_type (text/image).

Frontend Column Header: "Testing/Live AI Model"

Dropdown Options:

[Testing ▼]  ← Shows only the testing model
[Live ▼]     ← Shows only the live model

2.3.1 Costing & Pricing - Single Source of Truth

IMPORTANT: After removing CreditCostConfig, ALL costing/pricing lives in AIModelConfig.

Cost vs Price Definitions

Term	Definition	Where Stored
Cost	Our expense per token/image (what we pay to OpenAI, Runware, etc.)	`AIModelConfig.cost_per_1k_input`, `cost_per_1k_output`
Price	What we charge users in credits (revenue calculation)	`AIModelConfig.tokens_per_credit`, `credits_per_image`
Margin	Price - Cost = Revenue margin	Calculated in reports

AIModelConfig Fields for Costing

class AIModelConfig(models.Model):
    # ═══════════════════════════════════════════════════════════
    # COST (Our expense - what we pay providers)
    # ═══════════════════════════════════════════════════════════
    
    # Text models: Cost per 1K tokens (USD)
    cost_per_1k_input = models.DecimalField(...)   # e.g., $0.01 for GPT-4.1-mini input
    cost_per_1k_output = models.DecimalField(...)  # e.g., $0.03 for GPT-4.1-mini output
    
    # Image models: Cost per image (USD) - stored in metadata or calculated
    # e.g., DALL-E 3 = $0.04/image, Runware basic = $0.004/image
    
    # ═══════════════════════════════════════════════════════════
    # PRICE (What we charge users in credits)
    # ═══════════════════════════════════════════════════════════
    
    # Text models: How many tokens = 1 credit
    tokens_per_credit = models.IntegerField(...)   # e.g., 10000 tokens = 1 credit
    
    # Image models: How many credits per image
    credits_per_image = models.IntegerField(...)   # e.g., 1 credit, 5 credits, 15 credits

Usage in Reports

# AI Cost Analysis Report (Backend)
def calculate_margin(usage_log):
    model = usage_log.model_config
    
    # Calculate our cost
    if model.model_type == 'text':
        cost = (usage_log.input_tokens / 1000 * model.cost_per_1k_input +
                usage_log.output_tokens / 1000 * model.cost_per_1k_output)
    else:  # image
        cost = usage_log.images_generated * model.cost_per_image  # from metadata
    
    # Calculate what user paid (credits × credit value)
    credits_used = usage_log.credits_used
    credit_value = 0.01  # $0.01 per credit (configurable)
    revenue = credits_used * credit_value
    
    # Margin
    margin = revenue - cost
    margin_pct = (margin / revenue * 100) if revenue > 0 else 0
    
    return {
        'cost': cost,
        'revenue': revenue,
        'margin': margin,
        'margin_pct': margin_pct,
    }

What Gets Removed

Model	Reason
`CreditCostConfig`	Redundant - costs now in AIModelConfig per model
`BillingConfiguration`	Rarely used - can hardcode credit value ($0.01)

2.3.2 Settings Hierarchy - Where Defaults & Overrides Live

┌─────────────────────────────────────────────────────────────────────────────┐
│ SETTINGS HIERARCHY                                                          │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  GLOBAL DEFAULTS (Admin-only, one record each)                             │
│  ├── IntegrationProvider (one per provider: openai, runware, stripe, etc.) │
│  │   └── API keys, endpoints, sandbox mode                                 │
│  │                                                                          │
│  ├── AIModelConfig (one per model: gpt-4.1, dall-e-3, etc.)               │
│  │   └── Model pricing, costs, capabilities, is_testing flag              │
│  │                                                                          │
│  └── SystemAISettings (SINGLETON - one record total)                       │
│      └── Default temperature, max_tokens, image_style, image_quality       │
│                                                                             │
│  ════════════════════════════════════════════════════════════════════════   │
│                                                                             │
│  ACCOUNT OVERRIDES (Multiple key-value records per account)                │
│  └── AccountSettings (unique_together: account + key)                      │
│      ├── ai.temperature = 0.8        # Override SystemAISettings default   │
│      ├── ai.max_tokens = 4000        # Override SystemAISettings default   │
│      ├── ai.image_style = "vivid"    # Override SystemAISettings default   │
│      └── ... other account-specific settings                               │
│                                                                             │
│  ════════════════════════════════════════════════════════════════════════   │
│                                                                             │
│  SITE CONFIGURATION (One record per site)                                  │
│  └── SiteAIAutomationSettings (NEW unified model)                          │
│      ├── Automation: enabled, frequency, time                              │
│      ├── Stages: JSON config per stage (batch_size, per_run_limit, etc.)  │
│      └── Publishing: days, time_slots                                      │
│                                                                             │
│  ════════════════════════════════════════════════════════════════════════   │
│                                                                             │
│  Resolution Order: Site → Account → Global Default                         │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

2.3.3 AISettings Model - REMOVAL PLAN

Current State: AISettings in settings_models.py stores per-account integration configs.

Problem: Overlaps with AccountSettings which already has ai.* keys.

Migration:

Any data in AISettings.config → Move to AccountSettings keys
AISettings.model_preferences → AccountSettings['ai.model_preferences']
AISettings.cost_limits → Not needed (credit budget in SiteAIAutomationSettings)
Remove AISettings model and admin

After Removal:

Account AI settings use AccountSettings with keys like:
- ai.temperature
- ai.max_tokens
- ai.image_style
- ai.image_quality
- ai.model_preferences

2.4 SiteIntegration Model - REMOVAL PLAN

Status: CONFIRMED REDUNDANT - Safe to remove

Analysis:

What SiteIntegration Stores	Where It Actually Lives
`credentials_json.api_key`	`Site.wp_api_key` (single source of truth)
`config_json.url`	`Site.wp_url` (single source of truth)
`platform`	Always 'wordpress' (hardcoded)
`sync_status`, `last_sync_at`	Can move to `Site` model

Evidence from codebase:

# All services use Site model directly:
api_key = site.wp_api_key  # NOT SiteIntegration.credentials_json
url = site.wp_url          # NOT SiteIntegration.config_json

Removal Steps:

Add sync_status, last_sync_at fields to Site model
Update SyncEvent.integration FK → SyncEvent.site FK
Update sync_health_service.py to use Site directly
Remove SiteIntegration admin registration
Remove SiteIntegration model
Create migration to drop table

2.5 New Unified Model: SiteAIAutomationSettings

Replaces: AutomationConfig + PublishingSettings + SiteAIBudgetAllocation fields

class SiteAIAutomationSettings(AccountBaseModel):
    """
    Unified AI & Automation settings for a site.
    Consolidates AutomationConfig, PublishingSettings, and budget allocation.
    """
    site = models.OneToOneField('Site', on_delete=models.CASCADE)
    
    # ═══════════════════════════════════════════════════════════
    # AUTOMATION SCHEDULING
    # ═══════════════════════════════════════════════════════════
    automation_enabled = models.BooleanField(default=False)
    automation_frequency = models.CharField(max_length=20, default='daily',
        choices=[('hourly', 'Hourly'), ('daily', 'Daily'), ('weekly', 'Weekly')])
    automation_time = models.TimeField(default='02:00')
    
    # ═══════════════════════════════════════════════════════════
    # STAGE CONFIGURATION (JSON for flexibility)
    # ═══════════════════════════════════════════════════════════
    # Structure: {
    #   "1": {"enabled": true, "batch_size": 50, "per_run_limit": 0, "use_testing": false, "budget_pct": 15},
    #   "2": {"enabled": true, "batch_size": 1, "per_run_limit": 10, "use_testing": false, "budget_pct": 10},
    #   ...
    # }
    # NOTE: per_run_limit applies to INPUT items:
    #   Stage 1: max keywords to cluster
    #   Stage 2: max clusters to generate ideas from
    #   Stage 4: max tasks to generate content for
    #   etc.
    stage_config = models.JSONField(default=dict)
    
    # ═══════════════════════════════════════════════════════════
    # DELAYS
    # ═══════════════════════════════════════════════════════════
    within_stage_delay = models.IntegerField(default=3, help_text="Seconds between items")
    between_stage_delay = models.IntegerField(default=5, help_text="Seconds between stages")
    
    # ═══════════════════════════════════════════════════════════
    # PUBLISHING SCHEDULE (Simplified - Time Slots only)
    # ═══════════════════════════════════════════════════════════
    auto_approval_enabled = models.BooleanField(default=True)
    auto_publish_enabled = models.BooleanField(default=True)
    
    # Schedule: Days + Time Slots (NO stagger, NO limits)
    publish_days = models.JSONField(default=list)  # ['mon', 'tue', 'wed', 'thu', 'fri']
    publish_time_slots = models.JSONField(default=list)  # ['09:00', '14:00', '18:00']
    
    # REMOVED: scheduling_mode, daily/weekly/monthly limits, stagger_*, queue_limit
    
    class Meta:
        db_table = 'igny8_site_ai_automation_settings'

2.6 Publishing Schedule - SIMPLIFIED (Corrected)

What PublishingSettings Controls: SCHEDULING (when content gets published), NOT publishing itself.

REMOVED Features (Unnecessary Complexity):

❌ Staggered mode - Not needed
❌ Daily/weekly/monthly limits - Derived from time slots × days
❌ Queue limit - No artificial limit needed

SIMPLIFIED Model:

class PublishingSettings(AccountBaseModel):
    """Site-level publishing SCHEDULE configuration."""
    
    site = models.OneToOneField('Site', on_delete=models.CASCADE)
    
    # Auto-approval/publish toggles
    auto_approval_enabled = models.BooleanField(default=True)
    auto_publish_enabled = models.BooleanField(default=True)
    
    # Schedule Configuration (Time Slots mode ONLY)
    publish_days = models.JSONField(default=list)  # ['mon', 'tue', 'wed', 'thu', 'fri']
    publish_time_slots = models.JSONField(default=list)  # ['09:00', '14:00', '18:00']
    
    # REMOVED:
    # - scheduling_mode (only time_slots needed)
    # - daily_publish_limit (derived: len(time_slots))
    # - weekly_publish_limit (derived: len(time_slots) × len(publish_days))
    # - monthly_publish_limit (not needed)
    # - stagger_* fields (not needed)
    # - queue_limit (not needed)

How It Works:

┌─────────────────────────────────────────────────────────────────────────────┐
│ PUBLISHING SCHEDULE = Days × Time Slots                                     │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  Selected Days: [✓] Mon [✓] Tue [✓] Wed [✓] Thu [✓] Fri [ ] Sat [ ] Sun    │
│                 └── 5 days selected                                        │
│                                                                             │
│  Time Slots: [09:00] [14:00] [18:00]                                       │
│              └── 3 slots per day                                            │
│                                                                             │
│  ════════════════════════════════════════════════════════════════════════   │
│                                                                             │
│  Calculated Capacity:                                                       │
│    • Daily: 3 articles (one per time slot)                                 │
│    • Weekly: 15 articles (3 slots × 5 days)                                │
│    • Monthly: ~65 articles (15 × 4.3 weeks)                                │
│                                                                             │
│  ════════════════════════════════════════════════════════════════════════   │
│                                                                             │
│  Content Flow:                                                              │
│    [Approved] → [Scheduled for next available slot] → [Published]          │
│                                                                             │
│  Example (Wednesday 10:30 AM):                                              │
│    Content approved → Scheduled for Wed 14:00 (next available slot)        │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

No Queue Limit Needed:

Content is scheduled to the next available time slot
If all slots are full for today, schedule for tomorrow
Natural flow, no artificial limits

Part 3: Backend Changes

3.1 Add is_testing Flag to AIModelConfig

# In business/billing/models.py - AIModelConfig

class AIModelConfig(models.Model):
    # ... existing fields ...
    
    # REMOVE these fields:
    # - quality_tier (was: basic/quality/premium for images)
    # - model_tier (proposed but not needed)
    
    # ADD: Simple testing flag
    is_testing = models.BooleanField(
        default=False,
        db_index=True,
        help_text="Testing model (cheap, for testing only). One per model_type."
    )
    
    class Meta:
        constraints = [
            # Only one testing model per type (text/image)
            models.UniqueConstraint(
                fields=['model_type'],
                condition=models.Q(is_testing=True, is_active=True),
                name='unique_testing_model_per_type'
            )
        ]
    
    @classmethod
    def get_testing_model(cls, model_type: str):
        """Get the testing model for text or image"""
        return cls.objects.filter(
            model_type=model_type,
            is_testing=True,
            is_active=True
        ).first()
    
    @classmethod
    def get_live_model(cls, model_type: str):
        """Get the live (default production) model for text or image"""
        return cls.objects.filter(
            model_type=model_type,
            is_testing=False,
            is_default=True,
            is_active=True
        ).first()
    
    # NEW: Only one model per (model_type, model_tier) can be active
    @classmethod
    def get_model_for_tier(cls, model_type: str, tier: str):
        """Get the active model for a given type and tier"""
        return cls.objects.filter(
            model_type=model_type,
            model_tier=tier,
            is_active=True
        ).first()

3.2 Unified Settings API Endpoint

# New file: backend/igny8_core/api/unified_settings.py

class UnifiedSiteSettingsViewSet(viewsets.ViewSet):
    """
    Unified API for all site AI & automation settings.
    
    GET /api/v1/sites/{site_id}/unified-settings/
    PUT /api/v1/sites/{site_id}/unified-settings/
    """
    
    def retrieve(self, request, site_id):
        """Get all settings for a site in one response"""
        site = get_object_or_404(Site, id=site_id, account=request.user.account)
        
        # Get or create the unified settings
        settings, _ = SiteAIAutomationSettings.objects.get_or_create(
            site=site,
            defaults=self._get_defaults()
        )
        
        # Get available models (Testing vs Live)
        text_models = {
            'testing': AIModelConfig.get_testing_model('text'),
            'live': AIModelConfig.get_live_model('text'),
        }
        image_models = {
            'testing': AIModelConfig.get_testing_model('image'),
            'live': AIModelConfig.get_live_model('image'),
        }
        
        return Response({
            'automation': {
                'enabled': settings.automation_enabled,
                'frequency': settings.automation_frequency,
                'time': settings.automation_time.strftime('%H:%M'),
            },
            'stages': self._build_stage_matrix(settings),
            'delays': {
                'within_stage': settings.within_stage_delay,
                'between_stage': settings.between_stage_delay,
            },
            'publishing': {
                'auto_approval_enabled': settings.auto_approval_enabled,
                'auto_publish_enabled': settings.auto_publish_enabled,
                'publish_days': settings.publish_days,
                'time_slots': settings.publish_time_slots,
                # Calculated capacity (read-only, derived from days × slots)
                'daily_capacity': len(settings.publish_time_slots),
                'weekly_capacity': len(settings.publish_time_slots) * len(settings.publish_days),
            },
            'available_models': {
                'text': {
                    'testing': text_models['testing'].display_name if text_models['testing'] else None,
                    'live': text_models['live'].display_name if text_models['live'] else None,
                },
                'image': {
                    'testing': image_models['testing'].display_name if image_models['testing'] else None,
                    'live': image_models['live'].display_name if image_models['live'] else None,
                },
            },
        })
    
    def _build_stage_matrix(self, settings):
        """Build stage configuration matrix"""
        stage_config = settings.stage_config or {}
        
        stages = [
            {'number': 1, 'name': 'Keywords → Clusters', 'has_ai': True},
            {'number': 2, 'name': 'Clusters → Ideas', 'has_ai': True},
            {'number': 3, 'name': 'Ideas → Tasks', 'has_ai': False},
            {'number': 4, 'name': 'Tasks → Content', 'has_ai': True},
            {'number': 5, 'name': 'Content → Prompts', 'has_ai': True},
            {'number': 6, 'name': 'Prompts → Images', 'has_ai': True},
            {'number': 7, 'name': 'Review → Approved', 'has_ai': False},
        ]
        
        result = []
        for stage in stages:
            num = str(stage['number'])
            config = stage_config.get(num, {})
            
            result.append({
                'number': stage['number'],
                'name': stage['name'],
                'has_ai': stage['has_ai'],
                'enabled': config.get('enabled', True),
                'batch_size': config.get('batch_size', self._default_batch_size(stage['number'])),
                'per_run_limit': config.get('per_run_limit', 0),
                'model_tier': config.get('model_tier', 'standard') if stage['has_ai'] else None,
                'credit_percentage': config.get('credit_percentage', 20) if stage['has_ai'] else None,
            })
        
        return result

3.3 Remove Deprecated Models/Fields

Phase 1 (This Release):

Add deprecation warnings to GlobalIntegrationSettings API key fields
Add model_tier to AIModelConfig
Create SiteAIAutomationSettings model

Phase 2 (Next Release):

Remove API key fields from GlobalIntegrationSettings
Remove AISettings model (settings_models.py)
Migrate data from old models to new unified model

Phase 3 (v2.0):

Remove GlobalIntegrationSettings model entirely
Remove separate AutomationConfig, PublishingSettings models
Full migration to unified settings

Part 4: Frontend Changes

4.1 Unified Settings Page - Full UI Mockup

Location: frontend/src/pages/Sites/AIAutomationSettings.tsx
Access: Site Settings > AI & Automation tab

┌─────────────────────────────────────────────────────────────────────────────────────────────┐
│ AI & Automation Settings                                                    [Save Changes]  │
├─────────────────────────────────────────────────────────────────────────────────────────────┤
│                                                                                             │
│ ┌─── Quick Settings ────────────────────────────────────────────────────────────────────┐  │
│ │                                                                                        │  │
│ │  Automation         [✓] Enable Scheduled Runs    Frequency: [Daily ▼]  Time: [02:00]  │  │
│ │  Publishing         [✓] Auto-Approve Content     [✓] Auto-Publish to Site             │  │
│ │                                                                                        │  │
│ └────────────────────────────────────────────────────────────────────────────────────────┘  │
│                                                                                             │
│ ┌─── Stage Configuration Matrix ────────────────────────────────────────────────────────┐  │
│ │                                                                                        │  │
│ │  ┌────────────────────────┬────────┬───────────┬───────────┬──────────────┬─────────┐ │  │
│ │  │ Stage                  │ Enable │ Batch Size│ Per-Run   │ Testing/Live │ Budget  │ │  │
│ │  │                        │        │           │ Limit*    │ AI Model     │   %     │ │  │
│ │  ├────────────────────────┼────────┼───────────┼───────────┼──────────────┼─────────┤ │  │
│ │  │ 1. Keywords→Clusters   │  [✓]   │   [50]    │   [0]     │   Live  ▼    │  [15]   │ │  │
│ │  ├────────────────────────┼────────┼───────────┼───────────┼──────────────┼─────────┤ │  │
│ │  │ 2. Clusters→Ideas      │  [✓]   │   [1]     │   [10]    │   Live  ▼    │  [10]   │ │  │
│ │  ├────────────────────────┼────────┼───────────┼───────────┼──────────────┼─────────┤ │  │
│ │  │ 3. Ideas→Tasks         │  [✓]   │   [20]    │   [0]     │      -       │    -    │ │  │
│ │  │    (local operation)   │        │           │           │              │         │ │  │
│ │  ├────────────────────────┼────────┼───────────┼───────────┼──────────────┼─────────┤ │  │
│ │  │ 4. Tasks→Content       │  [✓]   │   [1]     │   [5]     │   Live  ▼    │  [40]   │ │  │
│ │  ├────────────────────────┼────────┼───────────┼───────────┼──────────────┼─────────┤ │  │
│ │  │ 5. Content→Prompts     │  [✓]   │   [1]     │   [5]     │   Live  ▼    │  [5]    │ │  │
│ │  ├────────────────────────┼────────┼───────────┼───────────┼──────────────┼─────────┤ │  │
│ │  │ 6. Prompts→Images      │  [✓]   │   [1]     │   [20]    │  Testing ▼   │  [30]   │ │  │
│ │  ├────────────────────────┼────────┼───────────┼───────────┼──────────────┼─────────┤ │  │
│ │  │ 7. Review→Approved     │  [✓]   │    -      │   [10]    │      -       │    -    │ │  │
│ │  │    (local operation)   │        │           │           │              │         │ │  │
│ │  └────────────────────────┴────────┴───────────┴───────────┴──────────────┴─────────┘ │  │
│ │                                                                                        │  │
│ │  * Per-Run Limit: Max INPUT items to process per run (0 = all pending)                │  │
│ │    Stage 1: max keywords to cluster                                                   │  │
│ │    Stage 2: max clusters to generate ideas from                                       │  │
│ │    Stage 4: max tasks to generate content for                                         │  │
│ │                                                                                        │  │
│ │  Budget %: Credit allocation per AI stage (must total ≤100%)          Total: 100% ✓   │  │
│ │                                                                                        │  │
│ │  Delays:  Between stages [5] seconds    Within stage [3] seconds                      │  │
│ │                                                                                        │  │
│ └────────────────────────────────────────────────────────────────────────────────────────┘  │
│                                                                                             │
│ ┌─── Publishing Schedule (Simplified) ──────────────────────────────────────────────────┐  │
│ │                                                                                        │  │
│ │  [✓] Auto-Approve Content     [✓] Auto-Publish to Site                                │  │
│ │                                                                                        │  │
│ │  Publish Days:  [✓] Mon  [✓] Tue  [✓] Wed  [✓] Thu  [✓] Fri  [ ] Sat  [ ] Sun        │  │
│ │                                                                                        │  │
│ │  Time Slots:  [09:00] [14:00] [18:00] [+ Add Slot]                                    │  │
│ │                                                                                        │  │
│ │  ┌─ Calculated Capacity ──────────────────────────────────────────────────────────┐   │  │
│ │  │  📊 Daily: 3 articles    Weekly: 15 articles    Monthly: ~65 articles          │   │  │
│ │  └────────────────────────────────────────────────────────────────────────────────┘   │  │
│ │                                                                                        │  │
│ │  ℹ️ Content is scheduled to the next available time slot automatically               │  │
│ │                                                                                        │  │
│ └────────────────────────────────────────────────────────────────────────────────────────┘  │
│                                                                                             │
│                                               [Reset to Defaults]     [Save Changes]        │
└─────────────────────────────────────────────────────────────────────────────────────────────┘

4.1.1 Stage Configuration Matrix - Column Reference

Column	Description	Input Type	Applicable Stages
Enable	Turn stage on/off	Toggle	All (1-7)
Batch Size	Items per batch	Number input	1, 2, 3, 4, 5, 6
Per-Run Limit	Max INPUT items per run (0=all)	Number input	All (1-7)
Testing/Live AI Model	Which model to use	Dropdown	1, 2, 4, 5, 6 (AI stages)
Budget %	Credit allocation	Number (0-100)	1, 2, 4, 5, 6 (AI stages)

Stages 3 and 7 show "-" for AI-related columns (they're local operations, no AI calls).

// components/AIModelSelect.tsx

interface AIModelSelectProps {
  modelType: 'text' | 'image';
  useTesting: boolean;
  onChange: (useTesting: boolean) => void;
  disabled?: boolean;
}

export function AIModelSelect({ modelType, useTesting, onChange, disabled }: AIModelSelectProps) {
  return (
    <Select 
      value={useTesting ? 'testing' : 'live'} 
      onValueChange={(v) => onChange(v === 'testing')} 
      disabled={disabled}
    >
      <SelectTrigger className="w-24">
        <SelectValue />
      </SelectTrigger>
      <SelectContent>
        <SelectItem value="testing">
          <div className="flex items-center gap-2">
            <span className="text-yellow-600">⚡</span>
            <span>Testing</span>
          </div>
        </SelectItem>
        <SelectItem value="live">
          <div className="flex items-center gap-2">
            <span className="text-green-600">●</span>
            <span>Live</span>
          </div>
        </SelectItem>
      </SelectContent>
    </Select>
  );
}

// Tooltip for Testing mode
<Tooltip>
  <TooltipTrigger>
    <span className="text-yellow-600">⚡ Testing</span>
  </TooltipTrigger>
  <TooltipContent>
    <p>100x cheaper. For testing only.</p>
    <p className="text-red-500">Not suitable for production content.</p>
  </TooltipContent>
</Tooltip>

// In AppSidebar.tsx - Updated menu structure

const menuSections: MenuSection[] = useMemo(() => {
  return [
    // Dashboard standalone
    {
      label: "",
      items: [{ icon: <GridIcon />, name: "Dashboard", path: "/" }],
    },
    
    // SETUP - Site configuration
    {
      label: "SETUP",
      items: [
        { icon: <ShootingStarIcon />, name: "Setup Wizard", path: "/setup/wizard" },
        { icon: <GridIcon />, name: "Sites", path: "/sites" },
        { icon: <DocsIcon />, name: "Keyword Library", path: "/setup/add-keywords" },
        // Thinker (admin only)
        isModuleEnabled('thinker') && {
          icon: <BoltIcon />,
          name: "Thinker",
          subItems: [
            { name: "Prompts", path: "/thinker/prompts" },
            { name: "Author Profiles", path: "/thinker/author-profiles" },
          ],
          adminOnly: true,
        },
      ].filter(Boolean),
    },
    
    // WORKFLOW - Content pipeline
    {
      label: "WORKFLOW",
      items: [
        isModuleEnabled('planner') && {
          icon: <ListIcon />,
          name: "Planner",
          subItems: [
            { name: "Keywords", path: "/planner/keywords" },
            { name: "Clusters", path: "/planner/clusters" },
            { name: "Ideas", path: "/planner/ideas" },
          ],
        },
        isModuleEnabled('writer') && {
          icon: <TaskIcon />,
          name: "Writer",
          subItems: [
            { name: "Content Queue", path: "/writer/tasks" },
            { name: "Content Drafts", path: "/writer/content" },
            { name: "Content Images", path: "/writer/images" },
          ],
        },
        {
          icon: <CalendarIcon />,
          name: "Publisher",
          subItems: [
            { name: "Content Review", path: "/writer/review" },
            { name: "Publish / Schedule", path: "/writer/approved" },
            { name: "Content Calendar", path: "/publisher/content-calendar" },
          ],
        },
        isModuleEnabled('automation') && {
          icon: <BoltIcon />,
          name: "Automation",
          subItems: [
            { name: "Overview", path: "/automation/overview" },
            { name: "Run Now", path: "/automation/run" },
          ],
        },
      ].filter(Boolean),
    },
    
    // ACCOUNT - User account management
    {
      label: "ACCOUNT",
      items: [
        { icon: <UserCircleIcon />, name: "Account Settings", path: "/account/settings" },
        { icon: <DollarLineIcon />, name: "Plans & Billing", path: "/account/plans" },
        { icon: <PieChartIcon />, name: "Usage", path: "/account/usage" },
        {
          icon: <PlugInIcon />,
          name: "AI Configuration",
          path: "/settings/ai-models",
          adminOnly: true,
        },
      ],
    },
    
    // HELP
    {
      label: "HELP",
      items: [
        { icon: <Bell />, name: "Notifications", path: "/account/notifications" },
        { icon: <DocsIcon />, name: "Help & Docs", path: "/help" },
      ],
    },
  ];
}, [isModuleEnabled]);

// In AppSidebar.tsx - Fix dropdown to stay open when inner page is active

useEffect(() => {
  const currentPath = location.pathname;
  let foundMatch = false;
  
  allSections.forEach((section, sectionIndex) => {
    section.items.forEach((nav, itemIndex) => {
      if (nav.subItems && !foundMatch) {
        // Check if any subitem matches current path
        const shouldOpen = nav.subItems.some((subItem) => {
          // Exact match
          if (currentPath === subItem.path) return true;
          // Prefix match for nested routes (e.g., /sites/123/settings matches /sites)
          if (currentPath.startsWith(subItem.path + '/')) return true;
          return false;
        });
        
        if (shouldOpen) {
          setOpenSubmenu({ sectionIndex, itemIndex });
          foundMatch = true;
        }
      }
    });
  });
  
  // IMPORTANT: Don't close if no match - user may have manually opened
}, [location.pathname, allSections]);

Part 5: Django Admin Error Prevention

5.1 Safe Admin Actions

Add try/except wrappers to all admin actions to prevent 500 errors:

# In admin/base.py - Enhanced base admin class

class Igny8ModelAdmin(ModelAdmin):
    """Base admin class with error handling"""
    
    def save_model(self, request, obj, form, change):
        """Safe save with error handling"""
        try:
            super().save_model(request, obj, form, change)
        except IntegrityError as e:
            messages.error(request, f"Database error: {str(e)}")
        except ValidationError as e:
            messages.error(request, f"Validation error: {str(e)}")
        except Exception as e:
            messages.error(request, f"Unexpected error: {str(e)}")
            logger.exception(f"Error saving {obj.__class__.__name__}")
    
    def delete_model(self, request, obj):
        """Safe delete with error handling"""
        try:
            super().delete_model(request, obj)
        except ProtectedError as e:
            messages.error(request, f"Cannot delete: {str(e)}")
        except Exception as e:
            messages.error(request, f"Delete error: {str(e)}")
            logger.exception(f"Error deleting {obj.__class__.__name__}")
    
    def delete_queryset(self, request, queryset):
        """Safe bulk delete"""
        try:
            super().delete_queryset(request, queryset)
        except ProtectedError as e:
            messages.error(request, f"Cannot delete some items: {str(e)}")
        except Exception as e:
            messages.error(request, f"Bulk delete error: {str(e)}")

5.2 Admin Error Logging

# Add to all admin classes that process data

class CreditTransactionAdmin(Igny8ModelAdmin):
    
    @admin.action(description='Process selected transactions')
    def process_transactions(self, request, queryset):
        success_count = 0
        error_count = 0
        
        for obj in queryset:
            try:
                obj.process()
                success_count += 1
            except Exception as e:
                error_count += 1
                logger.error(f"Failed to process transaction {obj.id}: {e}")
        
        if success_count:
            messages.success(request, f'{success_count} transaction(s) processed.')
        if error_count:
            messages.warning(request, f'{error_count} transaction(s) failed. Check logs.')

Part 6: Migration Plan

Phase 1: Preparation (Week 1)

Add model_tier field to AIModelConfig - Migration
Create SiteAIAutomationSettings model - Migration
Add deprecation warnings to GlobalIntegrationSettings
Create unified API endpoint (read-only initially)

Phase 2: Data Migration (Week 2)

Migrate existing AutomationConfig → SiteAIAutomationSettings
Migrate existing PublishingSettings → SiteAIAutomationSettings
Migrate existing SiteAIBudgetAllocation → SiteAIAutomationSettings.budget_allocation
Update automation service to use new model

Phase 3: Frontend Update (Week 3)

Create unified settings component
Update sidebar navigation
Update Site Settings to use new unified tab
Implement model tier dropdowns

Phase 4: Cleanup (Week 4)

Remove old separate settings pages
Mark old API endpoints as deprecated
Remove AISettings model
Update documentation

Phase 5: Final Migration (v2.0)

Remove GlobalIntegrationSettings model
Remove separate AutomationConfig, PublishingSettings models
Remove deprecated API endpoints

Part 7: Testing Checklist

Backend Tests

SiteAIAutomationSettings CRUD works correctly
Unified API returns complete settings
Unified API saves all settings atomically
is_testing flag works on AIModelConfig
Migration scripts run without errors
Old models still work during transition
SiteIntegration removal doesn't break SyncEvent

Frontend Tests

Site Settings > AI & Automation tab loads correctly
Stage matrix displays all 7 stages
Testing/Live AI Model dropdown works
Budget % validation (must total ≤100%)
Save button saves all settings atomically
Sidebar stays open on inner pages
No 500 errors in admin actions

Integration Tests

Automation runs use new unified settings
Publishing uses new settings
Credit budget (stage %) works correctly
Testing/Live model selection works in AI calls

Part 8: Files to Create/Modify

Backend Files

File	Action	Purpose
`business/billing/models.py`	MODIFY	Add `is_testing` field to AIModelConfig
`business/automation/models.py`	CREATE	Add `SiteAIAutomationSettings` model
`api/unified_settings.py`	CREATE	New unified API endpoint
`urls/api_v1.py`	MODIFY	Register new endpoint
`business/integration/models.py`	MODIFY	Remove SiteIntegration (Phase 2)
`admin/billing.py`	MODIFY	Update Django Admin grouping
`migrations/XXXX_*.py`	CREATE	Database migrations

Frontend Files

File	Action	Purpose
`pages/Sites/AIAutomationSettings.tsx`	CREATE	Main unified settings page
`components/Settings/StageMatrix.tsx`	CREATE	Reusable stage matrix component
`components/Settings/AIModelSelect.tsx`	CREATE	Testing/Live dropdown component
`services/unifiedSettingsService.ts`	CREATE	API service for unified endpoint
`pages/Sites/SiteSettings.tsx`	MODIFY	Add AI & Automation tab
`layout/AppSidebar.tsx`	MODIFY	Keep dropdowns open on inner pages

Files to Remove (Phase 2)

File	Reason
Separate automation settings pages	Replaced by unified page
Old settings API endpoints	Deprecated, use unified

Part 9: Quick Setup Presets (Future Enhancement)

┌─── Quick Setup Presets ─────────────────────────────────────────┐
│                                                                  │
│  [ ] Conservative - Low throughput, high quality                │
│      5 tasks/run, 3 images/run, 30min stagger                   │
│                                                                  │
│  [●] Balanced - Moderate throughput (Recommended)               │
│      10 tasks/run, 20 images/run, 15min stagger                 │
│                                                                  │
│  [ ] Aggressive - Maximum throughput                            │
│      Unlimited, 5min stagger                                    │
│                                                                  │
│  [ ] Custom - Configure manually                                │
│                                                                  │
└──────────────────────────────────────────────────────────────────┘

Summary

Before: 17 models, 40+ settings, 4+ pages to configure one workflow
After: ~10 models, unified page, matrix view, clear hierarchy

Metric	Before	After
Settings Models	17	~10
Frontend Pages for Config	4+	1
API Calls to Load	5+	1
API Calls to Save	5+	1
AI Model Options	GPT-4.1, Claude, etc. (confusing)	Testing / Live (simple)
Credit Budget Config	Manual max + allocation	Auto (all credits) + stage %
User Confusion	High	Low

Key Simplifications

AI Models: Just "Testing" vs "Live" - no quality tiers
Credit Budget: Always enabled, uses all available credits
Publishing: Only 3 modes + queue limit (removed daily/weekly targets)
Per-Run Limit: Clear - applies to INPUT items per stage
SiteIntegration: REMOVED - credentials in Site model

58 KiB Raw Blame History Unescape Escape