Files
igny8/docs/plans/SETTINGS-CONSOLIDATION-PLAN.md
2026-01-18 12:22:27 +00:00

1273 lines
58 KiB
Markdown
Raw Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Settings & AI Configuration Consolidation Plan
**Created:** January 17, 2026
**Updated:** January 17, 2026
**Status:** FINALIZED - READY FOR IMPLEMENTATION
**Priority:** 🟡 HIGH - Improve UX & Reduce Confusion
**Depends On:** None (standalone improvement)
> **📋 SINGLE PLAN FILE** - This document covers both backend AND frontend implementation.
---
## Executive Summary
The IGNY8 codebase has **40+ scattered settings** across **17 different models** with significant redundancy. This plan consolidates them into a clean, hierarchical structure with a unified frontend experience.
### Key Decisions (Finalized)
1. **AI Models:** Simple **Testing** vs **Live** toggle (no quality tiers)
2. **Credit Budget:** Enabled by default, uses all available credits
3. **Per-Run Limits:** Apply to INPUT items (keywords, clusters, etc.)
4. **SiteIntegration Model:** To be REMOVED (redundant with Site.wp_* fields)
5. **Terminology:** Use "Site" not "WordPress" throughout
---
## Part 1: Current State Analysis
### 1.1 Settings Model Inventory
| Scope | Model | Location | Purpose | Status |
|-------|-------|----------|---------|--------|
| **GLOBAL** | `IntegrationProvider` | modules/system/models.py | API keys - ONE record per provider (openai, runware, stripe, paypal, resend) | ✅ CANONICAL (SINGLE) |
| | `SystemAISettings` | modules/system/ai_settings.py | AI parameter defaults (temp, tokens, image style) | ✅ Active |
| | `GlobalIntegrationSettings` | modules/system/models.py | Legacy API keys + AI defaults | ⚠️ **DEPRECATED** |
| | `AIModelConfig` | business/billing/models.py | Model definitions + credit pricing | ✅ Active |
| | `CreditCostConfig` | business/billing/models.py | Fixed credits per operation | ✅ Active |
| | `BillingConfiguration` | business/billing/models.py | Token-credit ratios | ✅ Active |
| | `GlobalModuleSettings` | modules/system/models.py | Platform module toggles | ✅ Active |
| | `CreditPackage` | business/billing/models.py | Credit purchase bundles | ✅ Active |
| | `PaymentMethodConfig` | business/billing/models.py | Payment methods per country | ✅ Active |
| **ACCOUNT** | `AccountSettings` | modules/system/settings_models.py | Key-value store - MULTIPLE records per account (ai.temperature, ai.max_tokens, etc.) | ✅ CANONICAL |
| | `ModuleEnableSettings` | modules/system/settings_models.py | Account module toggles | ✅ Active |
| | `AISettings` | modules/system/settings_models.py | Integration configs per account (overlaps AccountSettings) | ⚠️ **TO REMOVE** - use AccountSettings instead |
| | `IntegrationSettings` | business/integration/models.py | Account integration overrides | ✅ Active |
| | `AccountPaymentMethod` | business/billing/models.py | Saved payment methods | ✅ Active |
| **SITE** | `AutomationConfig` | business/automation/models.py | Automation stages, limits | ✅ Active |
| | `PublishingSettings` | business/integration/models.py | Publishing schedule | ✅ Active |
| | `SiteAIBudgetAllocation` | business/billing/models.py | Credit % per AI function | ✅ Active |
| | `SiteIntegration` | business/integration/models.py | Site platform credentials | ⚠️ **TO BE REMOVED** - Redundant with Site.wp_* fields |
| **USER** | `UserSettings` | modules/system/settings_models.py | User preferences | ✅ Active |
### 1.2 Identified Problems
#### Problem 1: API Keys in Multiple Places
```
❌ GlobalIntegrationSettings.openai_api_key
❌ GlobalIntegrationSettings.anthropic_api_key
❌ GlobalIntegrationSettings.runware_api_key
✅ IntegrationProvider.encrypted_credentials (CANONICAL)
```
**Fix:** Remove API key fields from GlobalIntegrationSettings
#### Problem 2: AI Parameters Fragmented
```
SystemAISettings.temperature ← Global default
GlobalIntegrationSettings.openai_temperature ← Duplicate
AccountSettings[ai.temperature] ← Account override
IntegrationSettings.config.temperature ← Another override
```
**Fix:** Keep only SystemAISettings + AccountSettings override
#### Problem 3: Model Selection Confusion
```
AIModelConfig.is_default = True ← Should be canonical
GlobalIntegrationSettings.default_text_model ← Duplicate
GlobalIntegrationSettings.default_image_model ← Duplicate
```
**Fix:** Use AIModelConfig.is_default only
#### Problem 4: Frontend "Testing vs Live" Model Not Implemented
```
Current: Model names exposed directly (GPT-4.1, Claude 3.5, etc.)
Needed: "Testing Model" and "Live Model" abstraction
```
**Fix:** Add `model_tier` field to AIModelConfig
#### Problem 5: Redundant AISettings Model
```
AISettings.integration_type + config ← Overlaps with:
- AccountSettings (ai.* keys)
- IntegrationSettings.config
```
**Fix:** Remove AISettings model, consolidate into AccountSettings
---
## Part 2: Target Architecture
### 2.0 Frontend Location: CONFIRMED
**Location:** Site Settings > AI & Automation Tab
This is the CORRECT location for the unified settings page:
- Near existing site configuration
- Familiar location for users
- Keeps all site-specific settings together
- Rename existing "AI Settings" tab to "AI & Automation"
**NOT** in Setup menu - Setup is for initial configuration, not ongoing settings.
---
### 2.1 Django Admin Sidebar - Before & After
#### BEFORE (Current - From Screenshot)
```
Accounts & Users ▼
├── Accounts
├── Users
├── Sites
├── Sectors
└── Site Access
Plans & Billing ▼
├── Plans
├── Subscriptions
├── Invoices
├── Payments
├── Credit Packages
├── Payment Methods (Global)
└── Account Payment Methods
Credits ▼
├── Credit Transactions
├── Credit Usage Logs
├── Credit Cost Config ← Rarely used
└── Billing Configuration ← Rarely used
(AI Models scattered elsewhere)
```
#### AFTER (Proposed - Reorganized)
```
Dashboard & Reports ▼
├── Dashboard
├── Revenue Report
├── Usage Report
├── Content Report
├── AI Cost Analysis ← Margin/pricing reports here
└── Token Usage
Accounts & Users ▼
├── Accounts
├── Users
├── Sites
├── Sectors
└── Site Access
Plans & Billing ▼
├── Plans
├── Subscriptions
├── Invoices
├── Payments
├── Credit Packages
├── Payment Methods (Global)
└── Account Payment Methods
Credits & AI Usage ▼ ← CONSOLIDATED LOGS
├── Credit Transactions
├── Credit Usage Log
├── AI Task Logs ← Moved from AI Configuration
└── Plan Limits
AI Configuration ▼ ← SIMPLIFIED
├── AI Models (Testing/Live) ← Main config with COST fields
├── System AI Settings ← Default parameters
└── Integration Providers ← API keys (OpenAI, Runware, Stripe, etc.)
Content Pipeline ▼ ← RENAMED (was Planning + Writing)
├── Keywords
├── Clusters
├── Content Ideas
├── Tasks
├── Content
├── Images
└── Image Prompts
Publishing ▼
├── Publishing Records
├── Deployments
└── Sync Events
(Removed: Integrations - SiteIntegration removed)
Automation ▼ ← NEW SECTION
├── Automation Configs
└── Automation Runs
Global Settings ▼ ← SIMPLIFIED
├── Global AI Prompts
├── Module Settings
├── Author Profiles
└── Strategies
Account & User Settings ▼ ← CONSOLIDATED
├── Account Settings ← All key-value settings
├── User Settings
└── Module Enable Settings
Resources ▼
├── Industries
├── Industry Sectors
└── Seed Keywords
Plugin Management ▼
├── Plugins
├── Plugin Versions
├── Installations
└── Downloads
Email Settings ▼
├── Email Configuration
├── Email Templates
└── Email Logs
Logs & Monitoring ▼
├── System Health
├── API Monitor
├── Debug Console
├── Celery Tasks
└── Admin Log
Django Admin ▼
├── Groups
├── Permissions
├── Content Types
└── Sessions
Trash ▼
├── ... (all trash items)
```
#### Key Reorganization Changes
| Before | After | Reason |
|--------|-------|--------|
| AI Task Logs in "AI Configuration" | Moved to "Credits & AI Usage" | All logs together |
| Credit Cost Config, Billing Config | REMOVED | Costs in AIModelConfig |
| Planning + Writing (separate) | "Content Pipeline" (combined) | Logical flow |
| SiteIntegration in Publishing | REMOVED | Redundant with Site.wp_* |
| Global Settings (cluttered) | Simplified + separate Automation | Cleaner |
| System Configuration (scattered) | "Account & User Settings" | Consolidated |
---
### 2.2 Model Hierarchy (After Consolidation)
```
GLOBAL (Admin-Only Configuration)
├── IntegrationProvider # API keys for all providers (OpenAI, Anthropic, Runware)
├── AIModelConfig # Model definitions + pricing + is_testing flag
├── SystemAISettings # Default AI parameters (temp, tokens, image style)
└── GlobalModuleSettings # Platform module toggles
ACCOUNT (User Account Settings)
├── AccountSettings # Key-value store for all account config
├── ModuleEnableSettings # Which modules user can access
└── AccountPaymentMethod # Saved payment methods
SITE (Per-Site Configuration)
├── SiteAIAutomationSettings # NEW: Unified automation+AI+publishing
└── Site.wp_url, Site.wp_api_key # Integration credentials (moved from SiteIntegration)
USER
└── UserSettings # User preferences
```
**REMOVED Models:**
- `SiteIntegration` → credentials now in `Site` model
- `CreditCostConfig` → not actively used
- `BillingConfiguration` → hardcode values
- `GlobalIntegrationSettings` → replaced by `IntegrationProvider`
- `AISettings` (settings_models) → use `AccountSettings`
---
### 2.3 AI Model Designation: Testing vs Live (Simplified)
**REMOVED:** Standard, Premium, Quality tiers
**NEW:** Simple boolean `is_testing` field
| Field Value | Frontend Label | Description |
|-------------|---------------|-------------|
| `is_testing = True` | **Testing** | Cheap model for testing (100x cheaper, low quality) |
| `is_testing = False` | **Live** | Production model |
**Constraint:** Only ONE model can be `is_testing=True` per `model_type` (text/image).
**Frontend Column Header:** "Testing/Live AI Model"
**Dropdown Options:**
```
[Testing ▼] ← Shows only the testing model
[Live ▼] ← Shows only the live model
```
---
### 2.3.1 Costing & Pricing - Single Source of Truth
**IMPORTANT:** After removing CreditCostConfig, ALL costing/pricing lives in `AIModelConfig`.
#### Cost vs Price Definitions
| Term | Definition | Where Stored |
|------|------------|--------------|
| **Cost** | Our expense per token/image (what we pay to OpenAI, Runware, etc.) | `AIModelConfig.cost_per_1k_input`, `cost_per_1k_output` |
| **Price** | What we charge users in credits (revenue calculation) | `AIModelConfig.tokens_per_credit`, `credits_per_image` |
| **Margin** | Price - Cost = Revenue margin | Calculated in reports |
#### AIModelConfig Fields for Costing
```python
class AIModelConfig(models.Model):
# ═══════════════════════════════════════════════════════════
# COST (Our expense - what we pay providers)
# ═══════════════════════════════════════════════════════════
# Text models: Cost per 1K tokens (USD)
cost_per_1k_input = models.DecimalField(...) # e.g., $0.01 for GPT-4.1-mini input
cost_per_1k_output = models.DecimalField(...) # e.g., $0.03 for GPT-4.1-mini output
# Image models: Cost per image (USD) - stored in metadata or calculated
# e.g., DALL-E 3 = $0.04/image, Runware basic = $0.004/image
# ═══════════════════════════════════════════════════════════
# PRICE (What we charge users in credits)
# ═══════════════════════════════════════════════════════════
# Text models: How many tokens = 1 credit
tokens_per_credit = models.IntegerField(...) # e.g., 10000 tokens = 1 credit
# Image models: How many credits per image
credits_per_image = models.IntegerField(...) # e.g., 1 credit, 5 credits, 15 credits
```
#### Usage in Reports
```python
# AI Cost Analysis Report (Backend)
def calculate_margin(usage_log):
model = usage_log.model_config
# Calculate our cost
if model.model_type == 'text':
cost = (usage_log.input_tokens / 1000 * model.cost_per_1k_input +
usage_log.output_tokens / 1000 * model.cost_per_1k_output)
else: # image
cost = usage_log.images_generated * model.cost_per_image # from metadata
# Calculate what user paid (credits × credit value)
credits_used = usage_log.credits_used
credit_value = 0.01 # $0.01 per credit (configurable)
revenue = credits_used * credit_value
# Margin
margin = revenue - cost
margin_pct = (margin / revenue * 100) if revenue > 0 else 0
return {
'cost': cost,
'revenue': revenue,
'margin': margin,
'margin_pct': margin_pct,
}
```
#### What Gets Removed
| Model | Reason |
|-------|--------|
| `CreditCostConfig` | Redundant - costs now in AIModelConfig per model |
| `BillingConfiguration` | Rarely used - can hardcode credit value ($0.01) |
---
### 2.3.2 Settings Hierarchy - Where Defaults & Overrides Live
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ SETTINGS HIERARCHY │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ GLOBAL DEFAULTS (Admin-only, one record each) │
│ ├── IntegrationProvider (one per provider: openai, runware, stripe, etc.) │
│ │ └── API keys, endpoints, sandbox mode │
│ │ │
│ ├── AIModelConfig (one per model: gpt-4.1, dall-e-3, etc.) │
│ │ └── Model pricing, costs, capabilities, is_testing flag │
│ │ │
│ └── SystemAISettings (SINGLETON - one record total) │
│ └── Default temperature, max_tokens, image_style, image_quality │
│ │
│ ════════════════════════════════════════════════════════════════════════ │
│ │
│ ACCOUNT OVERRIDES (Multiple key-value records per account) │
│ └── AccountSettings (unique_together: account + key) │
│ ├── ai.temperature = 0.8 # Override SystemAISettings default │
│ ├── ai.max_tokens = 4000 # Override SystemAISettings default │
│ ├── ai.image_style = "vivid" # Override SystemAISettings default │
│ └── ... other account-specific settings │
│ │
│ ════════════════════════════════════════════════════════════════════════ │
│ │
│ SITE CONFIGURATION (One record per site) │
│ └── SiteAIAutomationSettings (NEW unified model) │
│ ├── Automation: enabled, frequency, time │
│ ├── Stages: JSON config per stage (batch_size, per_run_limit, etc.) │
│ └── Publishing: days, time_slots │
│ │
│ ════════════════════════════════════════════════════════════════════════ │
│ │
│ Resolution Order: Site → Account → Global Default │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
```
---
### 2.3.3 AISettings Model - REMOVAL PLAN
**Current State:** `AISettings` in `settings_models.py` stores per-account integration configs.
**Problem:** Overlaps with `AccountSettings` which already has `ai.*` keys.
**Migration:**
1. Any data in `AISettings.config` → Move to `AccountSettings` keys
2. `AISettings.model_preferences``AccountSettings['ai.model_preferences']`
3. `AISettings.cost_limits` → Not needed (credit budget in SiteAIAutomationSettings)
4. Remove `AISettings` model and admin
**After Removal:**
- Account AI settings use `AccountSettings` with keys like:
- `ai.temperature`
- `ai.max_tokens`
- `ai.image_style`
- `ai.image_quality`
- `ai.model_preferences`
---
### 2.4 SiteIntegration Model - REMOVAL PLAN
**Status:** CONFIRMED REDUNDANT - Safe to remove
**Analysis:**
| What SiteIntegration Stores | Where It Actually Lives |
|----------------------------|------------------------|
| `credentials_json.api_key` | `Site.wp_api_key` (single source of truth) |
| `config_json.url` | `Site.wp_url` (single source of truth) |
| `platform` | Always 'wordpress' (hardcoded) |
| `sync_status`, `last_sync_at` | Can move to `Site` model |
**Evidence from codebase:**
```python
# All services use Site model directly:
api_key = site.wp_api_key # NOT SiteIntegration.credentials_json
url = site.wp_url # NOT SiteIntegration.config_json
```
**Removal Steps:**
1. Add `sync_status`, `last_sync_at` fields to `Site` model
2. Update `SyncEvent.integration` FK → `SyncEvent.site` FK
3. Update `sync_health_service.py` to use Site directly
4. Remove `SiteIntegration` admin registration
5. Remove `SiteIntegration` model
6. Create migration to drop table
---
### 2.5 New Unified Model: SiteAIAutomationSettings
**Replaces:** AutomationConfig + PublishingSettings + SiteAIBudgetAllocation fields
```python
class SiteAIAutomationSettings(AccountBaseModel):
"""
Unified AI & Automation settings for a site.
Consolidates AutomationConfig, PublishingSettings, and budget allocation.
"""
site = models.OneToOneField('Site', on_delete=models.CASCADE)
# ═══════════════════════════════════════════════════════════
# AUTOMATION SCHEDULING
# ═══════════════════════════════════════════════════════════
automation_enabled = models.BooleanField(default=False)
automation_frequency = models.CharField(max_length=20, default='daily',
choices=[('hourly', 'Hourly'), ('daily', 'Daily'), ('weekly', 'Weekly')])
automation_time = models.TimeField(default='02:00')
# ═══════════════════════════════════════════════════════════
# STAGE CONFIGURATION (JSON for flexibility)
# ═══════════════════════════════════════════════════════════
# Structure: {
# "1": {"enabled": true, "batch_size": 50, "per_run_limit": 0, "use_testing": false, "budget_pct": 15},
# "2": {"enabled": true, "batch_size": 1, "per_run_limit": 10, "use_testing": false, "budget_pct": 10},
# ...
# }
# NOTE: per_run_limit applies to INPUT items:
# Stage 1: max keywords to cluster
# Stage 2: max clusters to generate ideas from
# Stage 4: max tasks to generate content for
# etc.
stage_config = models.JSONField(default=dict)
# ═══════════════════════════════════════════════════════════
# DELAYS
# ═══════════════════════════════════════════════════════════
within_stage_delay = models.IntegerField(default=3, help_text="Seconds between items")
between_stage_delay = models.IntegerField(default=5, help_text="Seconds between stages")
# ═══════════════════════════════════════════════════════════
# PUBLISHING SCHEDULE (Simplified - Time Slots only)
# ═══════════════════════════════════════════════════════════
auto_approval_enabled = models.BooleanField(default=True)
auto_publish_enabled = models.BooleanField(default=True)
# Schedule: Days + Time Slots (NO stagger, NO limits)
publish_days = models.JSONField(default=list) # ['mon', 'tue', 'wed', 'thu', 'fri']
publish_time_slots = models.JSONField(default=list) # ['09:00', '14:00', '18:00']
# REMOVED: scheduling_mode, daily/weekly/monthly limits, stagger_*, queue_limit
class Meta:
db_table = 'igny8_site_ai_automation_settings'
```
---
### 2.6 Publishing Schedule - SIMPLIFIED (Corrected)
**What PublishingSettings Controls:** SCHEDULING (when content gets published), NOT publishing itself.
**REMOVED Features (Unnecessary Complexity):**
- ❌ Staggered mode - Not needed
- ❌ Daily/weekly/monthly limits - Derived from time slots × days
- ❌ Queue limit - No artificial limit needed
**SIMPLIFIED Model:**
```python
class PublishingSettings(AccountBaseModel):
"""Site-level publishing SCHEDULE configuration."""
site = models.OneToOneField('Site', on_delete=models.CASCADE)
# Auto-approval/publish toggles
auto_approval_enabled = models.BooleanField(default=True)
auto_publish_enabled = models.BooleanField(default=True)
# Schedule Configuration (Time Slots mode ONLY)
publish_days = models.JSONField(default=list) # ['mon', 'tue', 'wed', 'thu', 'fri']
publish_time_slots = models.JSONField(default=list) # ['09:00', '14:00', '18:00']
# REMOVED:
# - scheduling_mode (only time_slots needed)
# - daily_publish_limit (derived: len(time_slots))
# - weekly_publish_limit (derived: len(time_slots) × len(publish_days))
# - monthly_publish_limit (not needed)
# - stagger_* fields (not needed)
# - queue_limit (not needed)
```
**How It Works:**
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ PUBLISHING SCHEDULE = Days × Time Slots │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ Selected Days: [✓] Mon [✓] Tue [✓] Wed [✓] Thu [✓] Fri [ ] Sat [ ] Sun │
│ └── 5 days selected │
│ │
│ Time Slots: [09:00] [14:00] [18:00] │
│ └── 3 slots per day │
│ │
│ ════════════════════════════════════════════════════════════════════════ │
│ │
│ Calculated Capacity: │
│ • Daily: 3 articles (one per time slot) │
│ • Weekly: 15 articles (3 slots × 5 days) │
│ • Monthly: ~65 articles (15 × 4.3 weeks) │
│ │
│ ════════════════════════════════════════════════════════════════════════ │
│ │
│ Content Flow: │
│ [Approved] → [Scheduled for next available slot] → [Published] │
│ │
│ Example (Wednesday 10:30 AM): │
│ Content approved → Scheduled for Wed 14:00 (next available slot) │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
```
**No Queue Limit Needed:**
- Content is scheduled to the next available time slot
- If all slots are full for today, schedule for tomorrow
- Natural flow, no artificial limits
---
## Part 3: Backend Changes
### 3.1 Add is_testing Flag to AIModelConfig
```python
# In business/billing/models.py - AIModelConfig
class AIModelConfig(models.Model):
# ... existing fields ...
# REMOVE these fields:
# - quality_tier (was: basic/quality/premium for images)
# - model_tier (proposed but not needed)
# ADD: Simple testing flag
is_testing = models.BooleanField(
default=False,
db_index=True,
help_text="Testing model (cheap, for testing only). One per model_type."
)
class Meta:
constraints = [
# Only one testing model per type (text/image)
models.UniqueConstraint(
fields=['model_type'],
condition=models.Q(is_testing=True, is_active=True),
name='unique_testing_model_per_type'
)
]
@classmethod
def get_testing_model(cls, model_type: str):
"""Get the testing model for text or image"""
return cls.objects.filter(
model_type=model_type,
is_testing=True,
is_active=True
).first()
@classmethod
def get_live_model(cls, model_type: str):
"""Get the live (default production) model for text or image"""
return cls.objects.filter(
model_type=model_type,
is_testing=False,
is_default=True,
is_active=True
).first()
# NEW: Only one model per (model_type, model_tier) can be active
@classmethod
def get_model_for_tier(cls, model_type: str, tier: str):
"""Get the active model for a given type and tier"""
return cls.objects.filter(
model_type=model_type,
model_tier=tier,
is_active=True
).first()
```
### 3.2 Unified Settings API Endpoint
```python
# New file: backend/igny8_core/api/unified_settings.py
class UnifiedSiteSettingsViewSet(viewsets.ViewSet):
"""
Unified API for all site AI & automation settings.
GET /api/v1/sites/{site_id}/unified-settings/
PUT /api/v1/sites/{site_id}/unified-settings/
"""
def retrieve(self, request, site_id):
"""Get all settings for a site in one response"""
site = get_object_or_404(Site, id=site_id, account=request.user.account)
# Get or create the unified settings
settings, _ = SiteAIAutomationSettings.objects.get_or_create(
site=site,
defaults=self._get_defaults()
)
# Get available models (Testing vs Live)
text_models = {
'testing': AIModelConfig.get_testing_model('text'),
'live': AIModelConfig.get_live_model('text'),
}
image_models = {
'testing': AIModelConfig.get_testing_model('image'),
'live': AIModelConfig.get_live_model('image'),
}
return Response({
'automation': {
'enabled': settings.automation_enabled,
'frequency': settings.automation_frequency,
'time': settings.automation_time.strftime('%H:%M'),
},
'stages': self._build_stage_matrix(settings),
'delays': {
'within_stage': settings.within_stage_delay,
'between_stage': settings.between_stage_delay,
},
'publishing': {
'auto_approval_enabled': settings.auto_approval_enabled,
'auto_publish_enabled': settings.auto_publish_enabled,
'publish_days': settings.publish_days,
'time_slots': settings.publish_time_slots,
# Calculated capacity (read-only, derived from days × slots)
'daily_capacity': len(settings.publish_time_slots),
'weekly_capacity': len(settings.publish_time_slots) * len(settings.publish_days),
},
'available_models': {
'text': {
'testing': text_models['testing'].display_name if text_models['testing'] else None,
'live': text_models['live'].display_name if text_models['live'] else None,
},
'image': {
'testing': image_models['testing'].display_name if image_models['testing'] else None,
'live': image_models['live'].display_name if image_models['live'] else None,
},
},
})
def _build_stage_matrix(self, settings):
"""Build stage configuration matrix"""
stage_config = settings.stage_config or {}
stages = [
{'number': 1, 'name': 'Keywords → Clusters', 'has_ai': True},
{'number': 2, 'name': 'Clusters → Ideas', 'has_ai': True},
{'number': 3, 'name': 'Ideas → Tasks', 'has_ai': False},
{'number': 4, 'name': 'Tasks → Content', 'has_ai': True},
{'number': 5, 'name': 'Content → Prompts', 'has_ai': True},
{'number': 6, 'name': 'Prompts → Images', 'has_ai': True},
{'number': 7, 'name': 'Review → Approved', 'has_ai': False},
]
result = []
for stage in stages:
num = str(stage['number'])
config = stage_config.get(num, {})
result.append({
'number': stage['number'],
'name': stage['name'],
'has_ai': stage['has_ai'],
'enabled': config.get('enabled', True),
'batch_size': config.get('batch_size', self._default_batch_size(stage['number'])),
'per_run_limit': config.get('per_run_limit', 0),
'model_tier': config.get('model_tier', 'standard') if stage['has_ai'] else None,
'credit_percentage': config.get('credit_percentage', 20) if stage['has_ai'] else None,
})
return result
```
### 3.3 Remove Deprecated Models/Fields
**Phase 1 (This Release):**
- Add deprecation warnings to `GlobalIntegrationSettings` API key fields
- Add `model_tier` to `AIModelConfig`
- Create `SiteAIAutomationSettings` model
**Phase 2 (Next Release):**
- Remove API key fields from `GlobalIntegrationSettings`
- Remove `AISettings` model (settings_models.py)
- Migrate data from old models to new unified model
**Phase 3 (v2.0):**
- Remove `GlobalIntegrationSettings` model entirely
- Remove separate `AutomationConfig`, `PublishingSettings` models
- Full migration to unified settings
---
## Part 4: Frontend Changes
### 4.1 Unified Settings Page - Full UI Mockup
**Location:** `frontend/src/pages/Sites/AIAutomationSettings.tsx`
**Access:** Site Settings > AI & Automation tab
```
┌─────────────────────────────────────────────────────────────────────────────────────────────┐
│ AI & Automation Settings [Save Changes] │
├─────────────────────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─── Quick Settings ────────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ Automation [✓] Enable Scheduled Runs Frequency: [Daily ▼] Time: [02:00] │ │
│ │ Publishing [✓] Auto-Approve Content [✓] Auto-Publish to Site │ │
│ │ │ │
│ └────────────────────────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌─── Stage Configuration Matrix ────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ ┌────────────────────────┬────────┬───────────┬───────────┬──────────────┬─────────┐ │ │
│ │ │ Stage │ Enable │ Batch Size│ Per-Run │ Testing/Live │ Budget │ │ │
│ │ │ │ │ │ Limit* │ AI Model │ % │ │ │
│ │ ├────────────────────────┼────────┼───────────┼───────────┼──────────────┼─────────┤ │ │
│ │ │ 1. Keywords→Clusters │ [✓] │ [50] │ [0] │ Live ▼ │ [15] │ │ │
│ │ ├────────────────────────┼────────┼───────────┼───────────┼──────────────┼─────────┤ │ │
│ │ │ 2. Clusters→Ideas │ [✓] │ [1] │ [10] │ Live ▼ │ [10] │ │ │
│ │ ├────────────────────────┼────────┼───────────┼───────────┼──────────────┼─────────┤ │ │
│ │ │ 3. Ideas→Tasks │ [✓] │ [20] │ [0] │ - │ - │ │ │
│ │ │ (local operation) │ │ │ │ │ │ │ │
│ │ ├────────────────────────┼────────┼───────────┼───────────┼──────────────┼─────────┤ │ │
│ │ │ 4. Tasks→Content │ [✓] │ [1] │ [5] │ Live ▼ │ [40] │ │ │
│ │ ├────────────────────────┼────────┼───────────┼───────────┼──────────────┼─────────┤ │ │
│ │ │ 5. Content→Prompts │ [✓] │ [1] │ [5] │ Live ▼ │ [5] │ │ │
│ │ ├────────────────────────┼────────┼───────────┼───────────┼──────────────┼─────────┤ │ │
│ │ │ 6. Prompts→Images │ [✓] │ [1] │ [20] │ Testing ▼ │ [30] │ │ │
│ │ ├────────────────────────┼────────┼───────────┼───────────┼──────────────┼─────────┤ │ │
│ │ │ 7. Review→Approved │ [✓] │ - │ [10] │ - │ - │ │ │
│ │ │ (local operation) │ │ │ │ │ │ │ │
│ │ └────────────────────────┴────────┴───────────┴───────────┴──────────────┴─────────┘ │ │
│ │ │ │
│ │ * Per-Run Limit: Max INPUT items to process per run (0 = all pending) │ │
│ │ Stage 1: max keywords to cluster │ │
│ │ Stage 2: max clusters to generate ideas from │ │
│ │ Stage 4: max tasks to generate content for │ │
│ │ │ │
│ │ Budget %: Credit allocation per AI stage (must total ≤100%) Total: 100% ✓ │ │
│ │ │ │
│ │ Delays: Between stages [5] seconds Within stage [3] seconds │ │
│ │ │ │
│ └────────────────────────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌─── Publishing Schedule (Simplified) ──────────────────────────────────────────────────┐ │
│ │ │ │
│ │ [✓] Auto-Approve Content [✓] Auto-Publish to Site │ │
│ │ │ │
│ │ Publish Days: [✓] Mon [✓] Tue [✓] Wed [✓] Thu [✓] Fri [ ] Sat [ ] Sun │ │
│ │ │ │
│ │ Time Slots: [09:00] [14:00] [18:00] [+ Add Slot] │ │
│ │ │ │
│ │ ┌─ Calculated Capacity ──────────────────────────────────────────────────────────┐ │ │
│ │ │ 📊 Daily: 3 articles Weekly: 15 articles Monthly: ~65 articles │ │ │
│ │ └────────────────────────────────────────────────────────────────────────────────┘ │ │
│ │ │ │
│ │ Content is scheduled to the next available time slot automatically │ │
│ │ │ │
│ └────────────────────────────────────────────────────────────────────────────────────────┘ │
│ │
│ [Reset to Defaults] [Save Changes] │
└─────────────────────────────────────────────────────────────────────────────────────────────┘
```
### 4.1.1 Stage Configuration Matrix - Column Reference
| Column | Description | Input Type | Applicable Stages |
|--------|-------------|------------|-------------------|
| **Enable** | Turn stage on/off | Toggle | All (1-7) |
| **Batch Size** | Items per batch | Number input | 1, 2, 3, 4, 5, 6 |
| **Per-Run Limit** | Max INPUT items per run (0=all) | Number input | All (1-7) |
| **Testing/Live AI Model** | Which model to use | Dropdown | 1, 2, 4, 5, 6 (AI stages) |
| **Budget %** | Credit allocation | Number (0-100) | 1, 2, 4, 5, 6 (AI stages) |
Stages 3 and 7 show "-" for AI-related columns (they're local operations, no AI calls).
### 4.2 Testing/Live AI Model Dropdown Component
```tsx
// components/AIModelSelect.tsx
interface AIModelSelectProps {
modelType: 'text' | 'image';
useTesting: boolean;
onChange: (useTesting: boolean) => void;
disabled?: boolean;
}
export function AIModelSelect({ modelType, useTesting, onChange, disabled }: AIModelSelectProps) {
return (
<Select
value={useTesting ? 'testing' : 'live'}
onValueChange={(v) => onChange(v === 'testing')}
disabled={disabled}
>
<SelectTrigger className="w-24">
<SelectValue />
</SelectTrigger>
<SelectContent>
<SelectItem value="testing">
<div className="flex items-center gap-2">
<span className="text-yellow-600"></span>
<span>Testing</span>
</div>
</SelectItem>
<SelectItem value="live">
<div className="flex items-center gap-2">
<span className="text-green-600"></span>
<span>Live</span>
</div>
</SelectItem>
</SelectContent>
</Select>
);
}
// Tooltip for Testing mode
<Tooltip>
<TooltipTrigger>
<span className="text-yellow-600"> Testing</span>
</TooltipTrigger>
<TooltipContent>
<p>100x cheaper. For testing only.</p>
<p className="text-red-500">Not suitable for production content.</p>
</TooltipContent>
</Tooltip>
```
### 4.3 Updated Sidebar Navigation
```tsx
// In AppSidebar.tsx - Updated menu structure
const menuSections: MenuSection[] = useMemo(() => {
return [
// Dashboard standalone
{
label: "",
items: [{ icon: <GridIcon />, name: "Dashboard", path: "/" }],
},
// SETUP - Site configuration
{
label: "SETUP",
items: [
{ icon: <ShootingStarIcon />, name: "Setup Wizard", path: "/setup/wizard" },
{ icon: <GridIcon />, name: "Sites", path: "/sites" },
{ icon: <DocsIcon />, name: "Keyword Library", path: "/setup/add-keywords" },
// Thinker (admin only)
isModuleEnabled('thinker') && {
icon: <BoltIcon />,
name: "Thinker",
subItems: [
{ name: "Prompts", path: "/thinker/prompts" },
{ name: "Author Profiles", path: "/thinker/author-profiles" },
],
adminOnly: true,
},
].filter(Boolean),
},
// WORKFLOW - Content pipeline
{
label: "WORKFLOW",
items: [
isModuleEnabled('planner') && {
icon: <ListIcon />,
name: "Planner",
subItems: [
{ name: "Keywords", path: "/planner/keywords" },
{ name: "Clusters", path: "/planner/clusters" },
{ name: "Ideas", path: "/planner/ideas" },
],
},
isModuleEnabled('writer') && {
icon: <TaskIcon />,
name: "Writer",
subItems: [
{ name: "Content Queue", path: "/writer/tasks" },
{ name: "Content Drafts", path: "/writer/content" },
{ name: "Content Images", path: "/writer/images" },
],
},
{
icon: <CalendarIcon />,
name: "Publisher",
subItems: [
{ name: "Content Review", path: "/writer/review" },
{ name: "Publish / Schedule", path: "/writer/approved" },
{ name: "Content Calendar", path: "/publisher/content-calendar" },
],
},
isModuleEnabled('automation') && {
icon: <BoltIcon />,
name: "Automation",
subItems: [
{ name: "Overview", path: "/automation/overview" },
{ name: "Run Now", path: "/automation/run" },
],
},
].filter(Boolean),
},
// ACCOUNT - User account management
{
label: "ACCOUNT",
items: [
{ icon: <UserCircleIcon />, name: "Account Settings", path: "/account/settings" },
{ icon: <DollarLineIcon />, name: "Plans & Billing", path: "/account/plans" },
{ icon: <PieChartIcon />, name: "Usage", path: "/account/usage" },
{
icon: <PlugInIcon />,
name: "AI Configuration",
path: "/settings/ai-models",
adminOnly: true,
},
],
},
// HELP
{
label: "HELP",
items: [
{ icon: <Bell />, name: "Notifications", path: "/account/notifications" },
{ icon: <DocsIcon />, name: "Help & Docs", path: "/help" },
],
},
];
}, [isModuleEnabled]);
```
### 4.4 Sidebar Dropdown Stay-Open Fix
```tsx
// In AppSidebar.tsx - Fix dropdown to stay open when inner page is active
useEffect(() => {
const currentPath = location.pathname;
let foundMatch = false;
allSections.forEach((section, sectionIndex) => {
section.items.forEach((nav, itemIndex) => {
if (nav.subItems && !foundMatch) {
// Check if any subitem matches current path
const shouldOpen = nav.subItems.some((subItem) => {
// Exact match
if (currentPath === subItem.path) return true;
// Prefix match for nested routes (e.g., /sites/123/settings matches /sites)
if (currentPath.startsWith(subItem.path + '/')) return true;
return false;
});
if (shouldOpen) {
setOpenSubmenu({ sectionIndex, itemIndex });
foundMatch = true;
}
}
});
});
// IMPORTANT: Don't close if no match - user may have manually opened
}, [location.pathname, allSections]);
```
---
## Part 5: Django Admin Error Prevention
### 5.1 Safe Admin Actions
Add try/except wrappers to all admin actions to prevent 500 errors:
```python
# In admin/base.py - Enhanced base admin class
class Igny8ModelAdmin(ModelAdmin):
"""Base admin class with error handling"""
def save_model(self, request, obj, form, change):
"""Safe save with error handling"""
try:
super().save_model(request, obj, form, change)
except IntegrityError as e:
messages.error(request, f"Database error: {str(e)}")
except ValidationError as e:
messages.error(request, f"Validation error: {str(e)}")
except Exception as e:
messages.error(request, f"Unexpected error: {str(e)}")
logger.exception(f"Error saving {obj.__class__.__name__}")
def delete_model(self, request, obj):
"""Safe delete with error handling"""
try:
super().delete_model(request, obj)
except ProtectedError as e:
messages.error(request, f"Cannot delete: {str(e)}")
except Exception as e:
messages.error(request, f"Delete error: {str(e)}")
logger.exception(f"Error deleting {obj.__class__.__name__}")
def delete_queryset(self, request, queryset):
"""Safe bulk delete"""
try:
super().delete_queryset(request, queryset)
except ProtectedError as e:
messages.error(request, f"Cannot delete some items: {str(e)}")
except Exception as e:
messages.error(request, f"Bulk delete error: {str(e)}")
```
### 5.2 Admin Error Logging
```python
# Add to all admin classes that process data
class CreditTransactionAdmin(Igny8ModelAdmin):
@admin.action(description='Process selected transactions')
def process_transactions(self, request, queryset):
success_count = 0
error_count = 0
for obj in queryset:
try:
obj.process()
success_count += 1
except Exception as e:
error_count += 1
logger.error(f"Failed to process transaction {obj.id}: {e}")
if success_count:
messages.success(request, f'{success_count} transaction(s) processed.')
if error_count:
messages.warning(request, f'{error_count} transaction(s) failed. Check logs.')
```
---
## Part 6: Migration Plan
### Phase 1: Preparation (Week 1)
1. Add `model_tier` field to `AIModelConfig` - Migration
2. Create `SiteAIAutomationSettings` model - Migration
3. Add deprecation warnings to `GlobalIntegrationSettings`
4. Create unified API endpoint (read-only initially)
### Phase 2: Data Migration (Week 2)
1. Migrate existing `AutomationConfig``SiteAIAutomationSettings`
2. Migrate existing `PublishingSettings``SiteAIAutomationSettings`
3. Migrate existing `SiteAIBudgetAllocation``SiteAIAutomationSettings.budget_allocation`
4. Update automation service to use new model
### Phase 3: Frontend Update (Week 3)
1. Create unified settings component
2. Update sidebar navigation
3. Update Site Settings to use new unified tab
4. Implement model tier dropdowns
### Phase 4: Cleanup (Week 4)
1. Remove old separate settings pages
2. Mark old API endpoints as deprecated
3. Remove `AISettings` model
4. Update documentation
### Phase 5: Final Migration (v2.0)
1. Remove `GlobalIntegrationSettings` model
2. Remove separate `AutomationConfig`, `PublishingSettings` models
3. Remove deprecated API endpoints
---
## Part 7: Testing Checklist
### Backend Tests
- [ ] `SiteAIAutomationSettings` CRUD works correctly
- [ ] Unified API returns complete settings
- [ ] Unified API saves all settings atomically
- [ ] `is_testing` flag works on AIModelConfig
- [ ] Migration scripts run without errors
- [ ] Old models still work during transition
- [ ] SiteIntegration removal doesn't break SyncEvent
### Frontend Tests
- [ ] Site Settings > AI & Automation tab loads correctly
- [ ] Stage matrix displays all 7 stages
- [ ] Testing/Live AI Model dropdown works
- [ ] Budget % validation (must total ≤100%)
- [ ] Save button saves all settings atomically
- [ ] Sidebar stays open on inner pages
- [ ] No 500 errors in admin actions
### Integration Tests
- [ ] Automation runs use new unified settings
- [ ] Publishing uses new settings
- [ ] Credit budget (stage %) works correctly
- [ ] Testing/Live model selection works in AI calls
---
## Part 8: Files to Create/Modify
### Backend Files
| File | Action | Purpose |
|------|--------|---------|
| `business/billing/models.py` | MODIFY | Add `is_testing` field to AIModelConfig |
| `business/automation/models.py` | CREATE | Add `SiteAIAutomationSettings` model |
| `api/unified_settings.py` | CREATE | New unified API endpoint |
| `urls/api_v1.py` | MODIFY | Register new endpoint |
| `business/integration/models.py` | MODIFY | Remove SiteIntegration (Phase 2) |
| `admin/billing.py` | MODIFY | Update Django Admin grouping |
| `migrations/XXXX_*.py` | CREATE | Database migrations |
### Frontend Files
| File | Action | Purpose |
|------|--------|---------|
| `pages/Sites/AIAutomationSettings.tsx` | CREATE | Main unified settings page |
| `components/Settings/StageMatrix.tsx` | CREATE | Reusable stage matrix component |
| `components/Settings/AIModelSelect.tsx` | CREATE | Testing/Live dropdown component |
| `services/unifiedSettingsService.ts` | CREATE | API service for unified endpoint |
| `pages/Sites/SiteSettings.tsx` | MODIFY | Add AI & Automation tab |
| `layout/AppSidebar.tsx` | MODIFY | Keep dropdowns open on inner pages |
### Files to Remove (Phase 2)
| File | Reason |
|------|--------|
| Separate automation settings pages | Replaced by unified page |
| Old settings API endpoints | Deprecated, use unified |
---
## Part 9: Quick Setup Presets (Future Enhancement)
```
┌─── Quick Setup Presets ─────────────────────────────────────────┐
│ │
│ [ ] Conservative - Low throughput, high quality │
│ 5 tasks/run, 3 images/run, 30min stagger │
│ │
│ [●] Balanced - Moderate throughput (Recommended) │
│ 10 tasks/run, 20 images/run, 15min stagger │
│ │
│ [ ] Aggressive - Maximum throughput │
│ Unlimited, 5min stagger │
│ │
│ [ ] Custom - Configure manually │
│ │
└──────────────────────────────────────────────────────────────────┘
```
---
## Summary
**Before:** 17 models, 40+ settings, 4+ pages to configure one workflow
**After:** ~10 models, unified page, matrix view, clear hierarchy
| Metric | Before | After |
|--------|--------|-------|
| Settings Models | 17 | ~10 |
| Frontend Pages for Config | 4+ | 1 |
| API Calls to Load | 5+ | 1 |
| API Calls to Save | 5+ | 1 |
| AI Model Options | GPT-4.1, Claude, etc. (confusing) | Testing / Live (simple) |
| Credit Budget Config | Manual max + allocation | Auto (all credits) + stage % |
| User Confusion | High | Low |
### Key Simplifications
1. **AI Models:** Just "Testing" vs "Live" - no quality tiers
2. **Credit Budget:** Always enabled, uses all available credits
3. **Publishing:** Only 3 modes + queue limit (removed daily/weekly targets)
4. **Per-Run Limit:** Clear - applies to INPUT items per stage
5. **SiteIntegration:** REMOVED - credentials in Site model