Files

IGNY8 VPS (Salman) 6e30d2d4e8 Django admin cleanup

2026-01-04 06:04:37 +00:00

35 KiB

Raw Blame History

Safe Migration & Testing Plan

Overview

This document ensures the refactor is done safely with zero breakage. All existing functionality must work identically before and after migration.

Current Working System Components

1. AI Functions (Core - Used by ALL paths)

All AI operations go through these functions in backend/igny8_core/ai/functions/:

Function	File	Operation Type	Credits	Called Via
Auto Cluster	`auto_cluster.py`	`clustering`	Token-based	AIEngine
Generate Ideas	`generate_ideas.py`	`idea_generation`	Token-based	AIEngine
Generate Content	`generate_content.py`	`content_generation`	Token-based	AIEngine
Generate Image Prompts	`generate_image_prompts.py`	`image_prompt_extraction`	Token-based	AIEngine
Generate Images	`generate_images.py`	`image_generation`	Per-image	AICore direct
Optimize Content	`optimize_content.py`	`content_optimization`	Token-based	OptimizerService

Operation Type Mapping (in engine.py):

mapping = {
    'auto_cluster': 'clustering',
    'generate_ideas': 'idea_generation',
    'generate_content': 'content_generation',
    'generate_image_prompts': 'image_prompt_extraction',
    'generate_images': 'image_generation',
    'generate_site_structure': 'site_structure_generation',
}

1.1 Non-AIEngine Credit Deduction Points

Service	Operation Type	Direct Credit Deduction
LinkerService	`internal_linking`	✅ Uses CreditService directly
OptimizerService	`content_optimization`	✅ Uses CreditService directly

2. Execution Paths (All use same AI functions)

┌─────────────────────────────────────────────────────────────────────────┐
│                    ALL EXECUTION PATHS                                   │
└─────────────────────────────────────────────────────────────────────────┘

PATH 1: Manual Module Page Buttons (via run_ai_task Celery task)
├── Planner Page → "Cluster Keywords" button → api/planner.py → run_ai_task.delay('auto_cluster')
├── Planner Page → "Generate Ideas" button → api/planner.py → run_ai_task.delay('generate_ideas')
├── Writer Page → "Generate Content" button → api/writer.py → run_ai_task.delay('generate_content')
├── Writer Page → "Extract Prompts" button → api/writer.py → run_ai_task.delay('generate_image_prompts')
├── Writer Page → "Generate Images" button → api/writer.py → process_image_generation_queue.delay()
└── All go through: Celery task → AIEngine → AI Function → AICore → API → CreditService

PATH 2: Automation Page Manual Run
├── Automation Page → "Run Now" button
├── automation/tasks.py → run_automation_task
├── AutomationService.run_automation()
│   Stage 1: _run_clustering() → AIEngine.execute('auto_cluster')
│   Stage 2: _run_idea_generation() → AIEngine.execute('generate_ideas')
│   Stage 3: _run_task_creation() → Database only (no AI)
│   Stage 4: _run_content_generation() → AIEngine.execute('generate_content')
│   Stage 5: _run_image_prompt_extraction() → AIEngine.execute('generate_image_prompts')
│   Stage 6: _run_image_generation() → process_image_generation_queue.delay()
│   Stage 7: _run_publishing() → WordPress API (no AI)
└── Same credit deduction path (stages 1,2,4,5 via AIEngine, stage 6 via AICore)

PATH 3: Scheduled Automation Run
├── Celery Beat scheduler → check_scheduled_automations task (hourly)
├── automation_tasks.py → run_automation_task.delay()
├── AutomationService.run_automation()
├── Calls same AI functions through AIEngine
└── Same credit deduction path

PATH 4: Direct Service Operations (Bypass AIEngine)
├── Linker Module → LinkerService → CreditService.deduct_credits_for_operation('internal_linking')
├── Optimizer Module → OptimizerService → CreditService.deduct_credits_for_operation('content_optimization')
└── These use direct credit deduction, not via AIEngine

3. Credit Flow (Current)

┌─────────────────────────────────────────────────────────────────────────┐
│                    CURRENT CREDIT FLOW                                   │
└─────────────────────────────────────────────────────────────────────────┘

For TEXT operations (clustering, ideas, content) - VIA AIEngine:
1. API endpoint receives request
2. Celery task run_ai_task triggered
3. AIEngine.execute(function_name, payload)
   a. Step 2.5: CreditService.check_credits() - PRE-CHECK
   b. Step 3: AICore.run_ai_request() - makes OpenAI API call
   c. Response includes token usage (input_tokens, output_tokens)
   d. Step 5.5: CreditService.deduct_credits_for_operation() - DEDUCTION
4. CreditService.calculate_credits_from_tokens() uses CreditCostConfig
5. Creates CreditTransaction (balance history)
6. Creates CreditUsageLog (detailed tracking)
7. Updates Account.credits

For IMAGE operations - VIA process_image_generation_queue:
1. API endpoint receives request
2. Celery task process_image_generation_queue triggered
3. For each image: AICore.generate_image()
   a. Makes Runware/OpenAI API call
   b. Calculates cost via ModelRegistry.calculate_cost()
   c. Returns image URL
4. ⚠️ Credit deduction handled AFTER all images generated
5. Uses CreditCostConfig.image_generation entry (min_credits, tokens_per_credit)
6. Creates CreditUsageLog

For DIRECT SERVICE OPERATIONS (Linker, Optimizer):
1. Service method called (e.g., LinkerService.generate_links())
2. AI call made directly to AICore or external service
3. CreditService.deduct_credits_for_operation() called directly
4. Uses operation-specific CreditCostConfig entry

3.1 Celery Task Definitions

Task File	Task Name	Entry Point For
`ai/tasks.py`	`run_ai_task`	All manual AI buttons (universal task)
`ai/tasks.py`	`process_image_generation_queue`	Image generation queue
`automation/tasks.py`	`check_scheduled_automations`	Hourly scheduler check
`automation/tasks.py`	`run_automation_task`	Full automation pipeline
`automation/tasks.py`	`resume_paused_automation`	Resume after pause

4. Key Files Currently Working

File	Purpose	Must Keep Working
`ai/engine.py`	Orchestrates AI functions	✅
`ai/ai_core.py`	API calls, key loading	✅
`ai/model_registry.py`	Model config lookup	✅
`ai/settings.py`	get_model_config()	✅
`ai/tasks.py`	Celery tasks	✅
`ai/functions/*.py`	All AI functions	✅
`billing/services/credit_service.py`	Credit calculation	✅
`billing/models.py`	CreditUsageLog, CreditCostConfig	✅
`automation/services/automation_service.py`	Automation runner	✅
`modules/system/global_settings_models.py`	API keys, defaults	✅

Potential Issues to Verify Before Migration

⚠️ Issue 1: Image Credit Deduction May Be Incomplete

The process_image_generation_queue task calculates cost via ModelRegistry.calculate_cost() but credit deduction needs verification:

Does it call CreditService.deduct_credits_for_operation()?
Or does it just log cost without deducting credits?

Verification Query:

docker exec igny8_backend python manage.py shell -c "
from igny8_core.business.billing.models import CreditUsageLog
logs = CreditUsageLog.objects.filter(operation_type='image_generation').order_by('-created_at')[:5]
for log in logs:
    print(f'{log.operation_type}: {log.credits_used} credits, cost=${log.cost_usd}, model={log.model_used}')
if not logs:
    print('⚠️ NO image_generation credit logs found - may not be tracking credits!')
"

⚠️ Issue 2: GlobalIntegrationSettings.runware_model Points to Non-Existent Model

Current value: bria:10@1 - but this model doesn't exist in AIModelConfig. This may cause fallback issues.

⚠️ Issue 3: Multiple Credit Calculation Paths

Credits are calculated in different ways depending on the path:

Via AIEngine → uses CreditCostConfig token-based calculation
Via direct service → uses CreditCostConfig with operation-specific rates
Via image generation → may use different calculation

Need to unify after migration.

Pre-Migration Baseline Capture

Step 1: Database State Snapshot

Run this BEFORE any changes to establish baseline:

docker exec igny8_backend python manage.py shell << 'EOF'
import json
from datetime import datetime

print("=" * 80)
print(f"BASELINE SNAPSHOT - {datetime.now().isoformat()}")
print("=" * 80)

# 1. AIModelConfig
print("\n=== AIModelConfig ===")
from igny8_core.business.billing.models import AIModelConfig
for m in AIModelConfig.objects.all().order_by('model_type', 'sort_order'):
    print(json.dumps({
        'model_name': m.model_name,
        'model_type': m.model_type,
        'provider': m.provider,
        'is_active': m.is_active,
        'is_default': m.is_default,
        'cost_per_image': str(m.cost_per_image) if m.cost_per_image else None,
        'input_cost_per_1m': str(m.input_cost_per_1m) if m.input_cost_per_1m else None,
        'output_cost_per_1m': str(m.output_cost_per_1m) if m.output_cost_per_1m else None,
    }))

# 2. CreditCostConfig
print("\n=== CreditCostConfig ===")
from igny8_core.business.billing.models import CreditCostConfig
for c in CreditCostConfig.objects.filter(is_active=True):
    print(json.dumps({
        'operation_type': c.operation_type,
        'tokens_per_credit': c.tokens_per_credit,
        'min_credits': c.min_credits,
        'price_per_credit_usd': str(c.price_per_credit_usd),
    }))

# 3. GlobalIntegrationSettings
print("\n=== GlobalIntegrationSettings ===")
from igny8_core.modules.system.global_settings_models import GlobalIntegrationSettings
g = GlobalIntegrationSettings.get_instance()
print(json.dumps({
    'openai_model': g.openai_model,
    'openai_temperature': g.openai_temperature,
    'openai_max_tokens': g.openai_max_tokens,
    'default_text_provider': g.default_text_provider,
    'dalle_model': g.dalle_model,
    'runware_model': g.runware_model,
    'default_image_service': g.default_image_service,
    'image_style': g.image_style,
    'max_in_article_images': g.max_in_article_images,
    'has_openai_key': bool(g.openai_api_key),
    'has_runware_key': bool(g.runware_api_key),
}))

# 4. Recent Credit Usage (last 20)
print("\n=== Recent CreditUsageLog (last 20) ===")
from igny8_core.business.billing.models import CreditUsageLog
for log in CreditUsageLog.objects.all().order_by('-created_at')[:20]:
    print(json.dumps({
        'operation_type': log.operation_type,
        'credits_used': log.credits_used,
        'model_used': log.model_used,
        'tokens_input': log.tokens_input,
        'tokens_output': log.tokens_output,
        'cost_usd': str(log.cost_usd) if log.cost_usd else None,
    }))

# 5. Account Credits
print("\n=== Account Credits ===")
from igny8_core.auth.models import Account
for acc in Account.objects.all()[:5]:
    print(json.dumps({
        'account_id': acc.id,
        'name': acc.name,
        'credits': acc.credits,
    }))

print("\n" + "=" * 80)
print("BASELINE CAPTURE COMPLETE")
print("=" * 80)
EOF

Save this output to: 4th-jan-refactor/baseline-snapshot.json

Integration Tests (Must Pass Before AND After)

Test File: `backend/igny8_core/tests/test_ai_system_integration.py`

"""
AI System Integration Tests
===========================
These tests verify the entire AI pipeline works end-to-end.
Run BEFORE migration to establish baseline.
Run AFTER each phase to verify nothing broke.

Usage:
    docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration
"""
from decimal import Decimal
from django.test import TestCase, TransactionTestCase
from django.db import transaction
from unittest.mock import patch, MagicMock
import json


class AIModelConfigTests(TestCase):
    """Test AIModelConfig loading and access"""
    
    def test_text_models_exist(self):
        """All required text models exist and are configured"""
        from igny8_core.business.billing.models import AIModelConfig
        
        required_text_models = ['gpt-5.1', 'gpt-4o-mini']
        for model_name in required_text_models:
            model = AIModelConfig.objects.filter(model_name=model_name).first()
            self.assertIsNotNone(model, f"Text model {model_name} not found")
            self.assertEqual(model.model_type, 'text')
    
    def test_image_models_exist(self):
        """All required image models exist and are configured"""
        from igny8_core.business.billing.models import AIModelConfig
        
        required_image_models = ['runware:97@1', 'dall-e-3', 'google:4@2']
        for model_name in required_image_models:
            model = AIModelConfig.objects.filter(model_name=model_name).first()
            self.assertIsNotNone(model, f"Image model {model_name} not found")
            self.assertEqual(model.model_type, 'image')
            self.assertIsNotNone(model.cost_per_image, f"{model_name} missing cost_per_image")
    
    def test_default_text_model_exists(self):
        """Exactly one default text model is set"""
        from igny8_core.business.billing.models import AIModelConfig
        
        defaults = AIModelConfig.objects.filter(model_type='text', is_default=True, is_active=True)
        self.assertEqual(defaults.count(), 1, "Should have exactly 1 default text model")
        self.assertEqual(defaults.first().model_name, 'gpt-5.1')
    
    def test_default_image_model_exists(self):
        """Exactly one default image model is set"""
        from igny8_core.business.billing.models import AIModelConfig
        
        defaults = AIModelConfig.objects.filter(model_type='image', is_default=True, is_active=True)
        self.assertEqual(defaults.count(), 1, "Should have exactly 1 default image model")


class ModelRegistryTests(TestCase):
    """Test ModelRegistry functionality"""
    
    def test_get_model_from_db(self):
        """ModelRegistry.get_model() returns model from database"""
        from igny8_core.ai.model_registry import ModelRegistry
        
        model = ModelRegistry.get_model('gpt-5.1')
        self.assertIsNotNone(model)
        # Should be AIModelConfig instance or dict with model_name
        if hasattr(model, 'model_name'):
            self.assertEqual(model.model_name, 'gpt-5.1')
        else:
            self.assertEqual(model.get('model_name'), 'gpt-5.1')
    
    def test_get_image_model(self):
        """ModelRegistry returns image models correctly"""
        from igny8_core.ai.model_registry import ModelRegistry
        
        for model_name in ['runware:97@1', 'dall-e-3', 'google:4@2']:
            model = ModelRegistry.get_model(model_name)
            self.assertIsNotNone(model, f"Model {model_name} not found")
    
    def test_calculate_cost_text(self):
        """Cost calculation for text models works"""
        from igny8_core.ai.model_registry import ModelRegistry
        
        cost = ModelRegistry.calculate_cost('gpt-5.1', input_tokens=1000, output_tokens=500)
        self.assertIsInstance(cost, (int, float, Decimal))
        self.assertGreater(cost, 0)
    
    def test_calculate_cost_image(self):
        """Cost calculation for image models works"""
        from igny8_core.ai.model_registry import ModelRegistry
        
        cost = ModelRegistry.calculate_cost('dall-e-3', num_images=1)
        self.assertIsInstance(cost, (int, float, Decimal))
        self.assertGreater(cost, 0)


class APIKeyLoadingTests(TestCase):
    """Test API key loading from GlobalIntegrationSettings"""
    
    def test_openai_key_loads(self):
        """OpenAI API key loads from GlobalIntegrationSettings"""
        from igny8_core.ai.ai_core import AICore
        
        ai_core = AICore()
        key = ai_core.get_api_key('openai')
        self.assertIsNotNone(key, "OpenAI API key not configured")
        self.assertTrue(len(key) > 10, "OpenAI API key too short")
    
    def test_runware_key_loads(self):
        """Runware API key loads from GlobalIntegrationSettings"""
        from igny8_core.ai.ai_core import AICore
        
        ai_core = AICore()
        key = ai_core.get_api_key('runware')
        self.assertIsNotNone(key, "Runware API key not configured")


class CreditServiceTests(TestCase):
    """Test credit calculation and deduction"""
    
    def test_calculate_credits_text_operation(self):
        """Credit calculation for text operations works"""
        from igny8_core.business.billing.services.credit_service import CreditService
        
        # Test content generation
        credits = CreditService.calculate_credits_from_tokens(
            'content_generation',
            tokens_input=1000,
            tokens_output=2000
        )
        self.assertIsInstance(credits, int)
        self.assertGreater(credits, 0)
    
    def test_calculate_credits_image_operation(self):
        """Credit calculation for image operations works"""
        from igny8_core.business.billing.services.credit_service import CreditService
        
        # Image generation uses min_credits from CreditCostConfig
        credits = CreditService.calculate_credits_from_tokens(
            'image_generation',
            tokens_input=0,
            tokens_output=0
        )
        self.assertIsInstance(credits, int)
        self.assertGreaterEqual(credits, 1)
    
    def test_credit_cost_config_exists(self):
        """All required CreditCostConfig entries exist"""
        from igny8_core.business.billing.models import CreditCostConfig
        
        required_ops = ['clustering', 'idea_generation', 'content_generation', 
                       'image_generation', 'image_prompt_extraction']
        for op in required_ops:
            config = CreditCostConfig.objects.filter(operation_type=op, is_active=True).first()
            self.assertIsNotNone(config, f"CreditCostConfig for {op} not found")


class AISettingsTests(TestCase):
    """Test get_model_config() function"""
    
    def test_get_model_config_returns_config(self):
        """get_model_config() returns valid configuration"""
        from igny8_core.ai.settings import get_model_config
        from igny8_core.auth.models import Account
        
        account = Account.objects.first()
        if account:
            config = get_model_config('content_generation', account)
            
            self.assertIn('model', config)
            self.assertIn('max_tokens', config)
            self.assertIn('temperature', config)
            self.assertIsNotNone(config['model'])


class AIFunctionValidationTests(TestCase):
    """Test AI function classes load correctly"""
    
    def test_auto_cluster_function_loads(self):
        """AutoClusterFunction loads and validates"""
        from igny8_core.ai.functions.auto_cluster import AutoClusterFunction
        
        fn = AutoClusterFunction()
        self.assertEqual(fn.get_name(), 'auto_cluster')
        self.assertIsNotNone(fn.get_metadata())
    
    def test_generate_ideas_function_loads(self):
        """GenerateIdeasFunction loads and validates"""
        from igny8_core.ai.functions.generate_ideas import GenerateIdeasFunction
        
        fn = GenerateIdeasFunction()
        self.assertEqual(fn.get_name(), 'generate_ideas')
    
    def test_generate_content_function_loads(self):
        """GenerateContentFunction loads and validates"""
        from igny8_core.ai.functions.generate_content import GenerateContentFunction
        
        fn = GenerateContentFunction()
        self.assertEqual(fn.get_name(), 'generate_content')
    
    def test_generate_images_function_loads(self):
        """GenerateImagesFunction loads and validates"""
        from igny8_core.ai.functions.generate_images import GenerateImagesFunction
        
        fn = GenerateImagesFunction()
        self.assertEqual(fn.get_name(), 'generate_images')


class GlobalIntegrationSettingsTests(TestCase):
    """Test GlobalIntegrationSettings singleton"""
    
    def test_singleton_loads(self):
        """GlobalIntegrationSettings singleton loads"""
        from igny8_core.modules.system.global_settings_models import GlobalIntegrationSettings
        
        settings = GlobalIntegrationSettings.get_instance()
        self.assertIsNotNone(settings)
        self.assertEqual(settings.pk, 1)
    
    def test_image_settings_exist(self):
        """Image settings are configured"""
        from igny8_core.modules.system.global_settings_models import GlobalIntegrationSettings
        
        settings = GlobalIntegrationSettings.get_instance()
        self.assertIsNotNone(settings.image_style)
        self.assertIsNotNone(settings.max_in_article_images)
        self.assertGreater(settings.max_in_article_images, 0)


class AutomationServiceTests(TestCase):
    """Test AutomationService uses same AI functions"""
    
    def test_automation_service_imports(self):
        """AutomationService can import AI functions"""
        from igny8_core.business.automation.services.automation_service import AutomationService
        
        # Just verify the import works and class exists
        self.assertIsNotNone(AutomationService)

Running Tests

# Run all integration tests
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration -v 2

# Run specific test class
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration.AIModelConfigTests -v 2

# Run with coverage
docker exec igny8_backend coverage run manage.py test igny8_core.tests.test_ai_system_integration
docker exec igny8_backend coverage report

Phased Migration with Verification

Phase 1: Add New Models (NO BREAKING CHANGES)

Changes:

Add IntegrationProvider model
Add new fields to AIModelConfig: credits_per_image, tokens_per_credit, quality_tier
Create migrations
Populate new data

Verification:

# 1. Run migrations
docker exec igny8_backend python manage.py makemigrations
docker exec igny8_backend python manage.py migrate

# 2. Run ALL tests - must pass
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration -v 2

# 3. Manual verification - existing features still work
# - Go to Writer page, generate content (check credits deducted)
# - Go to Writer page, generate images (check credits deducted)
# - Run automation manually (check it completes)

Rollback: Delete new tables/fields (no existing data affected)

Phase 2: Add Parallel Code Paths (OLD KEEPS WORKING)

Changes:

Add new methods to ModelRegistry (keep old ones)
Add new methods to CreditService (keep old ones)
Add comparison logging

Pattern:

# model_registry.py
@classmethod
def get_default_model_NEW(cls, model_type: str):
    """NEW: Get default model from AIModelConfig.is_default"""
    from igny8_core.business.billing.models import AIModelConfig
    return AIModelConfig.objects.filter(
        model_type=model_type, 
        is_default=True, 
        is_active=True
    ).first()

@classmethod  
def get_model(cls, model_id: str):
    """EXISTING: Keep working exactly as before"""
    # ... existing code unchanged ...

# credit_service.py
@staticmethod
def calculate_credits_for_image_NEW(model_name: str, num_images: int) -> int:
    """NEW: Calculate from AIModelConfig.credits_per_image"""
    from igny8_core.business.billing.models import AIModelConfig
    try:
        model = AIModelConfig.objects.get(model_name=model_name, is_active=True)
        if model.credits_per_image:
            return model.credits_per_image * num_images
    except AIModelConfig.DoesNotExist:
        pass
    return None  # Signal to use old method

@staticmethod
def calculate_credits_from_tokens(operation_type, tokens_input, tokens_output):
    """EXISTING: Keep working, add comparison logging"""
    # Calculate old way
    old_result = cls._calculate_old(operation_type, tokens_input, tokens_output)
    
    # Calculate new way (for comparison only)
    new_result = cls._calculate_new(operation_type, tokens_input, tokens_output)
    
    # Log if different
    if old_result != new_result:
        logger.warning(f"[MIGRATION] Credit calc mismatch: {operation_type} old={old_result} new={new_result}")
    
    # Return OLD (safe) for now
    return old_result

Verification:

# 1. Run tests
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration -v 2

# 2. Check logs for any mismatch warnings
docker logs igny8_backend 2>&1 | grep "MIGRATION"

# 3. Manual test all paths

Rollback: Remove new methods only

Phase 3: Add IntegrationProvider for API Keys (OLD FALLBACK)

Changes:

AICore._load_account_settings() tries IntegrationProvider first, falls back to GlobalIntegrationSettings

Pattern:

# ai_core.py
def _load_account_settings(self):
    """Load API keys - try new IntegrationProvider, fallback to old"""
    
    # Try NEW way first
    try:
        from igny8_core.business.billing.models import IntegrationProvider
        openai_provider = IntegrationProvider.objects.filter(
            provider_id='openai', is_active=True
        ).first()
        if openai_provider and openai_provider.api_key:
            self._openai_api_key = openai_provider.api_key
            logger.info("[MIGRATION] Loaded OpenAI key from IntegrationProvider")
        else:
            raise Exception("Fallback to old method")
    except Exception:
        # FALLBACK to old GlobalIntegrationSettings
        from igny8_core.modules.system.global_settings_models import GlobalIntegrationSettings
        global_settings = GlobalIntegrationSettings.get_instance()
        self._openai_api_key = global_settings.openai_api_key
        logger.info("[MIGRATION] Loaded OpenAI key from GlobalIntegrationSettings (fallback)")
    
    # Same pattern for runware...

Verification:

# 1. Run tests
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration -v 2

# 2. Verify API calls work
# - Generate content (uses OpenAI)
# - Generate images with Runware model
# - Generate images with DALL-E model

# 3. Check logs
docker logs igny8_backend 2>&1 | grep "MIGRATION"

Rollback: Revert _load_account_settings() to old version

Phase 4: Switch to New Credit Calculation

Changes:

Use AIModelConfig.credits_per_image for image operations
Use AIModelConfig.tokens_per_credit for text operations
Keep CreditCostConfig as fallback

Verification:

# 1. Before switching, compare calculations for 1 week
# Log both old and new results, verify they match

# 2. Run tests
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration -v 2

# 3. Verify credit logs show correct values
docker exec igny8_backend python manage.py shell -c "
from igny8_core.business.billing.models import CreditUsageLog
for log in CreditUsageLog.objects.order_by('-created_at')[:10]:
    print(f'{log.operation_type}: {log.credits_used} credits, model={log.model_used}')
"

Phase 5: Create API Endpoint

Changes:

Add /api/v1/system/ai-models/ endpoint
Returns models from database

Verification:

# 1. Test endpoint
curl -H "Authorization: Bearer $TOKEN" http://localhost:8000/api/v1/system/ai-models/

# 2. Verify response structure
# Should have text_models, image_models, image_settings

Phase 6: Update Frontend

Changes:

Remove hardcoded model choices from Settings.tsx
Load from API

Verification:

# 1. Frontend tests
cd frontend && npm run test

# 2. Manual verification
# - Open Sites > Settings > Image Settings
# - Verify dropdown shows models from API
# - Change model, save, verify it persists

Phase 7: Cleanup (After 1 Week Stable)

Remove from GlobalIntegrationSettings:

# API key fields (moved to IntegrationProvider)
- openai_api_key
- anthropic_api_key  # unused
- bria_api_key       # unused
- runware_api_key

# Model selection fields (now from AIModelConfig.is_default)
- openai_model
- dalle_model
- runware_model
- default_text_provider
- default_image_service

# Hardcoded CHOICES
- OPENAI_MODEL_CHOICES
- DALLE_MODEL_CHOICES
- RUNWARE_MODEL_CHOICES
- TEXT_PROVIDER_CHOICES
- IMAGE_SERVICE_CHOICES

Remove from constants.py:

- MODEL_RATES = {...}
- IMAGE_MODEL_RATES = {...}

Remove from model_registry.py:

- Fallback to constants.py (all lookups must use DB)
- Hardcoded default model names

Remove from ai_core.py:

- IMAGE_MODEL_RATES import and usage
- Direct GlobalIntegrationSettings key access (use IntegrationProvider)

Remove CreditCostConfig entries (optional - use AIModelConfig):

- image_generation entry (use AIModelConfig.credits_per_image instead)

Remove from frontend Settings.tsx:

- QUALITY_TO_CONFIG hardcoded mapping
- RUNWARE_MODEL_CHOICES hardcoded array
- DALLE_MODEL_CHOICES hardcoded array
- MODEL_LANDSCAPE_SIZES hardcoded mapping
- Any hardcoded model names

Verification:

# 1. Run full test suite
docker exec igny8_backend python manage.py test

# 2. Run integration tests
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration -v 2

# 3. Full manual test of all paths

Complete Test Checklist

Manual Test Checklist (Run After Each Phase)

Path 1: Manual Module Page Actions

Test	Page	Button/Action	Expected Result	Verify Credits	Pass?
Cluster Keywords	Planner	"Cluster" button	Creates clusters	Check CreditUsageLog for `clustering`	☐
Generate Ideas	Planner	"Generate Ideas" button	Creates ideas	Check CreditUsageLog for `idea_generation`	☐
Generate Content	Writer	"Generate" button	Creates content	Check CreditUsageLog for `content_generation`	☐
Generate Product Content	Writer	"Generate Product" button	Creates product content	Check CreditUsageLog for `content_generation`	☐
Generate Service Content	Writer	"Generate Service" button	Creates service content	Check CreditUsageLog for `content_generation`	☐
Extract Image Prompts	Writer	"Extract Prompts" button	Creates prompts	Check CreditUsageLog for `image_prompt_extraction`	☐
Generate Images (Basic)	Writer	Image gen w/ Runware basic	Creates images	Check CreditUsageLog: 1 credit/image	☐
Generate Images (Quality)	Writer	Image gen w/ DALL-E	Creates images	Check CreditUsageLog: 5 credits/image	☐
Generate Images (Premium)	Writer	Image gen w/ Google	Creates images	Check CreditUsageLog: 15 credits/image	☐
Internal Linking	Linker	"Generate Links" button	Creates links	Check CreditUsageLog for `internal_linking`	☐
Optimize Content	Optimizer	"Optimize" button	Optimizes content	Check CreditUsageLog for `content_optimization`	☐

Path 2: Automation Manual Run

Test	Page	Action	Expected Result	Verify Credits	Pass?
Run Full Automation	Automation	"Run Now" button	All stages complete	Credits for each stage	☐
Run Stage 1 Only	Automation	Run clustering stage	Clusters created	Check `clustering` credits	☐
Run Stage 2 Only	Automation	Run idea stage	Ideas created	Check `idea_generation` credits	☐
Run Stage 4 Only	Automation	Run content stage	Content created	Check `content_generation` credits	☐
Run Stage 5 Only	Automation	Run prompts stage	Prompts created	Check `image_prompt_extraction` credits	☐
Run Stage 6 Only	Automation	Run image stage	Images created	Check `image_generation` credits	☐
Pause/Resume	Automation	Pause then resume	Continues correctly	No double-charging	☐

Path 3: Scheduled Automation

Test	Setup	Expected	Verify	Pass?
Schedule Triggers	Set schedule for automation	Runs on schedule	Check logs at scheduled time	☐
Credits Deducted	After scheduled run	Credits reduced	Check account balance	☐
Multiple Automations	Multiple scheduled	All run	Each deducts credits	☐

Credit Verification Queries

# Check recent credit usage logs
docker exec igny8_backend python manage.py shell -c "
from igny8_core.business.billing.models import CreditUsageLog
for log in CreditUsageLog.objects.order_by('-created_at')[:20]:
    print(f'{log.created_at}: {log.operation_type} | {log.credits_used} credits | model={log.model_used} | tokens={log.tokens_input}+{log.tokens_output}')
"

# Check account balance
docker exec igny8_backend python manage.py shell -c "
from igny8_core.auth.models import Account
for acc in Account.objects.all()[:5]:
    print(f'{acc.name}: {acc.credits} credits')
"

# Check credit transactions
docker exec igny8_backend python manage.py shell -c "
from igny8_core.business.billing.models import CreditTransaction
for txn in CreditTransaction.objects.order_by('-created_at')[:10]:
    print(f'{txn.created_at}: {txn.transaction_type} | {txn.credits_amount} | balance={txn.balance_after}')
"

Frontend Verification

Test	Page	Action	Expected	Pass?
Settings Load	Sites > Settings	Open page	Image settings dropdown populated	☐
Model Selection	Sites > Settings	Change image model	Shows Basic/Quality/Premium	☐
Model Persists	Sites > Settings	Save and reload	Selected model persists	☐
Credits Display	Any page	Header/sidebar	Shows correct credit balance	☐
Usage Analytics	Analytics	View usage	Shows correct breakdown	☐

Rollback Procedures

If Phase 1 Fails

# Remove new migrations
docker exec igny8_backend python manage.py migrate billing <previous_migration>
docker exec igny8_backend python manage.py migrate system <previous_migration>

If Phase 2-6 Fails

# Revert code changes via git
git checkout -- backend/igny8_core/ai/
git checkout -- backend/igny8_core/business/billing/

# Restart containers
docker-compose restart backend

If Phase 7 Fails (Cleanup)

# This is why we wait 1 week before cleanup
# Restore from backup or revert git

Success Criteria

Migration is COMPLETE when:

✅ All integration tests pass
✅ All manual tests pass
✅ No errors in logs for 1 week
✅ Credit calculations match expected values
✅ All execution paths work (manual, automation, scheduled)
✅ Frontend loads models from API
✅ No legacy code remains:
- No MODEL_RATES / IMAGE_MODEL_RATES in constants.py
- No API keys in GlobalIntegrationSettings
- No hardcoded models in frontend
- No fallback to constants in ModelRegistry

35 KiB Raw Blame History

Safe Migration & Testing Plan

Overview

Current Working System Components

1. AI Functions (Core - Used by ALL paths)

1.1 Non-AIEngine Credit Deduction Points

2. Execution Paths (All use same AI functions)

3. Credit Flow (Current)

3.1 Celery Task Definitions

4. Key Files Currently Working

Potential Issues to Verify Before Migration

⚠️ Issue 1: Image Credit Deduction May Be Incomplete

⚠️ Issue 2: GlobalIntegrationSettings.runware_model Points to Non-Existent Model

⚠️ Issue 3: Multiple Credit Calculation Paths

Pre-Migration Baseline Capture

Step 1: Database State Snapshot

Integration Tests (Must Pass Before AND After)

Test File: backend/igny8_core/tests/test_ai_system_integration.py

Running Tests

Phased Migration with Verification

Phase 1: Add New Models (NO BREAKING CHANGES)

Phase 2: Add Parallel Code Paths (OLD KEEPS WORKING)

Phase 3: Add IntegrationProvider for API Keys (OLD FALLBACK)

Phase 4: Switch to New Credit Calculation

Phase 5: Create API Endpoint

Phase 6: Update Frontend

Phase 7: Cleanup (After 1 Week Stable)

Complete Test Checklist

Manual Test Checklist (Run After Each Phase)

Path 1: Manual Module Page Actions

Path 2: Automation Manual Run

Path 3: Scheduled Automation

Credit Verification Queries

Frontend Verification

Rollback Procedures

If Phase 1 Fails

If Phase 2-6 Fails

If Phase 7 Fails (Cleanup)

Success Criteria

35 KiB

Raw Blame History

Test File: `backend/igny8_core/tests/test_ai_system_integration.py`