refactor-4th-jan-plan
This commit is contained in:
IGNY8 VPS (Salman)
214
4th-jan-refactor/REFACTOR-OVERVIEW.md
Normal file
214
4th-jan-refactor/REFACTOR-OVERVIEW.md
Normal file
@@ -0,0 +1,214 @@
|
||||
# AI Configuration Refactor - Master Overview
|
||||
|
||||
**Created:** January 4, 2026
|
||||
**Goal:** Clean, unified AI configuration with single source of truth
|
||||
|
||||
---
|
||||
|
||||
## Documentation Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| [REFACTOR-OVERVIEW.md](REFACTOR-OVERVIEW.md) | This file - master overview |
|
||||
| [implementation-plan-for-ai-models-and-cost.md](implementation-plan-for-ai-models-and-cost.md) | Detailed implementation plan with schema changes |
|
||||
| [safe-migration-and-testing-plan.md](safe-migration-and-testing-plan.md) | Testing strategy and verification steps |
|
||||
|
||||
---
|
||||
|
||||
## What We're Fixing
|
||||
|
||||
### Current Problems
|
||||
|
||||
1. **Duplicate configuration sources**
|
||||
- `AIModelConfig` (database) has model definitions
|
||||
- `GlobalIntegrationSettings` (database) has hardcoded CHOICES duplicating the same info
|
||||
- `constants.py` has `MODEL_RATES` and `IMAGE_MODEL_RATES` hardcoded
|
||||
- Frontend `Settings.tsx` has hardcoded model choices
|
||||
|
||||
2. **Broken/inconsistent data**
|
||||
- `GlobalIntegrationSettings.runware_model = "bria:10@1"` but this model doesn't exist
|
||||
- `GlobalIntegrationSettings.openai_model = "gpt-4o-mini"` but default should be `gpt-5.1`
|
||||
|
||||
3. **API keys mixed with model config**
|
||||
- `GlobalIntegrationSettings` stores both API keys and model preferences
|
||||
- No separation of concerns
|
||||
- Can't easily extend to other integrations (email, payment, etc.)
|
||||
|
||||
4. **Credit calculation scattered**
|
||||
- `CreditCostConfig` for token-based operations
|
||||
- `ModelRegistry.calculate_cost()` for image costs
|
||||
- No clear per-model credit rate
|
||||
|
||||
---
|
||||
|
||||
## Target State
|
||||
|
||||
### Single Source of Truth
|
||||
|
||||
```
|
||||
IntegrationProvider (NEW)
|
||||
├── openai (api_key, active)
|
||||
├── runware (api_key, active)
|
||||
├── resend (api_key, active) ← future
|
||||
├── stripe (api_key, webhook_secret, active) ← future
|
||||
└── ...
|
||||
|
||||
AIModelConfig (ENHANCED)
|
||||
├── Text Models
|
||||
│ ├── gpt-5.1 (is_default=true, tokens_per_credit=1000)
|
||||
│ ├── gpt-4o (tokens_per_credit=1000)
|
||||
│ └── gpt-4o-mini (tokens_per_credit=10000)
|
||||
└── Image Models
|
||||
├── runware:97@1 (credits_per_image=1, quality_tier=basic)
|
||||
├── dall-e-3 (is_default=true, credits_per_image=5, quality_tier=quality)
|
||||
└── google:4@2 (credits_per_image=15, quality_tier=premium)
|
||||
```
|
||||
|
||||
### No More Hardcoding
|
||||
|
||||
- **Backend**: Remove `MODEL_RATES`, `IMAGE_MODEL_RATES` from constants.py
|
||||
- **Backend**: Remove hardcoded CHOICES from `GlobalIntegrationSettings`
|
||||
- **Backend**: `ModelRegistry` always uses database, no fallbacks
|
||||
- **Frontend**: Load model choices from API endpoint
|
||||
|
||||
### Clear Credit Calculation
|
||||
|
||||
| Model Type | How Credits Calculated |
|
||||
|------------|----------------------|
|
||||
| Text | `ceil(total_tokens / tokens_per_credit)` from AIModelConfig |
|
||||
| Image | `credits_per_image * num_images` from AIModelConfig |
|
||||
|
||||
---
|
||||
|
||||
## Migration Phases Summary
|
||||
|
||||
| Phase | What Changes | Risk | Rollback |
|
||||
|-------|--------------|------|----------|
|
||||
| 1 | Add new models/fields | None - additive only | Drop new tables |
|
||||
| 2 | Add parallel code paths | Low - old code untouched | Remove new methods |
|
||||
| 3 | IntegrationProvider for keys | Medium - API key loading | Revert key loading |
|
||||
| 4 | Switch to new credit calc | Medium - financial impact | Revert credit service |
|
||||
| 5 | Create API endpoint | None - new endpoint | Remove endpoint |
|
||||
| 6 | Update frontend | Low - UI only | Revert frontend |
|
||||
| 7 | Cleanup legacy code | High if bugs found | Restore from backup |
|
||||
|
||||
**Rule:** Run full test suite after EACH phase before proceeding.
|
||||
|
||||
---
|
||||
|
||||
## Files Being Modified
|
||||
|
||||
### Backend Files
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `billing/models.py` | Add IntegrationProvider, add fields to AIModelConfig |
|
||||
| `ai/model_registry.py` | Add `get_default_model()`, remove constants fallback |
|
||||
| `ai/ai_core.py` | Use IntegrationProvider for API keys |
|
||||
| `ai/constants.py` | Remove MODEL_RATES, IMAGE_MODEL_RATES |
|
||||
| `billing/services/credit_service.py` | Model-based credit calculation |
|
||||
| `modules/system/global_settings_models.py` | Remove API keys, hardcoded choices |
|
||||
| `api/views/system.py` | Add `/api/v1/system/ai-models/` endpoint |
|
||||
|
||||
### Frontend Files
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `src/pages/Sites/Settings.tsx` | Load models from API, remove hardcodes |
|
||||
|
||||
### New Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `tests/test_ai_system_integration.py` | Integration tests for migration |
|
||||
|
||||
---
|
||||
|
||||
## Credit Values (Final)
|
||||
|
||||
### Text Models (per 1 credit)
|
||||
|
||||
| Model | Tokens per Credit | Example: 5000 tokens |
|
||||
|-------|-------------------|---------------------|
|
||||
| gpt-5.1 | 1,000 | 5 credits |
|
||||
| gpt-4o | 1,000 | 5 credits |
|
||||
| gpt-4o-mini | 10,000 | 1 credit |
|
||||
|
||||
### Image Models (per image)
|
||||
|
||||
| Model | Credits per Image | UI Display |
|
||||
|-------|-------------------|------------|
|
||||
| runware:97@1 | 1 | "Basic (1 credit/image)" |
|
||||
| dall-e-3 | 5 | "Quality (5 credits/image)" |
|
||||
| google:4@2 | 15 | "Premium (15 credits/image)" |
|
||||
|
||||
---
|
||||
|
||||
## Execution Paths That Must Keep Working
|
||||
|
||||
All these paths use the same AI functions and must work identically after migration:
|
||||
|
||||
1. **Manual Buttons** (Planner/Writer pages)
|
||||
- Cluster Keywords → `auto_cluster`
|
||||
- Generate Ideas → `generate_ideas`
|
||||
- Generate Content → `generate_content`
|
||||
- Extract Prompts → `generate_image_prompts`
|
||||
- Generate Images → `generate_images`
|
||||
|
||||
2. **Automation Manual Run** (Automation page → Run Now)
|
||||
- Same AI functions called via AutomationService
|
||||
|
||||
3. **Scheduled Automation** (Celery Beat scheduler)
|
||||
- Same AI functions called via scheduled tasks
|
||||
|
||||
4. **Direct Services** (Linker, Optimizer modules)
|
||||
- Use CreditService directly for credit deduction
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
Migration is **COMPLETE** when:
|
||||
|
||||
- [ ] All integration tests pass
|
||||
- [ ] All manual test checklist items pass
|
||||
- [ ] No errors in production logs for 1 week
|
||||
- [ ] Credit calculations match expected values
|
||||
- [ ] All execution paths work (manual, automation, scheduled)
|
||||
- [ ] Frontend loads models dynamically from API
|
||||
- [ ] No legacy code remains:
|
||||
- [ ] No `MODEL_RATES` / `IMAGE_MODEL_RATES` in constants.py
|
||||
- [ ] No API keys in `GlobalIntegrationSettings`
|
||||
- [ ] No hardcoded models in frontend
|
||||
- [ ] No fallback to constants in `ModelRegistry`
|
||||
- [ ] No unused fields in `GlobalIntegrationSettings`
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference Commands
|
||||
|
||||
```bash
|
||||
# Run integration tests
|
||||
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration -v 2
|
||||
|
||||
# Check AIModelConfig
|
||||
docker exec igny8_backend python manage.py shell -c "
|
||||
from igny8_core.business.billing.models import AIModelConfig
|
||||
for m in AIModelConfig.objects.all().order_by('model_type'):
|
||||
print(f'{m.model_name}: {m.model_type}, default={m.is_default}')
|
||||
"
|
||||
|
||||
# Check recent credit usage
|
||||
docker exec igny8_backend python manage.py shell -c "
|
||||
from igny8_core.business.billing.models import CreditUsageLog
|
||||
for log in CreditUsageLog.objects.order_by('-created_at')[:10]:
|
||||
print(f'{log.operation_type}: {log.credits_used} credits')
|
||||
"
|
||||
|
||||
# Check account credits
|
||||
docker exec igny8_backend python manage.py shell -c "
|
||||
from igny8_core.auth.models import Account
|
||||
for acc in Account.objects.all():
|
||||
print(f'{acc.name}: {acc.credits} credits')
|
||||
"
|
||||
```
|
||||
491
4th-jan-refactor/implementation-plan-for-ai-models-and-cost.md
Normal file
491
4th-jan-refactor/implementation-plan-for-ai-models-and-cost.md
Normal file
@@ -0,0 +1,491 @@
|
||||
|
||||
## Complete End-to-End Analysis & Restructuring Plan
|
||||
|
||||
### Current State Summary (from actual database queries)
|
||||
|
||||
**AIModelConfig (6 records - all needed):**
|
||||
| model_name | type | provider | active | default | cost | tokens_per_credit |
|
||||
|------------|------|----------|--------|---------|------|-------------------|
|
||||
| gpt-4o-mini | text | openai | ✅ | ❌ | $0.15/$0.60 per 1M | 10,000 |
|
||||
| gpt-4o | text | openai | ❌ | ❌ | $2.50/$10.00 per 1M | 1,000 |
|
||||
| gpt-5.1 | text | openai | ✅ | ✅ | $1.25/$10.00 per 1M | 1,000 |
|
||||
| runware:97@1 | image | runware | ✅ | ❌ | $0.012/image | 1 credit/image |
|
||||
| dall-e-3 | image | openai | ✅ | ✅ | $0.04/image | 5 credits/image |
|
||||
| google:4@2 | image | runware | ✅ | ❌ | $0.14/image | 15 credits/image |
|
||||
|
||||
---
|
||||
|
||||
### Current End-to-End Flow (Traced from Code)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ CURRENT ARCHITECTURE │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
1. CONFIGURATION LAYER
|
||||
┌────────────────────────────────────────┐
|
||||
│ GlobalIntegrationSettings (singleton) │ ← API keys stored here
|
||||
│ - openai_api_key │
|
||||
│ - runware_api_key │
|
||||
│ - anthropic_api_key (unused) │
|
||||
│ - bria_api_key (unused) │
|
||||
│ - openai_model: gpt-4o-mini ❌ │ ← Should be gpt-5.1
|
||||
│ - runware_model: bria:10@1 ❌ │ ← Model doesn't exist!
|
||||
│ - HARDCODED CHOICES duplicating DB │
|
||||
└────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────────────┐
|
||||
│ AIModelConfig (database) │ ← Source of truth for models
|
||||
│ - model_name, provider, costs │
|
||||
│ - is_active, is_default │
|
||||
└────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────────────┐
|
||||
│ constants.py │ ← DUPLICATE/LEGACY
|
||||
│ - MODEL_RATES (hardcoded) │
|
||||
│ - IMAGE_MODEL_RATES (hardcoded) │
|
||||
└────────────────────────────────────────┘
|
||||
|
||||
2. SETTINGS RESOLUTION LAYER
|
||||
┌────────────────────────────────────────┐
|
||||
│ settings.py │
|
||||
│ get_model_config(function, account) │
|
||||
│ - Gets model from GlobalIntegration │
|
||||
│ - Gets max_tokens from AIModelConfig │
|
||||
│ - Allows IntegrationSettings override │
|
||||
└────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────────────┐
|
||||
│ model_registry.py │
|
||||
│ ModelRegistry.get_model(model_id) │
|
||||
│ - Try DB (AIModelConfig) first │
|
||||
│ - Fallback to constants.py ❌ │ ← Should only use DB
|
||||
└────────────────────────────────────────┘
|
||||
|
||||
3. AI EXECUTION LAYER
|
||||
┌────────────────────────────────────────┐
|
||||
│ engine.py (AIEngine) │
|
||||
│ - Orchestrates all AI functions │
|
||||
│ - Progress tracking, cost tracking │
|
||||
└────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────────────┐
|
||||
│ ai_core.py (AICore) │
|
||||
│ - _load_account_settings() │ ← Gets API keys from Global
|
||||
│ - run_ai_request() for text │
|
||||
│ - generate_image() for images │
|
||||
│ - Uses IMAGE_MODEL_RATES fallback ❌ │
|
||||
└────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────────────┐
|
||||
│ ai/functions/ │
|
||||
│ - generate_images.py │
|
||||
│ - generate_content.py │
|
||||
│ - auto_cluster.py │
|
||||
│ - Each function uses AICore │
|
||||
└────────────────────────────────────────┘
|
||||
|
||||
4. CREDIT CALCULATION LAYER
|
||||
┌────────────────────────────────────────┐
|
||||
│ CreditCostConfig (database) │
|
||||
│ - operation_type │
|
||||
│ - tokens_per_credit │
|
||||
│ - min_credits │
|
||||
│ - price_per_credit_usd │
|
||||
│ - For images: 50 tokens/credit, min 5 │
|
||||
└────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────────────┐
|
||||
│ CreditService │
|
||||
│ - calculate_credits_from_tokens() │
|
||||
│ - deduct_credits_for_operation() │
|
||||
│ - Text: tokens → credits AFTER call │
|
||||
│ - Images: ??? (not token-based) │
|
||||
└────────────────────────────────────────┘
|
||||
|
||||
5. FRONTEND (Sites/Settings.tsx)
|
||||
┌────────────────────────────────────────┐
|
||||
│ HARDCODED model choices ❌ │
|
||||
│ - QUALITY_TO_CONFIG │
|
||||
│ - RUNWARE_MODEL_CHOICES │
|
||||
│ - DALLE_MODEL_CHOICES │
|
||||
│ - MODEL_LANDSCAPE_SIZES │
|
||||
└────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Problems Identified
|
||||
|
||||
| # | Problem | Location | Impact |
|
||||
|---|---------|----------|--------|
|
||||
| 1 | GlobalIntegrationSettings has hardcoded model choices | global_settings_models.py | Duplicates AIModelConfig |
|
||||
| 2 | runware_model = "bria:10@1" but model doesn't exist | GlobalIntegrationSettings | Broken fallback |
|
||||
| 3 | API keys mixed with model config | GlobalIntegrationSettings | No separation of concerns |
|
||||
| 4 | IMAGE_MODEL_RATES still used as fallback | ai_core.py, model_registry.py | Inconsistent pricing |
|
||||
| 5 | Frontend hardcodes model choices | Settings.tsx | Not dynamic |
|
||||
| 6 | Image credit calculation unclear | CreditService | Not based on cost_per_image |
|
||||
| 7 | constants.py duplicates DB data | constants.py | Maintenance burden |
|
||||
|
||||
---
|
||||
|
||||
### Target Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ TARGET ARCHITECTURE │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
1. NEW: IntegrationProvider Model (stores ALL 3rd party API keys)
|
||||
┌────────────────────────────────────────┐
|
||||
│ IntegrationProvider │ ← Future-proof: ALL integrations
|
||||
│ - provider_id: str (primary key) │
|
||||
│ Examples: openai, runware, google, │
|
||||
│ resend, stripe, etc. │
|
||||
│ - display_name: str │
|
||||
│ - provider_type: ai | email | payment │
|
||||
│ - api_key: encrypted str │
|
||||
│ - api_endpoint: URL (optional) │
|
||||
│ - is_active: bool │
|
||||
│ - config: JSON (rate limits, etc.) │
|
||||
└────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
2. CLEANED: AIModelConfig (references IntegrationProvider)
|
||||
┌────────────────────────────────────────┐
|
||||
│ AIModelConfig │
|
||||
│ - model_name: str │
|
||||
│ - display_name: str │
|
||||
│ - model_type: text | image │
|
||||
│ - provider: str → IntegrationProvider │
|
||||
│ - cost fields (unchanged) │
|
||||
│ + credits_per_image: int (NEW) │ ← For image models
|
||||
│ + tokens_per_credit: int (NEW) │ ← For text models
|
||||
│ + quality_tier: basic|quality|premium │ ← For UI display
|
||||
│ - is_default: bool │ ← Loads automatically
|
||||
└────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
3. SIMPLIFIED: GlobalIntegrationSettings
|
||||
┌────────────────────────────────────────┐
|
||||
│ GlobalIntegrationSettings │
|
||||
│ NO hardcoded model names │
|
||||
│ NO API keys (in IntegrationProvider) │
|
||||
│ Loads defaults from AIModelConfig │
|
||||
│ where is_default=True │
|
||||
│ - image_style: str │
|
||||
│ - max_in_article_images: int │
|
||||
│ - image_quality: str │
|
||||
└────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
4. UNIFIED: Model Resolution
|
||||
┌────────────────────────────────────────┐
|
||||
│ ModelRegistry │
|
||||
│ - get_default_model(type) → from DB │
|
||||
│ - get_model(model_id) → AIModelConfig │
|
||||
│ - get_provider(id) → IntegrationProv │
|
||||
│ - get_api_key(provider) → key │
|
||||
│ - NO fallback to constants │
|
||||
│ - NO hardcoded defaults │
|
||||
└────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
5. DYNAMIC: Frontend API
|
||||
┌────────────────────────────────────────┐
|
||||
│ GET /api/v1/system/ai-models/ │
|
||||
│ Returns models from DB with defaults │
|
||||
│ marked, no hardcoding needed │
|
||||
└────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Implementation Plan (Complete)
|
||||
|
||||
#### Phase 1: Database Schema Changes
|
||||
|
||||
**1.1 Create IntegrationProvider Model (Future-proof for ALL integrations)**
|
||||
|
||||
File: models.py
|
||||
|
||||
```python
|
||||
class IntegrationProvider(models.Model):
|
||||
"""
|
||||
Centralized 3rd party integration provider configuration.
|
||||
Single location for ALL external service API keys and configs.
|
||||
"""
|
||||
PROVIDER_TYPE_CHOICES = [
|
||||
('ai', 'AI Provider'),
|
||||
('email', 'Email Service'),
|
||||
('payment', 'Payment Gateway'),
|
||||
('storage', 'Storage Service'),
|
||||
('other', 'Other'),
|
||||
]
|
||||
|
||||
provider_id = models.CharField(max_length=50, unique=True, primary_key=True)
|
||||
# Examples: openai, runware, google, resend, stripe, aws_s3, etc.
|
||||
|
||||
display_name = models.CharField(max_length=100)
|
||||
provider_type = models.CharField(max_length=20, choices=PROVIDER_TYPE_CHOICES, default='ai')
|
||||
api_key = models.CharField(max_length=500, blank=True) # Should be encrypted
|
||||
api_secret = models.CharField(max_length=500, blank=True) # For services needing secret
|
||||
api_endpoint = models.URLField(blank=True) # Custom endpoint if needed
|
||||
webhook_secret = models.CharField(max_length=500, blank=True) # For Stripe etc.
|
||||
is_active = models.BooleanField(default=True)
|
||||
config = models.JSONField(default=dict, blank=True) # Rate limits, regions, etc.
|
||||
created_at = models.DateTimeField(auto_now_add=True)
|
||||
updated_at = models.DateTimeField(auto_now=True)
|
||||
|
||||
class Meta:
|
||||
db_table = 'igny8_integration_providers'
|
||||
verbose_name = 'Integration Provider'
|
||||
verbose_name_plural = 'Integration Providers'
|
||||
```
|
||||
|
||||
**1.2 Add fields to AIModelConfig**
|
||||
|
||||
```python
|
||||
# Add to AIModelConfig for IMAGE models
|
||||
credits_per_image = models.IntegerField(
|
||||
null=True, blank=True,
|
||||
help_text="Fixed credits per image generated. For image models only."
|
||||
)
|
||||
|
||||
# Add to AIModelConfig for TEXT models
|
||||
tokens_per_credit = models.IntegerField(
|
||||
null=True, blank=True,
|
||||
help_text="Number of tokens that equal 1 credit. For text models only."
|
||||
)
|
||||
|
||||
# Add quality tier for UI display (image models)
|
||||
quality_tier = models.CharField(
|
||||
max_length=20,
|
||||
choices=[('basic', 'Basic'), ('quality', 'Quality'), ('premium', 'Premium')],
|
||||
null=True, blank=True,
|
||||
help_text="Quality tier for frontend UI display"
|
||||
)
|
||||
```
|
||||
|
||||
**1.3 Migration to populate data**
|
||||
|
||||
Create IntegrationProvider records:
|
||||
```
|
||||
| provider_id | display_name | provider_type | Notes |
|
||||
|-------------|-------------------|---------------|--------------------------|
|
||||
| openai | OpenAI | ai | GPT models, DALL-E |
|
||||
| runware | Runware | ai | Image generation |
|
||||
| google | Google Cloud | ai | Future: Gemini, etc. |
|
||||
| resend | Resend | email | Transactional email |
|
||||
| stripe | Stripe | payment | Payment processing |
|
||||
```
|
||||
|
||||
Update AIModelConfig with credit/token values:
|
||||
```
|
||||
| model_name | type | tokens_per_credit | credits_per_image | quality_tier |
|
||||
|---------------|-------|-------------------|-------------------|--------------|
|
||||
| gpt-4o-mini | text | 10000 | - | - |
|
||||
| gpt-4o | text | 1000 | - | - |
|
||||
| gpt-5.1 | text | 1000 | - | - |
|
||||
| runware:97@1 | image | - | 1 | basic |
|
||||
| dall-e-3 | image | - | 5 | quality |
|
||||
| google:4@2 | image | - | 15 | premium |
|
||||
```
|
||||
|
||||
#### Phase 2: Backend Code Changes
|
||||
|
||||
**2.1 Remove hardcoded constants**
|
||||
|
||||
File: constants.py
|
||||
- Remove: MODEL_RATES, IMAGE_MODEL_RATES
|
||||
- Keep: JSON_MODE_MODELS (or move to AIModelConfig.supports_json_mode check)
|
||||
|
||||
**2.2 Update ModelRegistry**
|
||||
|
||||
File: model_registry.py
|
||||
- Remove fallback to constants.py
|
||||
- Add: `get_default_model(model_type) → AIModelConfig where is_default=True`
|
||||
- Add: `get_provider(provider_id) → IntegrationProvider`
|
||||
- Add: `get_api_key(provider_id) → str`
|
||||
- **NO hardcoded model names** - always query DB
|
||||
|
||||
**2.3 Update AICore**
|
||||
|
||||
File: ai_core.py
|
||||
- Change `_load_account_settings()` to use IntegrationProvider for API keys
|
||||
- Remove IMAGE_MODEL_RATES import and usage
|
||||
- Use `ModelRegistry.get_default_model('text')` instead of hardcoded model
|
||||
- Use `ModelRegistry.calculate_cost()` exclusively
|
||||
|
||||
**2.4 Update CreditService**
|
||||
|
||||
File: credit_service.py
|
||||
|
||||
For IMAGE models:
|
||||
```python
|
||||
def calculate_credits_for_image(model_name: str, num_images: int) -> int:
|
||||
"""Calculate credits for image generation from AIModelConfig"""
|
||||
model = AIModelConfig.objects.get(model_name=model_name, is_active=True)
|
||||
return model.credits_per_image * num_images
|
||||
```
|
||||
|
||||
For TEXT models:
|
||||
```python
|
||||
def calculate_credits_from_tokens(model_name: str, total_tokens: int) -> int:
|
||||
"""Calculate credits from token usage based on model's tokens_per_credit"""
|
||||
model = AIModelConfig.objects.get(model_name=model_name, is_active=True)
|
||||
tokens_per_credit = model.tokens_per_credit or 1000 # fallback
|
||||
return math.ceil(total_tokens / tokens_per_credit)
|
||||
```
|
||||
|
||||
**2.5 Simplify GlobalIntegrationSettings**
|
||||
|
||||
File: global_settings_models.py
|
||||
- Remove: All API key fields (moved to IntegrationProvider)
|
||||
- Remove: All hardcoded CHOICES
|
||||
- Remove: Model name fields (defaults loaded from AIModelConfig.is_default)
|
||||
- Keep: image_style, max_in_article_images, image_quality
|
||||
- Add: Helper methods to get defaults from AIModelConfig
|
||||
|
||||
#### Phase 3: API Endpoints
|
||||
|
||||
**3.1 New endpoint: GET /api/v1/system/ai-models/**
|
||||
|
||||
Returns all active models from database with defaults marked (no hardcoding):
|
||||
```json
|
||||
{
|
||||
"text_models": [
|
||||
{
|
||||
"model_name": "gpt-5.1",
|
||||
"display_name": "GPT-5.1 Premium",
|
||||
"is_default": true,
|
||||
"tokens_per_credit": 1000,
|
||||
"max_output_tokens": 8192
|
||||
},
|
||||
{
|
||||
"model_name": "gpt-4o-mini",
|
||||
"display_name": "GPT-4o Mini",
|
||||
"is_default": false,
|
||||
"tokens_per_credit": 10000,
|
||||
"max_output_tokens": 16000
|
||||
}
|
||||
],
|
||||
"image_models": [
|
||||
{
|
||||
"model_name": "runware:97@1",
|
||||
"display_name": "Basic",
|
||||
"quality_tier": "basic",
|
||||
"is_default": false,
|
||||
"credits_per_image": 1,
|
||||
"valid_sizes": ["1024x1024", "1280x768"]
|
||||
},
|
||||
{
|
||||
"model_name": "dall-e-3",
|
||||
"display_name": "Quality",
|
||||
"quality_tier": "quality",
|
||||
"is_default": true,
|
||||
"credits_per_image": 5,
|
||||
"valid_sizes": ["1024x1024", "1792x1024"]
|
||||
},
|
||||
{
|
||||
"model_name": "google:4@2",
|
||||
"display_name": "Premium",
|
||||
"quality_tier": "premium",
|
||||
"is_default": false,
|
||||
"credits_per_image": 15,
|
||||
"valid_sizes": ["1024x1024", "1376x768"]
|
||||
}
|
||||
],
|
||||
"image_settings": {
|
||||
"style": "photorealistic",
|
||||
"max_in_article_images": 4,
|
||||
"quality": "hd"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Phase 4: Frontend Changes
|
||||
|
||||
**4.1 Settings.tsx**
|
||||
|
||||
- Remove: QUALITY_TO_CONFIG, RUNWARE_MODEL_CHOICES, DALLE_MODEL_CHOICES hardcodes
|
||||
- Remove: MODEL_LANDSCAPE_SIZES hardcodes
|
||||
- Add: Fetch models from `/api/v1/system/ai-models/`
|
||||
- Load valid_sizes from API response per model
|
||||
- Display to user (no provider/model names visible):
|
||||
- **"Basic (1 credit/image)"**
|
||||
- **"Quality (5 credits/image)"**
|
||||
- **"Premium (15 credits/image)"**
|
||||
- Default selection: model where `is_default=true` from API
|
||||
|
||||
#### Phase 5: Cleanup
|
||||
|
||||
**5.1 Files to clean/remove**
|
||||
- Remove unused fields from GlobalIntegrationSettings: anthropic_*, bria_*, all API key fields, hardcoded model fields
|
||||
- Remove deprecated methods from AICore
|
||||
- Update all imports removing constants.py usage
|
||||
- Remove CreditCostConfig dependency for image operations (use AIModelConfig.credits_per_image directly)
|
||||
|
||||
---
|
||||
|
||||
### Credit Calculation Summary
|
||||
|
||||
**Text Models (token-based):**
|
||||
| Model | tokens_per_credit | Example: 5000 tokens |
|
||||
|-------|-------------------|----------------------|
|
||||
| gpt-5.1 | 1,000 | 5 credits |
|
||||
| gpt-4o | 1,000 | 5 credits |
|
||||
| gpt-4o-mini | 10,000 | 1 credit |
|
||||
|
||||
**Image Models (per-image):**
|
||||
| Model | credits_per_image | quality_tier | Display |
|
||||
|-------|-------------------|--------------|---------|
|
||||
| runware:97@1 | 1 | basic | "Basic (1 credit/image)" |
|
||||
| dall-e-3 | 5 | quality | "Quality (5 credits/image)" |
|
||||
| google:4@2 | 15 | premium | "Premium (15 credits/image)" |
|
||||
|
||||
---
|
||||
|
||||
### Migration Order
|
||||
|
||||
1. Create IntegrationProvider model + migration
|
||||
2. Add credits_per_image, tokens_per_credit, quality_tier to AIModelConfig + migration
|
||||
3. Data migration: populate IntegrationProvider, update AIModelConfig with credit values
|
||||
4. Update ModelRegistry (remove constants fallback, add get_default_model)
|
||||
5. Update AICore (use IntegrationProvider for keys)
|
||||
6. Update CreditService (model-based credit calculation)
|
||||
7. Create API endpoint /api/v1/system/ai-models/
|
||||
8. Update frontend (load from API, no hardcodes)
|
||||
9. Cleanup GlobalIntegrationSettings (remove API keys, hardcoded choices)
|
||||
10. Remove constants.py hardcoded rates
|
||||
|
||||
---
|
||||
|
||||
### Files Changed Summary
|
||||
|
||||
| File | Action |
|
||||
|------|--------|
|
||||
| models.py | Add IntegrationProvider, update AIModelConfig with credit fields |
|
||||
| model_registry.py | Remove constants fallback, add get_default_model(), get_provider() |
|
||||
| ai_core.py | Use IntegrationProvider for keys, ModelRegistry for defaults |
|
||||
| constants.py | Remove MODEL_RATES, IMAGE_MODEL_RATES |
|
||||
| credit_service.py | Model-based credit calculation for both text and images |
|
||||
| global_settings_models.py | Remove API keys, hardcoded choices, model fields |
|
||||
| backend/igny8_core/api/views/system.py | Add ai-models endpoint |
|
||||
| Settings.tsx | Load models from API, remove all hardcodes |
|
||||
|
||||
---
|
||||
|
||||
### Key Principles
|
||||
|
||||
1. **No hardcoded model names** - GlobalIntegrationSettings loads defaults from AIModelConfig where is_default=True
|
||||
2. **Single source of truth** - AIModelConfig is THE source for all model info including credit costs
|
||||
3. **Future-proof** - IntegrationProvider handles ALL 3rd party integrations (AI, email, payment, etc.)
|
||||
4. **Dynamic frontend** - All model choices loaded from API, not hardcoded
|
||||
5. **Configurable credits** - Change credits_per_image or tokens_per_credit in admin, no code changes needed
|
||||
939
4th-jan-refactor/safe-migration-and-testing-plan.md
Normal file
939
4th-jan-refactor/safe-migration-and-testing-plan.md
Normal file
@@ -0,0 +1,939 @@
|
||||
# Safe Migration & Testing Plan
|
||||
|
||||
## Overview
|
||||
|
||||
This document ensures the refactor is done safely with zero breakage. All existing functionality must work identically before and after migration.
|
||||
|
||||
---
|
||||
|
||||
## Current Working System Components
|
||||
|
||||
### 1. AI Functions (Core - Used by ALL paths)
|
||||
|
||||
All AI operations go through these functions in `backend/igny8_core/ai/functions/`:
|
||||
|
||||
| Function | File | Operation Type | Credits | Called Via |
|
||||
|----------|------|----------------|---------|------------|
|
||||
| Auto Cluster | `auto_cluster.py` | `clustering` | Token-based | AIEngine |
|
||||
| Generate Ideas | `generate_ideas.py` | `idea_generation` | Token-based | AIEngine |
|
||||
| Generate Content | `generate_content.py` | `content_generation` | Token-based | AIEngine |
|
||||
| Generate Image Prompts | `generate_image_prompts.py` | `image_prompt_extraction` | Token-based | AIEngine |
|
||||
| Generate Images | `generate_images.py` | `image_generation` | Per-image | AICore direct |
|
||||
| Optimize Content | `optimize_content.py` | `content_optimization` | Token-based | OptimizerService |
|
||||
|
||||
**Operation Type Mapping** (in `engine.py`):
|
||||
```python
|
||||
mapping = {
|
||||
'auto_cluster': 'clustering',
|
||||
'generate_ideas': 'idea_generation',
|
||||
'generate_content': 'content_generation',
|
||||
'generate_image_prompts': 'image_prompt_extraction',
|
||||
'generate_images': 'image_generation',
|
||||
'generate_site_structure': 'site_structure_generation',
|
||||
}
|
||||
```
|
||||
|
||||
### 1.1 Non-AIEngine Credit Deduction Points
|
||||
|
||||
| Service | Operation Type | Direct Credit Deduction |
|
||||
|---------|---------------|------------------------|
|
||||
| LinkerService | `internal_linking` | ✅ Uses CreditService directly |
|
||||
| OptimizerService | `content_optimization` | ✅ Uses CreditService directly |
|
||||
|
||||
### 2. Execution Paths (All use same AI functions)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ ALL EXECUTION PATHS │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
PATH 1: Manual Module Page Buttons (via run_ai_task Celery task)
|
||||
├── Planner Page → "Cluster Keywords" button → api/planner.py → run_ai_task.delay('auto_cluster')
|
||||
├── Planner Page → "Generate Ideas" button → api/planner.py → run_ai_task.delay('generate_ideas')
|
||||
├── Writer Page → "Generate Content" button → api/writer.py → run_ai_task.delay('generate_content')
|
||||
├── Writer Page → "Extract Prompts" button → api/writer.py → run_ai_task.delay('generate_image_prompts')
|
||||
├── Writer Page → "Generate Images" button → api/writer.py → process_image_generation_queue.delay()
|
||||
└── All go through: Celery task → AIEngine → AI Function → AICore → API → CreditService
|
||||
|
||||
PATH 2: Automation Page Manual Run
|
||||
├── Automation Page → "Run Now" button
|
||||
├── automation/tasks.py → run_automation_task
|
||||
├── AutomationService.run_automation()
|
||||
│ Stage 1: _run_clustering() → AIEngine.execute('auto_cluster')
|
||||
│ Stage 2: _run_idea_generation() → AIEngine.execute('generate_ideas')
|
||||
│ Stage 3: _run_task_creation() → Database only (no AI)
|
||||
│ Stage 4: _run_content_generation() → AIEngine.execute('generate_content')
|
||||
│ Stage 5: _run_image_prompt_extraction() → AIEngine.execute('generate_image_prompts')
|
||||
│ Stage 6: _run_image_generation() → process_image_generation_queue.delay()
|
||||
│ Stage 7: _run_publishing() → WordPress API (no AI)
|
||||
└── Same credit deduction path (stages 1,2,4,5 via AIEngine, stage 6 via AICore)
|
||||
|
||||
PATH 3: Scheduled Automation Run
|
||||
├── Celery Beat scheduler → check_scheduled_automations task (hourly)
|
||||
├── automation_tasks.py → run_automation_task.delay()
|
||||
├── AutomationService.run_automation()
|
||||
├── Calls same AI functions through AIEngine
|
||||
└── Same credit deduction path
|
||||
|
||||
PATH 4: Direct Service Operations (Bypass AIEngine)
|
||||
├── Linker Module → LinkerService → CreditService.deduct_credits_for_operation('internal_linking')
|
||||
├── Optimizer Module → OptimizerService → CreditService.deduct_credits_for_operation('content_optimization')
|
||||
└── These use direct credit deduction, not via AIEngine
|
||||
```
|
||||
|
||||
### 3. Credit Flow (Current)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ CURRENT CREDIT FLOW │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
For TEXT operations (clustering, ideas, content) - VIA AIEngine:
|
||||
1. API endpoint receives request
|
||||
2. Celery task run_ai_task triggered
|
||||
3. AIEngine.execute(function_name, payload)
|
||||
a. Step 2.5: CreditService.check_credits() - PRE-CHECK
|
||||
b. Step 3: AICore.run_ai_request() - makes OpenAI API call
|
||||
c. Response includes token usage (input_tokens, output_tokens)
|
||||
d. Step 5.5: CreditService.deduct_credits_for_operation() - DEDUCTION
|
||||
4. CreditService.calculate_credits_from_tokens() uses CreditCostConfig
|
||||
5. Creates CreditTransaction (balance history)
|
||||
6. Creates CreditUsageLog (detailed tracking)
|
||||
7. Updates Account.credits
|
||||
|
||||
For IMAGE operations - VIA process_image_generation_queue:
|
||||
1. API endpoint receives request
|
||||
2. Celery task process_image_generation_queue triggered
|
||||
3. For each image: AICore.generate_image()
|
||||
a. Makes Runware/OpenAI API call
|
||||
b. Calculates cost via ModelRegistry.calculate_cost()
|
||||
c. Returns image URL
|
||||
4. ⚠️ Credit deduction handled AFTER all images generated
|
||||
5. Uses CreditCostConfig.image_generation entry (min_credits, tokens_per_credit)
|
||||
6. Creates CreditUsageLog
|
||||
|
||||
For DIRECT SERVICE OPERATIONS (Linker, Optimizer):
|
||||
1. Service method called (e.g., LinkerService.generate_links())
|
||||
2. AI call made directly to AICore or external service
|
||||
3. CreditService.deduct_credits_for_operation() called directly
|
||||
4. Uses operation-specific CreditCostConfig entry
|
||||
```
|
||||
|
||||
### 3.1 Celery Task Definitions
|
||||
|
||||
| Task File | Task Name | Entry Point For |
|
||||
|-----------|-----------|-----------------|
|
||||
| `ai/tasks.py` | `run_ai_task` | All manual AI buttons (universal task) |
|
||||
| `ai/tasks.py` | `process_image_generation_queue` | Image generation queue |
|
||||
| `automation/tasks.py` | `check_scheduled_automations` | Hourly scheduler check |
|
||||
| `automation/tasks.py` | `run_automation_task` | Full automation pipeline |
|
||||
| `automation/tasks.py` | `resume_paused_automation` | Resume after pause |
|
||||
|
||||
### 4. Key Files Currently Working
|
||||
|
||||
| File | Purpose | Must Keep Working |
|
||||
|------|---------|-------------------|
|
||||
| `ai/engine.py` | Orchestrates AI functions | ✅ |
|
||||
| `ai/ai_core.py` | API calls, key loading | ✅ |
|
||||
| `ai/model_registry.py` | Model config lookup | ✅ |
|
||||
| `ai/settings.py` | get_model_config() | ✅ |
|
||||
| `ai/tasks.py` | Celery tasks | ✅ |
|
||||
| `ai/functions/*.py` | All AI functions | ✅ |
|
||||
| `billing/services/credit_service.py` | Credit calculation | ✅ |
|
||||
| `billing/models.py` | CreditUsageLog, CreditCostConfig | ✅ |
|
||||
| `automation/services/automation_service.py` | Automation runner | ✅ |
|
||||
| `modules/system/global_settings_models.py` | API keys, defaults | ✅ |
|
||||
|
||||
---
|
||||
|
||||
## Potential Issues to Verify Before Migration
|
||||
|
||||
### ⚠️ Issue 1: Image Credit Deduction May Be Incomplete
|
||||
|
||||
The `process_image_generation_queue` task calculates cost via `ModelRegistry.calculate_cost()` but credit deduction needs verification:
|
||||
- Does it call `CreditService.deduct_credits_for_operation()`?
|
||||
- Or does it just log cost without deducting credits?
|
||||
|
||||
**Verification Query:**
|
||||
```bash
|
||||
docker exec igny8_backend python manage.py shell -c "
|
||||
from igny8_core.business.billing.models import CreditUsageLog
|
||||
logs = CreditUsageLog.objects.filter(operation_type='image_generation').order_by('-created_at')[:5]
|
||||
for log in logs:
|
||||
print(f'{log.operation_type}: {log.credits_used} credits, cost=${log.cost_usd}, model={log.model_used}')
|
||||
if not logs:
|
||||
print('⚠️ NO image_generation credit logs found - may not be tracking credits!')
|
||||
"
|
||||
```
|
||||
|
||||
### ⚠️ Issue 2: GlobalIntegrationSettings.runware_model Points to Non-Existent Model
|
||||
|
||||
Current value: `bria:10@1` - but this model doesn't exist in AIModelConfig.
|
||||
This may cause fallback issues.
|
||||
|
||||
### ⚠️ Issue 3: Multiple Credit Calculation Paths
|
||||
|
||||
Credits are calculated in different ways depending on the path:
|
||||
- Via AIEngine → uses `CreditCostConfig` token-based calculation
|
||||
- Via direct service → uses `CreditCostConfig` with operation-specific rates
|
||||
- Via image generation → may use different calculation
|
||||
|
||||
Need to unify after migration.
|
||||
|
||||
---
|
||||
|
||||
## Pre-Migration Baseline Capture
|
||||
|
||||
### Step 1: Database State Snapshot
|
||||
|
||||
Run this BEFORE any changes to establish baseline:
|
||||
|
||||
```bash
|
||||
docker exec igny8_backend python manage.py shell << 'EOF'
|
||||
import json
|
||||
from datetime import datetime
|
||||
|
||||
print("=" * 80)
|
||||
print(f"BASELINE SNAPSHOT - {datetime.now().isoformat()}")
|
||||
print("=" * 80)
|
||||
|
||||
# 1. AIModelConfig
|
||||
print("\n=== AIModelConfig ===")
|
||||
from igny8_core.business.billing.models import AIModelConfig
|
||||
for m in AIModelConfig.objects.all().order_by('model_type', 'sort_order'):
|
||||
print(json.dumps({
|
||||
'model_name': m.model_name,
|
||||
'model_type': m.model_type,
|
||||
'provider': m.provider,
|
||||
'is_active': m.is_active,
|
||||
'is_default': m.is_default,
|
||||
'cost_per_image': str(m.cost_per_image) if m.cost_per_image else None,
|
||||
'input_cost_per_1m': str(m.input_cost_per_1m) if m.input_cost_per_1m else None,
|
||||
'output_cost_per_1m': str(m.output_cost_per_1m) if m.output_cost_per_1m else None,
|
||||
}))
|
||||
|
||||
# 2. CreditCostConfig
|
||||
print("\n=== CreditCostConfig ===")
|
||||
from igny8_core.business.billing.models import CreditCostConfig
|
||||
for c in CreditCostConfig.objects.filter(is_active=True):
|
||||
print(json.dumps({
|
||||
'operation_type': c.operation_type,
|
||||
'tokens_per_credit': c.tokens_per_credit,
|
||||
'min_credits': c.min_credits,
|
||||
'price_per_credit_usd': str(c.price_per_credit_usd),
|
||||
}))
|
||||
|
||||
# 3. GlobalIntegrationSettings
|
||||
print("\n=== GlobalIntegrationSettings ===")
|
||||
from igny8_core.modules.system.global_settings_models import GlobalIntegrationSettings
|
||||
g = GlobalIntegrationSettings.get_instance()
|
||||
print(json.dumps({
|
||||
'openai_model': g.openai_model,
|
||||
'openai_temperature': g.openai_temperature,
|
||||
'openai_max_tokens': g.openai_max_tokens,
|
||||
'default_text_provider': g.default_text_provider,
|
||||
'dalle_model': g.dalle_model,
|
||||
'runware_model': g.runware_model,
|
||||
'default_image_service': g.default_image_service,
|
||||
'image_style': g.image_style,
|
||||
'max_in_article_images': g.max_in_article_images,
|
||||
'has_openai_key': bool(g.openai_api_key),
|
||||
'has_runware_key': bool(g.runware_api_key),
|
||||
}))
|
||||
|
||||
# 4. Recent Credit Usage (last 20)
|
||||
print("\n=== Recent CreditUsageLog (last 20) ===")
|
||||
from igny8_core.business.billing.models import CreditUsageLog
|
||||
for log in CreditUsageLog.objects.all().order_by('-created_at')[:20]:
|
||||
print(json.dumps({
|
||||
'operation_type': log.operation_type,
|
||||
'credits_used': log.credits_used,
|
||||
'model_used': log.model_used,
|
||||
'tokens_input': log.tokens_input,
|
||||
'tokens_output': log.tokens_output,
|
||||
'cost_usd': str(log.cost_usd) if log.cost_usd else None,
|
||||
}))
|
||||
|
||||
# 5. Account Credits
|
||||
print("\n=== Account Credits ===")
|
||||
from igny8_core.auth.models import Account
|
||||
for acc in Account.objects.all()[:5]:
|
||||
print(json.dumps({
|
||||
'account_id': acc.id,
|
||||
'name': acc.name,
|
||||
'credits': acc.credits,
|
||||
}))
|
||||
|
||||
print("\n" + "=" * 80)
|
||||
print("BASELINE CAPTURE COMPLETE")
|
||||
print("=" * 80)
|
||||
EOF
|
||||
```
|
||||
|
||||
Save this output to: `4th-jan-refactor/baseline-snapshot.json`
|
||||
|
||||
---
|
||||
|
||||
## Integration Tests (Must Pass Before AND After)
|
||||
|
||||
### Test File: `backend/igny8_core/tests/test_ai_system_integration.py`
|
||||
|
||||
```python
|
||||
"""
|
||||
AI System Integration Tests
|
||||
===========================
|
||||
These tests verify the entire AI pipeline works end-to-end.
|
||||
Run BEFORE migration to establish baseline.
|
||||
Run AFTER each phase to verify nothing broke.
|
||||
|
||||
Usage:
|
||||
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration
|
||||
"""
|
||||
from decimal import Decimal
|
||||
from django.test import TestCase, TransactionTestCase
|
||||
from django.db import transaction
|
||||
from unittest.mock import patch, MagicMock
|
||||
import json
|
||||
|
||||
|
||||
class AIModelConfigTests(TestCase):
|
||||
"""Test AIModelConfig loading and access"""
|
||||
|
||||
def test_text_models_exist(self):
|
||||
"""All required text models exist and are configured"""
|
||||
from igny8_core.business.billing.models import AIModelConfig
|
||||
|
||||
required_text_models = ['gpt-5.1', 'gpt-4o-mini']
|
||||
for model_name in required_text_models:
|
||||
model = AIModelConfig.objects.filter(model_name=model_name).first()
|
||||
self.assertIsNotNone(model, f"Text model {model_name} not found")
|
||||
self.assertEqual(model.model_type, 'text')
|
||||
|
||||
def test_image_models_exist(self):
|
||||
"""All required image models exist and are configured"""
|
||||
from igny8_core.business.billing.models import AIModelConfig
|
||||
|
||||
required_image_models = ['runware:97@1', 'dall-e-3', 'google:4@2']
|
||||
for model_name in required_image_models:
|
||||
model = AIModelConfig.objects.filter(model_name=model_name).first()
|
||||
self.assertIsNotNone(model, f"Image model {model_name} not found")
|
||||
self.assertEqual(model.model_type, 'image')
|
||||
self.assertIsNotNone(model.cost_per_image, f"{model_name} missing cost_per_image")
|
||||
|
||||
def test_default_text_model_exists(self):
|
||||
"""Exactly one default text model is set"""
|
||||
from igny8_core.business.billing.models import AIModelConfig
|
||||
|
||||
defaults = AIModelConfig.objects.filter(model_type='text', is_default=True, is_active=True)
|
||||
self.assertEqual(defaults.count(), 1, "Should have exactly 1 default text model")
|
||||
self.assertEqual(defaults.first().model_name, 'gpt-5.1')
|
||||
|
||||
def test_default_image_model_exists(self):
|
||||
"""Exactly one default image model is set"""
|
||||
from igny8_core.business.billing.models import AIModelConfig
|
||||
|
||||
defaults = AIModelConfig.objects.filter(model_type='image', is_default=True, is_active=True)
|
||||
self.assertEqual(defaults.count(), 1, "Should have exactly 1 default image model")
|
||||
|
||||
|
||||
class ModelRegistryTests(TestCase):
|
||||
"""Test ModelRegistry functionality"""
|
||||
|
||||
def test_get_model_from_db(self):
|
||||
"""ModelRegistry.get_model() returns model from database"""
|
||||
from igny8_core.ai.model_registry import ModelRegistry
|
||||
|
||||
model = ModelRegistry.get_model('gpt-5.1')
|
||||
self.assertIsNotNone(model)
|
||||
# Should be AIModelConfig instance or dict with model_name
|
||||
if hasattr(model, 'model_name'):
|
||||
self.assertEqual(model.model_name, 'gpt-5.1')
|
||||
else:
|
||||
self.assertEqual(model.get('model_name'), 'gpt-5.1')
|
||||
|
||||
def test_get_image_model(self):
|
||||
"""ModelRegistry returns image models correctly"""
|
||||
from igny8_core.ai.model_registry import ModelRegistry
|
||||
|
||||
for model_name in ['runware:97@1', 'dall-e-3', 'google:4@2']:
|
||||
model = ModelRegistry.get_model(model_name)
|
||||
self.assertIsNotNone(model, f"Model {model_name} not found")
|
||||
|
||||
def test_calculate_cost_text(self):
|
||||
"""Cost calculation for text models works"""
|
||||
from igny8_core.ai.model_registry import ModelRegistry
|
||||
|
||||
cost = ModelRegistry.calculate_cost('gpt-5.1', input_tokens=1000, output_tokens=500)
|
||||
self.assertIsInstance(cost, (int, float, Decimal))
|
||||
self.assertGreater(cost, 0)
|
||||
|
||||
def test_calculate_cost_image(self):
|
||||
"""Cost calculation for image models works"""
|
||||
from igny8_core.ai.model_registry import ModelRegistry
|
||||
|
||||
cost = ModelRegistry.calculate_cost('dall-e-3', num_images=1)
|
||||
self.assertIsInstance(cost, (int, float, Decimal))
|
||||
self.assertGreater(cost, 0)
|
||||
|
||||
|
||||
class APIKeyLoadingTests(TestCase):
|
||||
"""Test API key loading from GlobalIntegrationSettings"""
|
||||
|
||||
def test_openai_key_loads(self):
|
||||
"""OpenAI API key loads from GlobalIntegrationSettings"""
|
||||
from igny8_core.ai.ai_core import AICore
|
||||
|
||||
ai_core = AICore()
|
||||
key = ai_core.get_api_key('openai')
|
||||
self.assertIsNotNone(key, "OpenAI API key not configured")
|
||||
self.assertTrue(len(key) > 10, "OpenAI API key too short")
|
||||
|
||||
def test_runware_key_loads(self):
|
||||
"""Runware API key loads from GlobalIntegrationSettings"""
|
||||
from igny8_core.ai.ai_core import AICore
|
||||
|
||||
ai_core = AICore()
|
||||
key = ai_core.get_api_key('runware')
|
||||
self.assertIsNotNone(key, "Runware API key not configured")
|
||||
|
||||
|
||||
class CreditServiceTests(TestCase):
|
||||
"""Test credit calculation and deduction"""
|
||||
|
||||
def test_calculate_credits_text_operation(self):
|
||||
"""Credit calculation for text operations works"""
|
||||
from igny8_core.business.billing.services.credit_service import CreditService
|
||||
|
||||
# Test content generation
|
||||
credits = CreditService.calculate_credits_from_tokens(
|
||||
'content_generation',
|
||||
tokens_input=1000,
|
||||
tokens_output=2000
|
||||
)
|
||||
self.assertIsInstance(credits, int)
|
||||
self.assertGreater(credits, 0)
|
||||
|
||||
def test_calculate_credits_image_operation(self):
|
||||
"""Credit calculation for image operations works"""
|
||||
from igny8_core.business.billing.services.credit_service import CreditService
|
||||
|
||||
# Image generation uses min_credits from CreditCostConfig
|
||||
credits = CreditService.calculate_credits_from_tokens(
|
||||
'image_generation',
|
||||
tokens_input=0,
|
||||
tokens_output=0
|
||||
)
|
||||
self.assertIsInstance(credits, int)
|
||||
self.assertGreaterEqual(credits, 1)
|
||||
|
||||
def test_credit_cost_config_exists(self):
|
||||
"""All required CreditCostConfig entries exist"""
|
||||
from igny8_core.business.billing.models import CreditCostConfig
|
||||
|
||||
required_ops = ['clustering', 'idea_generation', 'content_generation',
|
||||
'image_generation', 'image_prompt_extraction']
|
||||
for op in required_ops:
|
||||
config = CreditCostConfig.objects.filter(operation_type=op, is_active=True).first()
|
||||
self.assertIsNotNone(config, f"CreditCostConfig for {op} not found")
|
||||
|
||||
|
||||
class AISettingsTests(TestCase):
|
||||
"""Test get_model_config() function"""
|
||||
|
||||
def test_get_model_config_returns_config(self):
|
||||
"""get_model_config() returns valid configuration"""
|
||||
from igny8_core.ai.settings import get_model_config
|
||||
from igny8_core.auth.models import Account
|
||||
|
||||
account = Account.objects.first()
|
||||
if account:
|
||||
config = get_model_config('content_generation', account)
|
||||
|
||||
self.assertIn('model', config)
|
||||
self.assertIn('max_tokens', config)
|
||||
self.assertIn('temperature', config)
|
||||
self.assertIsNotNone(config['model'])
|
||||
|
||||
|
||||
class AIFunctionValidationTests(TestCase):
|
||||
"""Test AI function classes load correctly"""
|
||||
|
||||
def test_auto_cluster_function_loads(self):
|
||||
"""AutoClusterFunction loads and validates"""
|
||||
from igny8_core.ai.functions.auto_cluster import AutoClusterFunction
|
||||
|
||||
fn = AutoClusterFunction()
|
||||
self.assertEqual(fn.get_name(), 'auto_cluster')
|
||||
self.assertIsNotNone(fn.get_metadata())
|
||||
|
||||
def test_generate_ideas_function_loads(self):
|
||||
"""GenerateIdeasFunction loads and validates"""
|
||||
from igny8_core.ai.functions.generate_ideas import GenerateIdeasFunction
|
||||
|
||||
fn = GenerateIdeasFunction()
|
||||
self.assertEqual(fn.get_name(), 'generate_ideas')
|
||||
|
||||
def test_generate_content_function_loads(self):
|
||||
"""GenerateContentFunction loads and validates"""
|
||||
from igny8_core.ai.functions.generate_content import GenerateContentFunction
|
||||
|
||||
fn = GenerateContentFunction()
|
||||
self.assertEqual(fn.get_name(), 'generate_content')
|
||||
|
||||
def test_generate_images_function_loads(self):
|
||||
"""GenerateImagesFunction loads and validates"""
|
||||
from igny8_core.ai.functions.generate_images import GenerateImagesFunction
|
||||
|
||||
fn = GenerateImagesFunction()
|
||||
self.assertEqual(fn.get_name(), 'generate_images')
|
||||
|
||||
|
||||
class GlobalIntegrationSettingsTests(TestCase):
|
||||
"""Test GlobalIntegrationSettings singleton"""
|
||||
|
||||
def test_singleton_loads(self):
|
||||
"""GlobalIntegrationSettings singleton loads"""
|
||||
from igny8_core.modules.system.global_settings_models import GlobalIntegrationSettings
|
||||
|
||||
settings = GlobalIntegrationSettings.get_instance()
|
||||
self.assertIsNotNone(settings)
|
||||
self.assertEqual(settings.pk, 1)
|
||||
|
||||
def test_image_settings_exist(self):
|
||||
"""Image settings are configured"""
|
||||
from igny8_core.modules.system.global_settings_models import GlobalIntegrationSettings
|
||||
|
||||
settings = GlobalIntegrationSettings.get_instance()
|
||||
self.assertIsNotNone(settings.image_style)
|
||||
self.assertIsNotNone(settings.max_in_article_images)
|
||||
self.assertGreater(settings.max_in_article_images, 0)
|
||||
|
||||
|
||||
class AutomationServiceTests(TestCase):
|
||||
"""Test AutomationService uses same AI functions"""
|
||||
|
||||
def test_automation_service_imports(self):
|
||||
"""AutomationService can import AI functions"""
|
||||
from igny8_core.business.automation.services.automation_service import AutomationService
|
||||
|
||||
# Just verify the import works and class exists
|
||||
self.assertIsNotNone(AutomationService)
|
||||
```
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Run all integration tests
|
||||
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration -v 2
|
||||
|
||||
# Run specific test class
|
||||
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration.AIModelConfigTests -v 2
|
||||
|
||||
# Run with coverage
|
||||
docker exec igny8_backend coverage run manage.py test igny8_core.tests.test_ai_system_integration
|
||||
docker exec igny8_backend coverage report
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phased Migration with Verification
|
||||
|
||||
### Phase 1: Add New Models (NO BREAKING CHANGES)
|
||||
|
||||
**Changes:**
|
||||
- Add `IntegrationProvider` model
|
||||
- Add new fields to `AIModelConfig`: `credits_per_image`, `tokens_per_credit`, `quality_tier`
|
||||
- Create migrations
|
||||
- Populate new data
|
||||
|
||||
**Verification:**
|
||||
```bash
|
||||
# 1. Run migrations
|
||||
docker exec igny8_backend python manage.py makemigrations
|
||||
docker exec igny8_backend python manage.py migrate
|
||||
|
||||
# 2. Run ALL tests - must pass
|
||||
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration -v 2
|
||||
|
||||
# 3. Manual verification - existing features still work
|
||||
# - Go to Writer page, generate content (check credits deducted)
|
||||
# - Go to Writer page, generate images (check credits deducted)
|
||||
# - Run automation manually (check it completes)
|
||||
```
|
||||
|
||||
**Rollback:** Delete new tables/fields (no existing data affected)
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Add Parallel Code Paths (OLD KEEPS WORKING)
|
||||
|
||||
**Changes:**
|
||||
- Add new methods to `ModelRegistry` (keep old ones)
|
||||
- Add new methods to `CreditService` (keep old ones)
|
||||
- Add comparison logging
|
||||
|
||||
**Pattern:**
|
||||
```python
|
||||
# model_registry.py
|
||||
@classmethod
|
||||
def get_default_model_NEW(cls, model_type: str):
|
||||
"""NEW: Get default model from AIModelConfig.is_default"""
|
||||
from igny8_core.business.billing.models import AIModelConfig
|
||||
return AIModelConfig.objects.filter(
|
||||
model_type=model_type,
|
||||
is_default=True,
|
||||
is_active=True
|
||||
).first()
|
||||
|
||||
@classmethod
|
||||
def get_model(cls, model_id: str):
|
||||
"""EXISTING: Keep working exactly as before"""
|
||||
# ... existing code unchanged ...
|
||||
```
|
||||
|
||||
```python
|
||||
# credit_service.py
|
||||
@staticmethod
|
||||
def calculate_credits_for_image_NEW(model_name: str, num_images: int) -> int:
|
||||
"""NEW: Calculate from AIModelConfig.credits_per_image"""
|
||||
from igny8_core.business.billing.models import AIModelConfig
|
||||
try:
|
||||
model = AIModelConfig.objects.get(model_name=model_name, is_active=True)
|
||||
if model.credits_per_image:
|
||||
return model.credits_per_image * num_images
|
||||
except AIModelConfig.DoesNotExist:
|
||||
pass
|
||||
return None # Signal to use old method
|
||||
|
||||
@staticmethod
|
||||
def calculate_credits_from_tokens(operation_type, tokens_input, tokens_output):
|
||||
"""EXISTING: Keep working, add comparison logging"""
|
||||
# Calculate old way
|
||||
old_result = cls._calculate_old(operation_type, tokens_input, tokens_output)
|
||||
|
||||
# Calculate new way (for comparison only)
|
||||
new_result = cls._calculate_new(operation_type, tokens_input, tokens_output)
|
||||
|
||||
# Log if different
|
||||
if old_result != new_result:
|
||||
logger.warning(f"[MIGRATION] Credit calc mismatch: {operation_type} old={old_result} new={new_result}")
|
||||
|
||||
# Return OLD (safe) for now
|
||||
return old_result
|
||||
```
|
||||
|
||||
**Verification:**
|
||||
```bash
|
||||
# 1. Run tests
|
||||
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration -v 2
|
||||
|
||||
# 2. Check logs for any mismatch warnings
|
||||
docker logs igny8_backend 2>&1 | grep "MIGRATION"
|
||||
|
||||
# 3. Manual test all paths
|
||||
```
|
||||
|
||||
**Rollback:** Remove new methods only
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Add IntegrationProvider for API Keys (OLD FALLBACK)
|
||||
|
||||
**Changes:**
|
||||
- `AICore._load_account_settings()` tries `IntegrationProvider` first, falls back to `GlobalIntegrationSettings`
|
||||
|
||||
**Pattern:**
|
||||
```python
|
||||
# ai_core.py
|
||||
def _load_account_settings(self):
|
||||
"""Load API keys - try new IntegrationProvider, fallback to old"""
|
||||
|
||||
# Try NEW way first
|
||||
try:
|
||||
from igny8_core.business.billing.models import IntegrationProvider
|
||||
openai_provider = IntegrationProvider.objects.filter(
|
||||
provider_id='openai', is_active=True
|
||||
).first()
|
||||
if openai_provider and openai_provider.api_key:
|
||||
self._openai_api_key = openai_provider.api_key
|
||||
logger.info("[MIGRATION] Loaded OpenAI key from IntegrationProvider")
|
||||
else:
|
||||
raise Exception("Fallback to old method")
|
||||
except Exception:
|
||||
# FALLBACK to old GlobalIntegrationSettings
|
||||
from igny8_core.modules.system.global_settings_models import GlobalIntegrationSettings
|
||||
global_settings = GlobalIntegrationSettings.get_instance()
|
||||
self._openai_api_key = global_settings.openai_api_key
|
||||
logger.info("[MIGRATION] Loaded OpenAI key from GlobalIntegrationSettings (fallback)")
|
||||
|
||||
# Same pattern for runware...
|
||||
```
|
||||
|
||||
**Verification:**
|
||||
```bash
|
||||
# 1. Run tests
|
||||
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration -v 2
|
||||
|
||||
# 2. Verify API calls work
|
||||
# - Generate content (uses OpenAI)
|
||||
# - Generate images with Runware model
|
||||
# - Generate images with DALL-E model
|
||||
|
||||
# 3. Check logs
|
||||
docker logs igny8_backend 2>&1 | grep "MIGRATION"
|
||||
```
|
||||
|
||||
**Rollback:** Revert `_load_account_settings()` to old version
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Switch to New Credit Calculation
|
||||
|
||||
**Changes:**
|
||||
- Use `AIModelConfig.credits_per_image` for image operations
|
||||
- Use `AIModelConfig.tokens_per_credit` for text operations
|
||||
- Keep `CreditCostConfig` as fallback
|
||||
|
||||
**Verification:**
|
||||
```bash
|
||||
# 1. Before switching, compare calculations for 1 week
|
||||
# Log both old and new results, verify they match
|
||||
|
||||
# 2. Run tests
|
||||
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration -v 2
|
||||
|
||||
# 3. Verify credit logs show correct values
|
||||
docker exec igny8_backend python manage.py shell -c "
|
||||
from igny8_core.business.billing.models import CreditUsageLog
|
||||
for log in CreditUsageLog.objects.order_by('-created_at')[:10]:
|
||||
print(f'{log.operation_type}: {log.credits_used} credits, model={log.model_used}')
|
||||
"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: Create API Endpoint
|
||||
|
||||
**Changes:**
|
||||
- Add `/api/v1/system/ai-models/` endpoint
|
||||
- Returns models from database
|
||||
|
||||
**Verification:**
|
||||
```bash
|
||||
# 1. Test endpoint
|
||||
curl -H "Authorization: Bearer $TOKEN" http://localhost:8000/api/v1/system/ai-models/
|
||||
|
||||
# 2. Verify response structure
|
||||
# Should have text_models, image_models, image_settings
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 6: Update Frontend
|
||||
|
||||
**Changes:**
|
||||
- Remove hardcoded model choices from `Settings.tsx`
|
||||
- Load from API
|
||||
|
||||
**Verification:**
|
||||
```bash
|
||||
# 1. Frontend tests
|
||||
cd frontend && npm run test
|
||||
|
||||
# 2. Manual verification
|
||||
# - Open Sites > Settings > Image Settings
|
||||
# - Verify dropdown shows models from API
|
||||
# - Change model, save, verify it persists
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 7: Cleanup (After 1 Week Stable)
|
||||
|
||||
**Remove from GlobalIntegrationSettings:**
|
||||
```python
|
||||
# API key fields (moved to IntegrationProvider)
|
||||
- openai_api_key
|
||||
- anthropic_api_key # unused
|
||||
- bria_api_key # unused
|
||||
- runware_api_key
|
||||
|
||||
# Model selection fields (now from AIModelConfig.is_default)
|
||||
- openai_model
|
||||
- dalle_model
|
||||
- runware_model
|
||||
- default_text_provider
|
||||
- default_image_service
|
||||
|
||||
# Hardcoded CHOICES
|
||||
- OPENAI_MODEL_CHOICES
|
||||
- DALLE_MODEL_CHOICES
|
||||
- RUNWARE_MODEL_CHOICES
|
||||
- TEXT_PROVIDER_CHOICES
|
||||
- IMAGE_SERVICE_CHOICES
|
||||
```
|
||||
|
||||
**Remove from constants.py:**
|
||||
```python
|
||||
- MODEL_RATES = {...}
|
||||
- IMAGE_MODEL_RATES = {...}
|
||||
```
|
||||
|
||||
**Remove from model_registry.py:**
|
||||
```python
|
||||
- Fallback to constants.py (all lookups must use DB)
|
||||
- Hardcoded default model names
|
||||
```
|
||||
|
||||
**Remove from ai_core.py:**
|
||||
```python
|
||||
- IMAGE_MODEL_RATES import and usage
|
||||
- Direct GlobalIntegrationSettings key access (use IntegrationProvider)
|
||||
```
|
||||
|
||||
**Remove CreditCostConfig entries (optional - use AIModelConfig):**
|
||||
```python
|
||||
- image_generation entry (use AIModelConfig.credits_per_image instead)
|
||||
```
|
||||
|
||||
**Remove from frontend Settings.tsx:**
|
||||
```python
|
||||
- QUALITY_TO_CONFIG hardcoded mapping
|
||||
- RUNWARE_MODEL_CHOICES hardcoded array
|
||||
- DALLE_MODEL_CHOICES hardcoded array
|
||||
- MODEL_LANDSCAPE_SIZES hardcoded mapping
|
||||
- Any hardcoded model names
|
||||
```
|
||||
|
||||
**Verification:**
|
||||
```bash
|
||||
# 1. Run full test suite
|
||||
docker exec igny8_backend python manage.py test
|
||||
|
||||
# 2. Run integration tests
|
||||
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration -v 2
|
||||
|
||||
# 3. Full manual test of all paths
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Complete Test Checklist
|
||||
|
||||
### Manual Test Checklist (Run After Each Phase)
|
||||
|
||||
#### Path 1: Manual Module Page Actions
|
||||
|
||||
| Test | Page | Button/Action | Expected Result | Verify Credits | Pass? |
|
||||
|------|------|---------------|-----------------|----------------|-------|
|
||||
| Cluster Keywords | Planner | "Cluster" button | Creates clusters | Check CreditUsageLog for `clustering` | ☐ |
|
||||
| Generate Ideas | Planner | "Generate Ideas" button | Creates ideas | Check CreditUsageLog for `idea_generation` | ☐ |
|
||||
| Generate Content | Writer | "Generate" button | Creates content | Check CreditUsageLog for `content_generation` | ☐ |
|
||||
| Generate Product Content | Writer | "Generate Product" button | Creates product content | Check CreditUsageLog for `content_generation` | ☐ |
|
||||
| Generate Service Content | Writer | "Generate Service" button | Creates service content | Check CreditUsageLog for `content_generation` | ☐ |
|
||||
| Extract Image Prompts | Writer | "Extract Prompts" button | Creates prompts | Check CreditUsageLog for `image_prompt_extraction` | ☐ |
|
||||
| Generate Images (Basic) | Writer | Image gen w/ Runware basic | Creates images | Check CreditUsageLog: 1 credit/image | ☐ |
|
||||
| Generate Images (Quality) | Writer | Image gen w/ DALL-E | Creates images | Check CreditUsageLog: 5 credits/image | ☐ |
|
||||
| Generate Images (Premium) | Writer | Image gen w/ Google | Creates images | Check CreditUsageLog: 15 credits/image | ☐ |
|
||||
| Internal Linking | Linker | "Generate Links" button | Creates links | Check CreditUsageLog for `internal_linking` | ☐ |
|
||||
| Optimize Content | Optimizer | "Optimize" button | Optimizes content | Check CreditUsageLog for `content_optimization` | ☐ |
|
||||
|
||||
#### Path 2: Automation Manual Run
|
||||
|
||||
| Test | Page | Action | Expected Result | Verify Credits | Pass? |
|
||||
|------|------|--------|-----------------|----------------|-------|
|
||||
| Run Full Automation | Automation | "Run Now" button | All stages complete | Credits for each stage | ☐ |
|
||||
| Run Stage 1 Only | Automation | Run clustering stage | Clusters created | Check `clustering` credits | ☐ |
|
||||
| Run Stage 2 Only | Automation | Run idea stage | Ideas created | Check `idea_generation` credits | ☐ |
|
||||
| Run Stage 4 Only | Automation | Run content stage | Content created | Check `content_generation` credits | ☐ |
|
||||
| Run Stage 5 Only | Automation | Run prompts stage | Prompts created | Check `image_prompt_extraction` credits | ☐ |
|
||||
| Run Stage 6 Only | Automation | Run image stage | Images created | Check `image_generation` credits | ☐ |
|
||||
| Pause/Resume | Automation | Pause then resume | Continues correctly | No double-charging | ☐ |
|
||||
|
||||
#### Path 3: Scheduled Automation
|
||||
|
||||
| Test | Setup | Expected | Verify | Pass? |
|
||||
|------|-------|----------|--------|-------|
|
||||
| Schedule Triggers | Set schedule for automation | Runs on schedule | Check logs at scheduled time | ☐ |
|
||||
| Credits Deducted | After scheduled run | Credits reduced | Check account balance | ☐ |
|
||||
| Multiple Automations | Multiple scheduled | All run | Each deducts credits | ☐ |
|
||||
|
||||
#### Credit Verification Queries
|
||||
|
||||
```bash
|
||||
# Check recent credit usage logs
|
||||
docker exec igny8_backend python manage.py shell -c "
|
||||
from igny8_core.business.billing.models import CreditUsageLog
|
||||
for log in CreditUsageLog.objects.order_by('-created_at')[:20]:
|
||||
print(f'{log.created_at}: {log.operation_type} | {log.credits_used} credits | model={log.model_used} | tokens={log.tokens_input}+{log.tokens_output}')
|
||||
"
|
||||
|
||||
# Check account balance
|
||||
docker exec igny8_backend python manage.py shell -c "
|
||||
from igny8_core.auth.models import Account
|
||||
for acc in Account.objects.all()[:5]:
|
||||
print(f'{acc.name}: {acc.credits} credits')
|
||||
"
|
||||
|
||||
# Check credit transactions
|
||||
docker exec igny8_backend python manage.py shell -c "
|
||||
from igny8_core.business.billing.models import CreditTransaction
|
||||
for txn in CreditTransaction.objects.order_by('-created_at')[:10]:
|
||||
print(f'{txn.created_at}: {txn.transaction_type} | {txn.credits_amount} | balance={txn.balance_after}')
|
||||
"
|
||||
```
|
||||
|
||||
#### Frontend Verification
|
||||
|
||||
| Test | Page | Action | Expected | Pass? |
|
||||
|------|------|--------|----------|-------|
|
||||
| Settings Load | Sites > Settings | Open page | Image settings dropdown populated | ☐ |
|
||||
| Model Selection | Sites > Settings | Change image model | Shows Basic/Quality/Premium | ☐ |
|
||||
| Model Persists | Sites > Settings | Save and reload | Selected model persists | ☐ |
|
||||
| Credits Display | Any page | Header/sidebar | Shows correct credit balance | ☐ |
|
||||
| Usage Analytics | Analytics | View usage | Shows correct breakdown | ☐ |
|
||||
|
||||
---
|
||||
|
||||
## Rollback Procedures
|
||||
|
||||
### If Phase 1 Fails
|
||||
```bash
|
||||
# Remove new migrations
|
||||
docker exec igny8_backend python manage.py migrate billing <previous_migration>
|
||||
docker exec igny8_backend python manage.py migrate system <previous_migration>
|
||||
```
|
||||
|
||||
### If Phase 2-6 Fails
|
||||
```bash
|
||||
# Revert code changes via git
|
||||
git checkout -- backend/igny8_core/ai/
|
||||
git checkout -- backend/igny8_core/business/billing/
|
||||
|
||||
# Restart containers
|
||||
docker-compose restart backend
|
||||
```
|
||||
|
||||
### If Phase 7 Fails (Cleanup)
|
||||
```bash
|
||||
# This is why we wait 1 week before cleanup
|
||||
# Restore from backup or revert git
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
Migration is COMPLETE when:
|
||||
|
||||
1. ✅ All integration tests pass
|
||||
2. ✅ All manual tests pass
|
||||
3. ✅ No errors in logs for 1 week
|
||||
4. ✅ Credit calculations match expected values
|
||||
5. ✅ All execution paths work (manual, automation, scheduled)
|
||||
6. ✅ Frontend loads models from API
|
||||
7. ✅ No legacy code remains:
|
||||
- No `MODEL_RATES` / `IMAGE_MODEL_RATES` in constants.py
|
||||
- No API keys in `GlobalIntegrationSettings`
|
||||
- No hardcoded models in frontend
|
||||
- No fallback to constants in `ModelRegistry`
|
||||
165
new-updated.md
Normal file
165
new-updated.md
Normal file
@@ -0,0 +1,165 @@
|
||||
## IGNY8 AI & Configuration Settings Report
|
||||
|
||||
---
|
||||
|
||||
### 1. AI Mode Configuration
|
||||
|
||||
**Architecture**: Two-tier database-driven model configuration system
|
||||
|
||||
#### AIModelConfig Model (Primary)
|
||||
Stores all AI model configurations in database, replacing legacy hardcoded constants.
|
||||
|
||||
**Model Types**:
|
||||
- Text Generation ✅
|
||||
- Image Generation ✅
|
||||
|
||||
|
||||
**Supported Providers**:
|
||||
- OpenAI ✅
|
||||
- Runware ✅
|
||||
|
||||
**Key Configuration Fields**:
|
||||
|
||||
| Category | Fields |
|
||||
|----------|--------|
|
||||
| **Identity** | `model_id`, `display_name`, `model_type`, `provider` |
|
||||
| **Text Pricing** | `input_token_rate`, `output_token_rate` (per 1M tokens in USD) |
|
||||
| **Text Limits** | `max_input_tokens`, `max_output_tokens` |
|
||||
| **Image Pricing** | `cost_per_image` (fixed USD per image) |
|
||||
| **Image Config** | `available_sizes` (JSON array of valid dimensions) |
|
||||
| **Status** | `is_active`, `is_default`, `sort_order` |
|
||||
| **Metadata** | `notes`, `release_date`, `deprecation_date` |
|
||||
|
||||
**Seeded Models**:
|
||||
- **OpenAI Text**: gpt-4o-mini, gpt-4o, gpt-5.1(default)
|
||||
- **OpenAI Image**: dall-e-3 (default)
|
||||
- **Runware**: runware:97@1, google:4@2
|
||||
|
||||
---
|
||||
|
||||
### 2. Global Integration Settings
|
||||
|
||||
**Model**: `GlobalIntegrationSettings` (Singleton - always pk=1)
|
||||
|
||||
Stores **platform-wide** API keys and default settings used by ALL accounts.
|
||||
|
||||
| Provider | API Key Field | Default Model | Parameters |
|
||||
|----------|---------------|---------------|------------|
|
||||
| **OpenAI** | `openai_api_key` | gpt-5.1 | temperature: 0.7, max_tokens: 8192 |
|
||||
| **DALL-E** | `dalle_api_key` | dall-e-3 | size: 1024x1024 |
|
||||
| **Runware** | `runware_api_key` | runware:97@1 & google:4@2 | — |
|
||||
|
||||
|
||||
**Default Provider Settings**:
|
||||
- `default_text_provider`: 'openai'
|
||||
- `default_image_service`: 'openai'
|
||||
|
||||
**Universal Image Settings**:
|
||||
- `image_quality`: standard/hd
|
||||
- `image_style`: photorealistic, illustration, etc.
|
||||
- `max_in_article_images`: Default 2
|
||||
|
||||
|
||||
**Critical Security Rule**: API keys exist ONLY in GlobalIntegrationSettings - never stored at account/user level.
|
||||
|
||||
---
|
||||
|
||||
|
||||
### 4. User-Specific Record Creation (Save Mechanism)
|
||||
|
||||
**Three-Tier Hierarchy**:
|
||||
```
|
||||
Global (Platform) → Account (Tenant) → User (Personal)
|
||||
```
|
||||
|
||||
#### Models Involved
|
||||
|
||||
| Model | Scope | Unique Key |
|
||||
|-------|-------|------------|
|
||||
| `GlobalIntegrationSettings` | Platform-wide | Singleton (pk=1) |
|
||||
| `AccountSettings` | Per-tenant | account + key |
|
||||
| `IntegrationSettings` | Per-tenant overrides | account + integration_type |
|
||||
| `UserSettings` | Per-user preferences | user + account + key |
|
||||
|
||||
#### Save Flow When User Changes Config
|
||||
|
||||
1. **Frontend** calls POST/PUT to `/api/v1/system/settings/user/`
|
||||
2. **Backend** ViewSet extracts user and account from authenticated request
|
||||
3. **Check existing**: Query for existing setting with same user + account + key
|
||||
4. **Create or Update**:
|
||||
- If not exists → `serializer.save(user=user, account=account)` creates new record
|
||||
- If exists → Updates the `value` JSON field
|
||||
5. **Validation**: Schema validation runs against `SETTINGS_SCHEMAS` before save
|
||||
6. **Response** returns the saved setting object
|
||||
|
||||
#### Integration Override Pattern
|
||||
|
||||
For AI/integration settings specifically:
|
||||
1. User changes model/temperature (NOT API keys)
|
||||
2. System strips any API key fields from request (security)
|
||||
3. `IntegrationSettings.objects.get_or_create(account=account, integration_type=type)`
|
||||
4. Only allowed override fields saved in `config` JSON field
|
||||
5. On read, system merges: Global defaults → Account overrides
|
||||
|
||||
---
|
||||
|
||||
### 6. Image Generation: Internal Cost vs Customer Credit Allocation
|
||||
|
||||
#### Internal Cost (What Platform Pays to Providers)
|
||||
|
||||
| Model | Provider | Cost Per Image (USD) |
|
||||
|-------|----------|---------------------|
|
||||
| dall-e-3 | OpenAI | $0.05 |
|
||||
| runware:97@1 | Runware - Hi Dream Full | ~$0.013 |
|
||||
| google:4@2 | Runware - Google Nano Banaan | ~$0.15 |
|
||||
|
||||
**Storage**: AIModelConfig.`cost_per_image` field + legacy `IMAGE_MODEL_RATES` constants
|
||||
|
||||
!!! This need to be fixed rates tobe laoded and used form configured AI Models !! not from hard coded location
|
||||
|
||||
#### Customer Credit Cost (What Customer Pays)
|
||||
|
||||
| Operation | Credits Charged | Price per Credit | Min Charge |
|
||||
|-----------|-----------------|------------------|------------|
|
||||
| Image Generation | 5 credits | $0.02 | $0.10 |
|
||||
| Image Prompt Extraction | 2 credits | $0.01 | $0.02 |
|
||||
|
||||
!!!morre robust image gneartion csoting and pricing mecahnishm required, withotu long chains or workarounds!!!
|
||||
|
||||
**Configuration Model**: `CreditCostConfig`
|
||||
- `tokens_per_credit`: 50 (image gen uses fewer tokens per credit = higher cost)
|
||||
- `min_credits`: 5
|
||||
- `price_per_credit_usd`: $0.02
|
||||
|
||||
#### Margin Calculation
|
||||
|
||||
| Metric | DALL-E 3 Example |
|
||||
|--------|------------------|
|
||||
| Provider Cost | $0.040 |
|
||||
| Customer Charge | $0.10 (5 credits × $0.02) |
|
||||
| **Margin** | $0.06 (60% of customer charge) |
|
||||
| **Markup** | ~150% |
|
||||
|
||||
**Flow**:
|
||||
1. **Before AI call**: Calculate required credits based on image count
|
||||
2. **Check balance**: Verify account has sufficient credits
|
||||
3. **Deduct credits**: Remove from balance, log transaction
|
||||
4. **Execute**: Make AI provider API call
|
||||
5. **Track**: `CreditUsageLog` stores both `credits_used` (customer) and `cost_usd` (actual)
|
||||
|
||||
**Revenue Analytics** queries `CreditUsageLog` to calculate:
|
||||
- Total revenue = Σ(credits_used × credit_price)
|
||||
- Total cost = Σ(cost_usd from provider)
|
||||
- Margin = Revenue - Cost
|
||||
|
||||
---
|
||||
|
||||
### Summary
|
||||
|
||||
The IGNY8 platform implements a sophisticated multi-tier configuration system:
|
||||
|
||||
- **AI configuration** is database-driven with fallback to legacy constants
|
||||
- **Global settings** hold platform API keys; accounts only override model/parameters
|
||||
- **User settings** create per-user records keyed by user + account + key combination
|
||||
- **Credit system** charges customers a markup (~150%) over actual provider costs
|
||||
- **Several fields are deprecated** including `mobile_image_size`, `reference_id`, and the `get_model()` method
|
||||
218
your-analysis.md
Normal file
218
your-analysis.md
Normal file
@@ -0,0 +1,218 @@
|
||||
## IGNY8 AI & Configuration Settings Report
|
||||
|
||||
---
|
||||
|
||||
### 1. AI Mode Configuration
|
||||
|
||||
**Architecture**: Two-tier database-driven model configuration system
|
||||
|
||||
#### AIModelConfig Model (Primary)
|
||||
Stores all AI model configurations in database, replacing legacy hardcoded constants.
|
||||
|
||||
**Model Types**:
|
||||
- Text Generation
|
||||
- Image Generation
|
||||
- Embedding
|
||||
|
||||
**Supported Providers**:
|
||||
- OpenAI
|
||||
- Anthropic
|
||||
- Runware
|
||||
- Google
|
||||
|
||||
**Key Configuration Fields**:
|
||||
|
||||
| Category | Fields |
|
||||
|----------|--------|
|
||||
| **Identity** | `model_id`, `display_name`, `model_type`, `provider` |
|
||||
| **Text Pricing** | `input_token_rate`, `output_token_rate` (per 1M tokens in USD) |
|
||||
| **Text Limits** | `max_input_tokens`, `max_output_tokens` |
|
||||
| **Image Pricing** | `cost_per_image` (fixed USD per image) |
|
||||
| **Image Config** | `available_sizes` (JSON array of valid dimensions) |
|
||||
| **Capabilities** | `supports_json_mode`, `supports_vision`, `supports_tools` |
|
||||
| **Status** | `is_active`, `is_default`, `sort_order` |
|
||||
| **Metadata** | `notes`, `release_date`, `deprecation_date` |
|
||||
|
||||
**Seeded Models**:
|
||||
- **OpenAI Text**: gpt-4.1 (default), gpt-4o-mini, gpt-4o, gpt-5.1, gpt-5.2
|
||||
- **Anthropic Text**: claude-3-5-sonnet, claude-3-opus, claude-3-haiku variants
|
||||
- **OpenAI Image**: dall-e-3 (default), dall-e-2, gpt-image-1, gpt-image-1-mini
|
||||
- **Runware/Bria**: runware:100@1, bria-2.3, bria-2.3-fast, bria-2.2
|
||||
|
||||
---
|
||||
|
||||
### 2. Global Integration Settings
|
||||
|
||||
**Model**: `GlobalIntegrationSettings` (Singleton - always pk=1)
|
||||
|
||||
Stores **platform-wide** API keys and default settings used by ALL accounts.
|
||||
|
||||
| Provider | API Key Field | Default Model | Parameters |
|
||||
|----------|---------------|---------------|------------|
|
||||
| **OpenAI** | `openai_api_key` | gpt-4o-mini | temperature: 0.7, max_tokens: 8192 |
|
||||
| **Anthropic** | `anthropic_api_key` | claude-3-5-sonnet-20241022 | temperature: 0.7, max_tokens: 8192 |
|
||||
| **DALL-E** | `dalle_api_key` | dall-e-3 | size: 1024x1024 |
|
||||
| **Runware** | `runware_api_key` | runware:97@1 | — |
|
||||
| **Bria** | `bria_api_key` | bria-2.3 | — |
|
||||
|
||||
**Default Provider Settings**:
|
||||
- `default_text_provider`: 'openai' or 'anthropic'
|
||||
- `default_image_service`: 'openai' or 'runware'
|
||||
|
||||
**Universal Image Settings**:
|
||||
- `image_quality`: standard/hd
|
||||
- `image_style`: photorealistic, illustration, etc.
|
||||
- `max_in_article_images`: Default 2
|
||||
- `desktop_image_size`: Default 1024x1024
|
||||
|
||||
**Critical Security Rule**: API keys exist ONLY in GlobalIntegrationSettings - never stored at account/user level.
|
||||
|
||||
---
|
||||
|
||||
### 3. Frontend Configuration Settings Panel
|
||||
|
||||
**Structure**: Three main setting hierarchies
|
||||
|
||||
#### Account Section (`/account/*`)
|
||||
| Page | Tabs | Purpose |
|
||||
|------|------|---------|
|
||||
| Account Settings | Account, Profile, Team | User account management |
|
||||
| Content Settings | Content, Publishing, Images | Content creation workflow |
|
||||
| Plans & Billing | Plan, Upgrade, Invoices | Subscription management |
|
||||
| Usage Analytics | Overview, Credits, Activity | Usage tracking |
|
||||
|
||||
#### Settings Section (`/settings/*`)
|
||||
| Page | Purpose |
|
||||
|------|---------|
|
||||
| General | Table settings, app preferences |
|
||||
| System | Global platform settings |
|
||||
| AI Settings | AI model configuration |
|
||||
| Integration | API integrations (Admin only) |
|
||||
| Publishing | Publishing destinations & rules |
|
||||
|
||||
#### Site-Level Settings (`/sites/:id/settings`)
|
||||
**Tabs**: general, content-generation, image-settings, integrations, publishing, content-types
|
||||
|
||||
**State Management**: Zustand store with persistence middleware (`useSettingsStore`)
|
||||
|
||||
**Available Settings Keys**:
|
||||
- `table_settings`: records_per_page, default_sort, sort_direction
|
||||
- `user_preferences`: theme, language, notifications
|
||||
- `ai_settings`: model overrides, temperature, max_tokens
|
||||
- `planner_automation`: automation rules
|
||||
- `writer_automation`: content generation rules
|
||||
|
||||
---
|
||||
|
||||
### 4. User-Specific Record Creation (Save Mechanism)
|
||||
|
||||
**Three-Tier Hierarchy**:
|
||||
```
|
||||
Global (Platform) → Account (Tenant) → User (Personal)
|
||||
```
|
||||
|
||||
#### Models Involved
|
||||
|
||||
| Model | Scope | Unique Key |
|
||||
|-------|-------|------------|
|
||||
| `GlobalIntegrationSettings` | Platform-wide | Singleton (pk=1) |
|
||||
| `AccountSettings` | Per-tenant | account + key |
|
||||
| `IntegrationSettings` | Per-tenant overrides | account + integration_type |
|
||||
| `UserSettings` | Per-user preferences | user + account + key |
|
||||
|
||||
#### Save Flow When User Changes Config
|
||||
|
||||
1. **Frontend** calls POST/PUT to `/api/v1/system/settings/user/`
|
||||
2. **Backend** ViewSet extracts user and account from authenticated request
|
||||
3. **Check existing**: Query for existing setting with same user + account + key
|
||||
4. **Create or Update**:
|
||||
- If not exists → `serializer.save(user=user, account=account)` creates new record
|
||||
- If exists → Updates the `value` JSON field
|
||||
5. **Validation**: Schema validation runs against `SETTINGS_SCHEMAS` before save
|
||||
6. **Response** returns the saved setting object
|
||||
|
||||
#### Integration Override Pattern
|
||||
|
||||
For AI/integration settings specifically:
|
||||
1. User changes model/temperature (NOT API keys)
|
||||
2. System strips any API key fields from request (security)
|
||||
3. `IntegrationSettings.objects.get_or_create(account=account, integration_type=type)`
|
||||
4. Only allowed override fields saved in `config` JSON field
|
||||
5. On read, system merges: Global defaults → Account overrides
|
||||
|
||||
---
|
||||
|
||||
### 5. Unused/Deprecated Fields
|
||||
|
||||
| Field/Item | Location | Status |
|
||||
|------------|----------|--------|
|
||||
| `reference_id` | CreditTransaction model | **DEPRECATED** - Use `payment` FK instead |
|
||||
| `mobile_image_size` | GlobalIntegrationSettings | **REMOVED** - No longer needed |
|
||||
| `max_items` parameter | validators.py | **Deprecated** - No longer enforced |
|
||||
| `get_model()` method | AICore class | **DEPRECATED** - Raises ValueError, model must be passed directly |
|
||||
| `run_request()` method | AICore class | **DEPRECATED** - Redirects to `run_ai_request()` |
|
||||
| `persist_task_metadata_to_content()` | MetadataMappingService | **DEPRECATED** - Content model no longer has task field |
|
||||
| `DeploymentService` | publishing/services/ | **DEPRECATED** - Legacy SiteBlueprint service |
|
||||
| SiteBlueprint model references | Multiple files | **REMOVED** - SiteBuilder deprecated |
|
||||
|
||||
---
|
||||
|
||||
### 6. Image Generation: Internal Cost vs Customer Credit Allocation
|
||||
|
||||
#### Internal Cost (What Platform Pays to Providers)
|
||||
|
||||
| Model | Provider | Cost Per Image (USD) |
|
||||
|-------|----------|---------------------|
|
||||
| dall-e-3 | OpenAI | $0.040 |
|
||||
| dall-e-2 | OpenAI | $0.020 |
|
||||
| gpt-image-1 | OpenAI | $0.042 |
|
||||
| gpt-image-1-mini | OpenAI | $0.011 |
|
||||
| runware:100@1 | Runware | ~$0.008-0.009 |
|
||||
| bria-2.3 | Bria | ~$0.015 |
|
||||
|
||||
**Storage**: AIModelConfig.`cost_per_image` field + legacy `IMAGE_MODEL_RATES` constants
|
||||
|
||||
#### Customer Credit Cost (What Customer Pays)
|
||||
|
||||
| Operation | Credits Charged | Price per Credit | Min Charge |
|
||||
|-----------|-----------------|------------------|------------|
|
||||
| Image Generation | 5 credits | $0.02 | $0.10 |
|
||||
| Image Prompt Extraction | 2 credits | $0.01 | $0.02 |
|
||||
|
||||
**Configuration Model**: `CreditCostConfig`
|
||||
- `tokens_per_credit`: 50 (image gen uses fewer tokens per credit = higher cost)
|
||||
- `min_credits`: 5
|
||||
- `price_per_credit_usd`: $0.02
|
||||
|
||||
#### Margin Calculation
|
||||
|
||||
| Metric | DALL-E 3 Example |
|
||||
|--------|------------------|
|
||||
| Provider Cost | $0.040 |
|
||||
| Customer Charge | $0.10 (5 credits × $0.02) |
|
||||
| **Margin** | $0.06 (60% of customer charge) |
|
||||
| **Markup** | ~150% |
|
||||
|
||||
**Flow**:
|
||||
1. **Before AI call**: Calculate required credits based on image count
|
||||
2. **Check balance**: Verify account has sufficient credits
|
||||
3. **Deduct credits**: Remove from balance, log transaction
|
||||
4. **Execute**: Make AI provider API call
|
||||
5. **Track**: `CreditUsageLog` stores both `credits_used` (customer) and `cost_usd` (actual)
|
||||
|
||||
**Revenue Analytics** queries `CreditUsageLog` to calculate:
|
||||
- Total revenue = Σ(credits_used × credit_price)
|
||||
- Total cost = Σ(cost_usd from provider)
|
||||
- Margin = Revenue - Cost
|
||||
|
||||
---
|
||||
|
||||
### Summary
|
||||
|
||||
The IGNY8 platform implements a sophisticated multi-tier configuration system:
|
||||
|
||||
- **AI configuration** is database-driven with fallback to legacy constants
|
||||
- **Global settings** hold platform API keys; accounts only override model/parameters
|
||||
- **User settings** create per-user records keyed by user + account + key combination
|
||||
- **Credit system** charges customers a markup (~150%) over actual provider costs
|
||||
- **Several fields are deprecated** including `mobile_image_size`, `reference_id`, and the `get_model()` method
|
||||
Reference in New Issue
Block a user