dos updates
This commit is contained in:
IGNY8 VPS (Salman)
@@ -1,343 +0,0 @@
|
||||
# 📋 COMPREHENSIVE IMAGE MODELS IMPLEMENTATION PLAN
|
||||
|
||||
## Model Reference Summary
|
||||
|
||||
| Model | AIR ID | Tier | Supported Dimensions (Square/Landscape) |
|
||||
|-------|--------|------|----------------------------------------|
|
||||
| **Hi Dream Full** | `runware:97@1` | Basic/Cheap | 1024×1024, 1280×768 (general diffusion) |
|
||||
| **Bria 3.2** | `bria:10@1` | Good Quality | 1024×1024, 1344×768 (16:9) or 1216×832 (3:2) |
|
||||
| **Nano Banana** | `google:4@2` | Premium | 1024×1024, 1376×768 (16:9) or 1264×848 (3:2) |
|
||||
|
||||
---
|
||||
|
||||
## TASK 1: Image Generation Progress Modal Width Increase
|
||||
|
||||
**Files to modify:**
|
||||
- ImageQueueModal.tsx
|
||||
|
||||
**Changes:**
|
||||
1. Locate the modal container `max-w-*` or width class
|
||||
2. Increase width by 50% (e.g., `max-w-2xl` → `max-w-4xl`, or explicit width)
|
||||
3. Ensure responsive behavior is maintained for smaller screens
|
||||
|
||||
---
|
||||
|
||||
## TASK 2: Fix Duplicate In-Article Image Names (Unique Field Storage)
|
||||
|
||||
**Files to modify:**
|
||||
- Backend: models.py (Images model)
|
||||
- Backend: generate_images.py
|
||||
- Backend: ai_processor.py
|
||||
|
||||
**Issue:** First 2 in-article images may have duplicate field names causing overwrite
|
||||
|
||||
**Changes:**
|
||||
1. Ensure `position` field is properly enforced (0, 1, 2, 3) for in-article images
|
||||
2. Update image creation logic to use unique combination: `content_id + image_type + position`
|
||||
3. Add validation to prevent duplicate position values for same content
|
||||
4. Ensure image storage generates unique filenames (timestamp + uuid + position)
|
||||
|
||||
---
|
||||
|
||||
## TASK 3: Image Settings Configuration - Remove Mobile Options
|
||||
|
||||
**Files to modify:**
|
||||
- Frontend: Settings.tsx (Site Settings)
|
||||
- Frontend: Integration.tsx
|
||||
- Backend: global_settings_models.py
|
||||
|
||||
**Changes:**
|
||||
1. Remove `mobile_enabled` option from settings UI
|
||||
2. Remove `mobile_image_size` option
|
||||
3. Remove `IMAGE_TYPE_CHOICES` → `mobile` option
|
||||
4. Clean up related state and form fields
|
||||
5. Update `ImageSettings` interface to remove mobile fields
|
||||
|
||||
---
|
||||
|
||||
## TASK 4: Fixed Image Sizes Configuration
|
||||
|
||||
**Specification:**
|
||||
- **Featured Image:** Fixed size (use primary model's best landscape dimension)
|
||||
- **In-Article Images:**
|
||||
- 2 × Square: `1024×1024`
|
||||
- 2 × Landscape: `1280×768` (for Hi Dream) / `1344×768` (for Bria/Nano Banana)
|
||||
|
||||
**Files to modify:**
|
||||
- Backend: global_settings_models.py
|
||||
- Backend: ai_core.py
|
||||
- Backend: AI functions for image generation
|
||||
|
||||
**Changes:**
|
||||
1. Add constants for fixed sizes:
|
||||
```
|
||||
FEATURED_SIZE = "1792x1024" (landscape, prominent)
|
||||
SQUARE_SIZE = "1024x1024"
|
||||
LANDSCAPE_SIZE = model-dependent (see model config)
|
||||
```
|
||||
2. Remove user-selectable size options where fixed
|
||||
3. Update global settings with fixed defaults
|
||||
|
||||
---
|
||||
|
||||
## TASK 5: Update AI Function Calls - Alternating Square/Landscape Pattern
|
||||
|
||||
**Files to modify:**
|
||||
- Backend: generate_images.py
|
||||
- Backend: ai_core.py (generate_image method)
|
||||
|
||||
**Pattern:** Request 4 in-article images alternating:
|
||||
- Image 1 (position 0): **Square** 1024×1024
|
||||
- Image 2 (position 1): **Landscape** 1280×768 or model-specific
|
||||
- Image 3 (position 2): **Square** 1024×1024
|
||||
- Image 4 (position 3): **Landscape** 1280×768 or model-specific
|
||||
|
||||
**Changes:**
|
||||
1. Modify `extract_image_prompts` to include size specification per image
|
||||
2. Update batch generation to pass correct dimensions based on position
|
||||
3. Store `aspect_ratio` or `dimensions` in Images model for template use
|
||||
|
||||
---
|
||||
|
||||
## TASK 6: Content View Template - Image Layout Rules
|
||||
|
||||
**Files to modify:**
|
||||
- Frontend: frontend/src/pages/Writer/components/ContentViewTemplate.tsx or similar template file
|
||||
|
||||
**Layout Rules:**
|
||||
| Image Shape | Layout Style |
|
||||
|-------------|--------------|
|
||||
| **Single Square** | 50% content width, centered or left-aligned |
|
||||
| **Single Landscape** | 100% content width |
|
||||
| **2 Square Images** | Side by side (50% each) |
|
||||
| **Square + Landscape** | Display individually per above rules |
|
||||
|
||||
**Changes:**
|
||||
1. Add `aspect_ratio` or `dimensions` detection from image record
|
||||
2. Create layout wrapper components:
|
||||
- `SingleSquareImage` (max-w-1/2)
|
||||
- `SingleLandscapeImage` (w-full)
|
||||
- `TwoSquareImages` (grid-cols-2)
|
||||
3. Update in-article image rendering to use layout rules
|
||||
4. Group consecutive square images for side-by-side display
|
||||
|
||||
---
|
||||
|
||||
## TASK 7: Backend AI Model Configuration Update
|
||||
|
||||
### 7A. Update Model Definitions in Database/Admin
|
||||
|
||||
**Files to modify:**
|
||||
- Backend: models.py (AIModelConfig)
|
||||
- Backend: admin (Admin configuration)
|
||||
- Backend: New migration file
|
||||
|
||||
**Changes:**
|
||||
1. **Add/Update AIModelConfig records for 3 models:**
|
||||
|
||||
| Model | Display Name | AIR ID | Tier |
|
||||
|-------|--------------|--------|------|
|
||||
| Hi Dream Full | "Hi Dream Full - Basic" | `runware:97@1` | Basic |
|
||||
| Bria 3.2 | "Bria 3.2 - Quality" | `bria:10@1` | Good |
|
||||
| Nano Banana | "Nano Banana - Premium" | `google:4@2` | Premium |
|
||||
|
||||
2. **Add new fields to AIModelConfig:**
|
||||
- `parameter_preset` (CharField with dropdown): quick config presets
|
||||
- `supported_sizes` (JSONField): checkboxes for valid dimensions
|
||||
- `one_liner_description` (CharField): brief model explainer
|
||||
|
||||
### 7B. Correct Parameter Configuration Per Model
|
||||
|
||||
**Based on Runware Documentation:**
|
||||
|
||||
#### **Hi Dream Full (runware:97@1)** - General Diffusion Model
|
||||
```python
|
||||
{
|
||||
"model": "runware:97@1",
|
||||
"steps": 20, # Default, adjustable 1-100
|
||||
"CFGScale": 7, # Default
|
||||
"scheduler": "Euler", # Default model scheduler
|
||||
"supported_dimensions": [
|
||||
"1024x1024", # 1:1 square
|
||||
"1280x768", # ~5:3 landscape (close to 16:9)
|
||||
"768x1280", # ~3:5 portrait
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
#### **Bria 3.2 (bria:10@1)** - Commercial-Ready
|
||||
```python
|
||||
{
|
||||
"model": "bria:10@1",
|
||||
"steps": 8, # Bria default: 4-10
|
||||
"supported_dimensions": [
|
||||
"1024x1024", # 1:1
|
||||
"1344x768", # 16:9 landscape
|
||||
"768x1344", # 9:16 portrait
|
||||
"1216x832", # 3:2 landscape
|
||||
"832x1216", # 2:3 portrait
|
||||
],
|
||||
"providerSettings": {
|
||||
"bria": {
|
||||
"promptEnhancement": true,
|
||||
"enhanceImage": true,
|
||||
"medium": "photography", # or "art"
|
||||
"contentModeration": true
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### **Nano Banana (google:4@2)** - Premium Quality
|
||||
```python
|
||||
{
|
||||
"model": "google:4@2",
|
||||
"supported_dimensions": [
|
||||
# 1K tier (default)
|
||||
"1024x1024", # 1:1
|
||||
"1376x768", # 16:9 landscape (1K)
|
||||
"768x1376", # 9:16 portrait (1K)
|
||||
"1264x848", # 3:2 landscape (1K)
|
||||
# 2K tier (optional)
|
||||
"2048x2048", # 1:1 2K
|
||||
"2752x1536", # 16:9 2K
|
||||
],
|
||||
"resolution": "1k" # or "2k", "4k"
|
||||
}
|
||||
```
|
||||
|
||||
### 7C. Admin Interface Enhancements
|
||||
|
||||
**Files to modify:**
|
||||
- Backend: backend/igny8_core/admin/billing_admin.py or similar
|
||||
|
||||
**Changes:**
|
||||
1. **Add model edit form with:**
|
||||
- Dropdown for `parameter_preset` with explainer tooltip
|
||||
- Checkboxes for `supported_sizes` (multi-select)
|
||||
- TextField for `one_liner_description`
|
||||
|
||||
2. **Preset Dropdown Options:**
|
||||
```
|
||||
- "Speed Optimized" (fewer steps, lighter scheduler)
|
||||
- "Balanced" (default settings)
|
||||
- "Quality Focused" (more steps, CFG tuning)
|
||||
```
|
||||
|
||||
3. **Size Checkboxes:**
|
||||
```
|
||||
☑ 1024×1024 (Square)
|
||||
☑ 1280×768 (Landscape)
|
||||
☐ 768×1280 (Portrait)
|
||||
☑ 1344×768 (Wide Landscape)
|
||||
```
|
||||
|
||||
### 7D. Global Integration Settings Update
|
||||
|
||||
**Files to modify:**
|
||||
- Backend: global_settings_models.py
|
||||
|
||||
**Changes:**
|
||||
1. Update `RUNWARE_MODEL_CHOICES`:
|
||||
```python
|
||||
RUNWARE_MODEL_CHOICES = [
|
||||
('runware:97@1', 'Hi Dream Full - Basic'),
|
||||
('bria:10@1', 'Bria 3.2 - Quality'),
|
||||
('google:4@2', 'Nano Banana - Premium'),
|
||||
]
|
||||
```
|
||||
|
||||
2. Add landscape size mapping per model:
|
||||
```python
|
||||
MODEL_LANDSCAPE_SIZES = {
|
||||
'runware:97@1': '1280x768',
|
||||
'bria:10@1': '1344x768',
|
||||
'google:4@2': '1376x768',
|
||||
}
|
||||
```
|
||||
|
||||
3. Add `default_square_size` = "1024x1024" (universal)
|
||||
|
||||
### 7E. AI Core Provider Settings
|
||||
|
||||
**Files to modify:**
|
||||
- Backend: ai_core.py
|
||||
|
||||
**Changes:**
|
||||
1. Update `_generate_image_runware` to handle provider-specific settings
|
||||
2. Add Bria-specific parameters when model is `bria:*`
|
||||
3. Add Google-specific handling for `google:*` models
|
||||
4. Implement model-aware dimension validation
|
||||
|
||||
---
|
||||
|
||||
## TASK 8: Frontend Integration Settings UI
|
||||
|
||||
**Files to modify:**
|
||||
- Frontend: Integration.tsx
|
||||
- Frontend: Settings.tsx
|
||||
|
||||
**Changes (for user override capability):**
|
||||
1. **Model Selection Dropdown:**
|
||||
- Hi Dream Full (Basic - Fast & Cheap)
|
||||
- Bria 3.2 (Quality - Commercial Safe)
|
||||
- Nano Banana (Premium - Best Quality)
|
||||
|
||||
2. **Image Size Selection (if override enabled):**
|
||||
- Square: 1024×1024 (all models)
|
||||
- Landscape: Auto-detected based on model OR user override
|
||||
|
||||
3. **Remove options:**
|
||||
- Remove desktop/mobile toggle
|
||||
- Remove custom size inputs
|
||||
- Use fixed sizes from configuration
|
||||
|
||||
---
|
||||
|
||||
## MIGRATION FILE REQUIREMENTS
|
||||
|
||||
Create migration: `0024_update_image_models_and_settings.py`
|
||||
|
||||
**Operations:**
|
||||
1. Add `parameter_preset`, `supported_sizes`, `one_liner_description` to AIModelConfig
|
||||
2. Update existing model records with new data
|
||||
3. Remove deprecated fields (mobile_image_size, etc.)
|
||||
4. Set default values for new fields
|
||||
|
||||
---
|
||||
|
||||
## EXECUTION ORDER
|
||||
|
||||
1. **Phase 1 - Backend Models & Config** (Tasks 7A, 7B, 7D, 7E) ✅
|
||||
2. **Phase 2 - Backend Logic** (Tasks 2, 5) ✅
|
||||
3. **Phase 3 - Frontend Settings** (Tasks 3, 8) ✅
|
||||
4. **Phase 4 - UI Components** (Tasks 1, 6) ✅
|
||||
5. **Phase 5 - Testing & Migration** (Run migrations in Docker)
|
||||
|
||||
---
|
||||
|
||||
## VALIDATION CHECKLIST
|
||||
|
||||
- [x] All 3 models configured with correct AIR IDs
|
||||
- [x] Parameter presets documented and selectable
|
||||
- [x] Image sizes correctly mapped per model
|
||||
- [x] No duplicate in-article images (unique storage)
|
||||
- [x] Modal width increased by 50%
|
||||
- [x] Mobile options removed from UI
|
||||
- [x] Alternating square/landscape pattern working
|
||||
- [x] Content template displays images per layout rules
|
||||
- [x] Global settings updated
|
||||
- [ ] Admin interface has model config dropdowns (optional enhancement)
|
||||
|
||||
---
|
||||
|
||||
Completed: *Create comprehensive implementation plan* (1/1)
|
||||
|
||||
This plan is ready for execution. The key technical findings are:
|
||||
|
||||
**Model Specifications Confirmed:**
|
||||
| Model | AIR ID | Square | Landscape | Steps |
|
||||
|-------|--------|--------|-----------|-------|
|
||||
| Hi Dream Full | `runware:97@1` | 1024×1024 | 1280×768 | 20 |
|
||||
| Bria 3.2 | `bria:10@1` | 1024×1024 | 1344×768 | 20 (min 20, max 50) |
|
||||
| Nano Banana | `google:4@2` | 1024×1024 | 1376×768 | Auto |
|
||||
@@ -1,214 +0,0 @@
|
||||
# AI Configuration Refactor - Master Overview
|
||||
|
||||
**Created:** January 4, 2026
|
||||
**Goal:** Clean, unified AI configuration with single source of truth
|
||||
|
||||
---
|
||||
|
||||
## Documentation Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| [REFACTOR-OVERVIEW.md](REFACTOR-OVERVIEW.md) | This file - master overview |
|
||||
| [implementation-plan-for-ai-models-and-cost.md](implementation-plan-for-ai-models-and-cost.md) | Detailed implementation plan with schema changes |
|
||||
| [safe-migration-and-testing-plan.md](safe-migration-and-testing-plan.md) | Testing strategy and verification steps |
|
||||
|
||||
---
|
||||
|
||||
## What We're Fixing
|
||||
|
||||
### Current Problems
|
||||
|
||||
1. **Duplicate configuration sources**
|
||||
- `AIModelConfig` (database) has model definitions
|
||||
- `GlobalIntegrationSettings` (database) has hardcoded CHOICES duplicating the same info
|
||||
- `constants.py` has `MODEL_RATES` and `IMAGE_MODEL_RATES` hardcoded
|
||||
- Frontend `Settings.tsx` has hardcoded model choices
|
||||
|
||||
2. **Broken/inconsistent data**
|
||||
- `GlobalIntegrationSettings.runware_model = "bria:10@1"` but this model doesn't exist
|
||||
- `GlobalIntegrationSettings.openai_model = "gpt-4o-mini"` but default should be `gpt-5.1`
|
||||
|
||||
3. **API keys mixed with model config**
|
||||
- `GlobalIntegrationSettings` stores both API keys and model preferences
|
||||
- No separation of concerns
|
||||
- Can't easily extend to other integrations (email, payment, etc.)
|
||||
|
||||
4. **Credit calculation scattered**
|
||||
- `CreditCostConfig` for token-based operations
|
||||
- `ModelRegistry.calculate_cost()` for image costs
|
||||
- No clear per-model credit rate
|
||||
|
||||
---
|
||||
|
||||
## Target State
|
||||
|
||||
### Single Source of Truth
|
||||
|
||||
```
|
||||
IntegrationProvider (NEW)
|
||||
├── openai (api_key, active)
|
||||
├── runware (api_key, active)
|
||||
├── resend (api_key, active) ← future
|
||||
├── stripe (api_key, webhook_secret, active) ← future
|
||||
└── ...
|
||||
|
||||
AIModelConfig (ENHANCED)
|
||||
├── Text Models
|
||||
│ ├── gpt-5.1 (is_default=true, tokens_per_credit=1000)
|
||||
│ ├── gpt-4o (tokens_per_credit=1000)
|
||||
│ └── gpt-4o-mini (tokens_per_credit=10000)
|
||||
└── Image Models
|
||||
├── runware:97@1 (credits_per_image=1, quality_tier=basic)
|
||||
├── dall-e-3 (is_default=true, credits_per_image=5, quality_tier=quality)
|
||||
└── google:4@2 (credits_per_image=15, quality_tier=premium)
|
||||
```
|
||||
|
||||
### No More Hardcoding
|
||||
|
||||
- **Backend**: Remove `MODEL_RATES`, `IMAGE_MODEL_RATES` from constants.py
|
||||
- **Backend**: Remove hardcoded CHOICES from `GlobalIntegrationSettings`
|
||||
- **Backend**: `ModelRegistry` always uses database, no fallbacks
|
||||
- **Frontend**: Load model choices from API endpoint
|
||||
|
||||
### Clear Credit Calculation
|
||||
|
||||
| Model Type | How Credits Calculated |
|
||||
|------------|----------------------|
|
||||
| Text | `ceil(total_tokens / tokens_per_credit)` from AIModelConfig |
|
||||
| Image | `credits_per_image * num_images` from AIModelConfig |
|
||||
|
||||
---
|
||||
|
||||
## Migration Phases Summary
|
||||
|
||||
| Phase | What Changes | Risk | Rollback |
|
||||
|-------|--------------|------|----------|
|
||||
| 1 | Add new models/fields | None - additive only | Drop new tables |
|
||||
| 2 | Add parallel code paths | Low - old code untouched | Remove new methods |
|
||||
| 3 | IntegrationProvider for keys | Medium - API key loading | Revert key loading |
|
||||
| 4 | Switch to new credit calc | Medium - financial impact | Revert credit service |
|
||||
| 5 | Create API endpoint | None - new endpoint | Remove endpoint |
|
||||
| 6 | Update frontend | Low - UI only | Revert frontend |
|
||||
| 7 | Cleanup legacy code | High if bugs found | Restore from backup |
|
||||
|
||||
**Rule:** Run full test suite after EACH phase before proceeding.
|
||||
|
||||
---
|
||||
|
||||
## Files Being Modified
|
||||
|
||||
### Backend Files
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `billing/models.py` | Add IntegrationProvider, add fields to AIModelConfig |
|
||||
| `ai/model_registry.py` | Add `get_default_model()`, remove constants fallback |
|
||||
| `ai/ai_core.py` | Use IntegrationProvider for API keys |
|
||||
| `ai/constants.py` | Remove MODEL_RATES, IMAGE_MODEL_RATES |
|
||||
| `billing/services/credit_service.py` | Model-based credit calculation |
|
||||
| `modules/system/global_settings_models.py` | Remove API keys, hardcoded choices |
|
||||
| `api/views/system.py` | Add `/api/v1/system/ai-models/` endpoint |
|
||||
|
||||
### Frontend Files
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `src/pages/Sites/Settings.tsx` | Load models from API, remove hardcodes |
|
||||
|
||||
### New Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `tests/test_ai_system_integration.py` | Integration tests for migration |
|
||||
|
||||
---
|
||||
|
||||
## Credit Values (Final)
|
||||
|
||||
### Text Models (per 1 credit)
|
||||
|
||||
| Model | Tokens per Credit | Example: 5000 tokens |
|
||||
|-------|-------------------|---------------------|
|
||||
| gpt-5.1 | 1,000 | 5 credits |
|
||||
| gpt-4o | 1,000 | 5 credits |
|
||||
| gpt-4o-mini | 10,000 | 1 credit |
|
||||
|
||||
### Image Models (per image)
|
||||
|
||||
| Model | Credits per Image | UI Display |
|
||||
|-------|-------------------|------------|
|
||||
| runware:97@1 | 1 | "Basic (1 credit/image)" |
|
||||
| dall-e-3 | 5 | "Quality (5 credits/image)" |
|
||||
| google:4@2 | 15 | "Premium (15 credits/image)" |
|
||||
|
||||
---
|
||||
|
||||
## Execution Paths That Must Keep Working
|
||||
|
||||
All these paths use the same AI functions and must work identically after migration:
|
||||
|
||||
1. **Manual Buttons** (Planner/Writer pages)
|
||||
- Cluster Keywords → `auto_cluster`
|
||||
- Generate Ideas → `generate_ideas`
|
||||
- Generate Content → `generate_content`
|
||||
- Extract Prompts → `generate_image_prompts`
|
||||
- Generate Images → `generate_images`
|
||||
|
||||
2. **Automation Manual Run** (Automation page → Run Now)
|
||||
- Same AI functions called via AutomationService
|
||||
|
||||
3. **Scheduled Automation** (Celery Beat scheduler)
|
||||
- Same AI functions called via scheduled tasks
|
||||
|
||||
4. **Direct Services** (Linker, Optimizer modules)
|
||||
- Use CreditService directly for credit deduction
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
Migration is **COMPLETE** when:
|
||||
|
||||
- [ ] All integration tests pass
|
||||
- [ ] All manual test checklist items pass
|
||||
- [ ] No errors in production logs for 1 week
|
||||
- [ ] Credit calculations match expected values
|
||||
- [ ] All execution paths work (manual, automation, scheduled)
|
||||
- [ ] Frontend loads models dynamically from API
|
||||
- [ ] No legacy code remains:
|
||||
- [ ] No `MODEL_RATES` / `IMAGE_MODEL_RATES` in constants.py
|
||||
- [ ] No API keys in `GlobalIntegrationSettings`
|
||||
- [ ] No hardcoded models in frontend
|
||||
- [ ] No fallback to constants in `ModelRegistry`
|
||||
- [ ] No unused fields in `GlobalIntegrationSettings`
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference Commands
|
||||
|
||||
```bash
|
||||
# Run integration tests
|
||||
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration -v 2
|
||||
|
||||
# Check AIModelConfig
|
||||
docker exec igny8_backend python manage.py shell -c "
|
||||
from igny8_core.business.billing.models import AIModelConfig
|
||||
for m in AIModelConfig.objects.all().order_by('model_type'):
|
||||
print(f'{m.model_name}: {m.model_type}, default={m.is_default}')
|
||||
"
|
||||
|
||||
# Check recent credit usage
|
||||
docker exec igny8_backend python manage.py shell -c "
|
||||
from igny8_core.business.billing.models import CreditUsageLog
|
||||
for log in CreditUsageLog.objects.order_by('-created_at')[:10]:
|
||||
print(f'{log.operation_type}: {log.credits_used} credits')
|
||||
"
|
||||
|
||||
# Check account credits
|
||||
docker exec igny8_backend python manage.py shell -c "
|
||||
from igny8_core.auth.models import Account
|
||||
for acc in Account.objects.all():
|
||||
print(f'{acc.name}: {acc.credits} credits')
|
||||
"
|
||||
```
|
||||
@@ -1,254 +0,0 @@
|
||||
# Django Admin Cleanup - Implementation Status
|
||||
|
||||
## Status: COMPLETED (January 4, 2026)
|
||||
|
||||
---
|
||||
|
||||
## What Was Done
|
||||
|
||||
### 1. Fixed Duplicate Model Registration
|
||||
**File:** `backend/igny8_core/business/billing/admin.py`
|
||||
|
||||
- `AccountPaymentMethod` was registered in BOTH:
|
||||
- `modules/billing/admin.py` (with AccountAdminMixin - KEPT)
|
||||
- `business/billing/admin.py` (simpler version - REMOVED)
|
||||
- Commented out the duplicate registration in `business/billing/admin.py`
|
||||
|
||||
### 2. Simplified Admin Site Configuration
|
||||
**File:** `backend/igny8_core/admin/site.py`
|
||||
|
||||
- Removed complex `get_app_list()` override (was 250+ lines)
|
||||
- Removed `get_sidebar_list()` override
|
||||
- Removed `each_context()` override with debug logging
|
||||
- Kept only:
|
||||
- Custom URLs for dashboard, reports, and monitoring
|
||||
- Index redirect to dashboard
|
||||
- Navigation is now handled by Unfold's built-in `SIDEBAR.navigation` setting
|
||||
|
||||
### 3. Added Proper Unfold Navigation Configuration
|
||||
**File:** `backend/igny8_core/settings.py`
|
||||
|
||||
Added complete `UNFOLD["SIDEBAR"]["navigation"]` config with:
|
||||
- Dashboard link
|
||||
- Reports section (6 reports)
|
||||
- Accounts & Users group
|
||||
- Plans & Billing group
|
||||
- Credits group
|
||||
- Planning group
|
||||
- Writing group
|
||||
- Taxonomy group
|
||||
- Publishing group
|
||||
- Automation group
|
||||
- AI Configuration group (NEW - consolidated)
|
||||
- Global Settings group
|
||||
- Resources group
|
||||
- Logs & Monitoring group
|
||||
- Django Admin group
|
||||
|
||||
Each item has proper Material Design icons.
|
||||
|
||||
### 4. Added Site Logo Configuration
|
||||
**File:** `backend/igny8_core/settings.py`
|
||||
|
||||
```python
|
||||
"SITE_ICON": {
|
||||
"light": lambda request: "/static/admin/img/logo-light.svg",
|
||||
"dark": lambda request: "/static/admin/img/logo-dark.svg",
|
||||
},
|
||||
```
|
||||
|
||||
**Note:** Logo SVG files need to be created at these paths for the logo to display.
|
||||
|
||||
---
|
||||
|
||||
## Verification Results
|
||||
|
||||
```bash
|
||||
# Django system check
|
||||
$ docker exec igny8_backend python manage.py check
|
||||
System check identified no issues (0 silenced).
|
||||
|
||||
# Admin registry test
|
||||
$ docker exec igny8_backend python manage.py shell -c "..."
|
||||
Total registered models: 63
|
||||
Admin site ready!
|
||||
|
||||
# UNFOLD config test
|
||||
Navigation items: 20
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## What Was NOT Done (and why)
|
||||
|
||||
### Models NOT Hidden from Admin
|
||||
|
||||
These models were originally planned for removal but are **actively used**:
|
||||
|
||||
| Model | Reason Kept |
|
||||
|-------|-------------|
|
||||
| `IntegrationSettings` | Used by AI functions, settings, integration views |
|
||||
| `AIPrompt` | Used by ai/prompts.py, management commands |
|
||||
| `AuthorProfile` | Used by content generation |
|
||||
| `Strategy` | Used by content planning |
|
||||
|
||||
---
|
||||
|
||||
## Admin Sidebar Structure (Final)
|
||||
|
||||
```
|
||||
Dashboard
|
||||
Reports
|
||||
├── Revenue
|
||||
├── Usage
|
||||
├── Content
|
||||
├── Data Quality
|
||||
├── Token Usage
|
||||
└── AI Cost Analysis
|
||||
|
||||
─── Core ───
|
||||
Accounts & Users
|
||||
├── Accounts
|
||||
├── Users
|
||||
├── Sites
|
||||
├── Sectors
|
||||
└── Site Access
|
||||
|
||||
Plans & Billing
|
||||
├── Plans
|
||||
├── Subscriptions
|
||||
├── Invoices
|
||||
├── Payments
|
||||
├── Credit Packages
|
||||
└── Payment Methods
|
||||
|
||||
Credits
|
||||
├── Transactions
|
||||
├── Usage Log
|
||||
└── Plan Limits
|
||||
|
||||
─── Content ───
|
||||
Planning
|
||||
├── Keywords
|
||||
├── Clusters
|
||||
└── Content Ideas
|
||||
|
||||
Writing
|
||||
├── Tasks
|
||||
├── Content
|
||||
├── Images
|
||||
└── Image Prompts
|
||||
|
||||
Taxonomy
|
||||
├── Taxonomies
|
||||
├── Relations
|
||||
├── Attributes
|
||||
└── Cluster Maps
|
||||
|
||||
─── Automation ───
|
||||
Publishing
|
||||
├── Integrations
|
||||
├── Publishing Records
|
||||
├── Deployments
|
||||
└── Sync Events
|
||||
|
||||
Automation
|
||||
├── Configs
|
||||
└── Runs
|
||||
|
||||
─── Configuration ───
|
||||
AI Configuration
|
||||
├── AI Models
|
||||
├── Credit Costs
|
||||
├── Billing Config
|
||||
└── AI Task Logs
|
||||
|
||||
Global Settings
|
||||
├── Integration Settings
|
||||
├── Module Settings
|
||||
├── AI Prompts
|
||||
├── Author Profiles
|
||||
└── Strategies
|
||||
|
||||
Resources
|
||||
├── Industries
|
||||
├── Industry Sectors
|
||||
└── Seed Keywords
|
||||
|
||||
─── System ───
|
||||
Logs & Monitoring
|
||||
├── System Health
|
||||
├── API Monitor
|
||||
├── Debug Console
|
||||
├── Celery Tasks
|
||||
└── Admin Log
|
||||
|
||||
Django Admin
|
||||
├── Groups
|
||||
├── Permissions
|
||||
├── Content Types
|
||||
└── Sessions
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Files Changed
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `backend/igny8_core/settings.py` | Added full UNFOLD navigation config |
|
||||
| `backend/igny8_core/admin/site.py` | Simplified to ~60 lines (was ~350) |
|
||||
| `backend/igny8_core/business/billing/admin.py` | Commented out duplicate AccountPaymentMethod |
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: AI Models & Credits Refactor - COMPLETED
|
||||
|
||||
### IntegrationProvider Model Created
|
||||
- New model: `IntegrationProvider` in `modules/system/models.py`
|
||||
- Centralized storage for ALL external service API keys
|
||||
- Supports: AI providers, payment gateways, email services, storage
|
||||
- Migrated OpenAI and Runware API keys from GlobalIntegrationSettings
|
||||
- Admin interface added in `modules/system/admin.py`
|
||||
- Added to admin sidebar under "Global Settings"
|
||||
|
||||
### AIModelConfig Enhanced
|
||||
- Added `tokens_per_credit` - for text models (e.g., 1000 tokens = 1 credit)
|
||||
- Added `credits_per_image` - for image models (e.g., 1, 5, 15 credits)
|
||||
- Added `quality_tier` - for frontend UI (basic/quality/premium)
|
||||
- Migration `0025_add_aimodel_credit_fields` adds fields
|
||||
- Migration `0026_populate_aimodel_credits` sets initial values
|
||||
|
||||
### ModelRegistry Updated
|
||||
- Removed fallback to `constants.py` - database is now authoritative
|
||||
- Added `get_provider()`, `get_api_key()`, `get_api_secret()`, `get_webhook_secret()`
|
||||
- Provider caching with TTL
|
||||
|
||||
### CreditService Updated
|
||||
- Added `calculate_credits_for_image(model_name, num_images)` - uses AIModelConfig.credits_per_image
|
||||
- Added `calculate_credits_from_tokens_by_model(model_name, total_tokens)` - uses AIModelConfig.tokens_per_credit
|
||||
- Added `deduct_credits_for_image()` - convenience method
|
||||
|
||||
### Files Changed (Phase 2)
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `modules/system/models.py` | Added IntegrationProvider model |
|
||||
| `modules/system/admin.py` | Added IntegrationProviderAdmin |
|
||||
| `business/billing/models.py` | Added tokens_per_credit, credits_per_image, quality_tier to AIModelConfig |
|
||||
| `business/billing/services/credit_service.py` | Added image/model-based credit calculation |
|
||||
| `ai/model_registry.py` | Removed constants fallback, added provider methods |
|
||||
| `ai/ai_core.py` | Use ModelRegistry for API keys, removed constants fallback |
|
||||
| `ai/constants.py` | Marked MODEL_RATES, IMAGE_MODEL_RATES as DEPRECATED |
|
||||
| `ai/settings.py` | Use ModelRegistry for model validation |
|
||||
| `ai/validators.py` | Removed constants fallback |
|
||||
| `modules/system/integration_views.py` | Use ModelRegistry for cost calculation |
|
||||
| `modules/billing/serializers.py` | Added new fields to AIModelConfigSerializer |
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. ✅ **Django Admin Cleanup** - DONE
|
||||
2. ⏳ **Simplify AI Settings** - Merge content + image settings into AccountSettings
|
||||
3. ✅ **Create IntegrationProvider** - DONE (API keys now in dedicated model)
|
||||
4. ✅ **AIModelConfig Enhancement** - DONE (tokens_per_credit, credits_per_image, quality_tier added)
|
||||
@@ -1,491 +0,0 @@
|
||||
|
||||
## Complete End-to-End Analysis & Restructuring Plan
|
||||
|
||||
### Current State Summary (from actual database queries)
|
||||
|
||||
**AIModelConfig (6 records - all needed):**
|
||||
| model_name | type | provider | active | default | cost | tokens_per_credit |
|
||||
|------------|------|----------|--------|---------|------|-------------------|
|
||||
| gpt-4o-mini | text | openai | ✅ | ❌ | $0.15/$0.60 per 1M | 10,000 |
|
||||
| gpt-4o | text | openai | ❌ | ❌ | $2.50/$10.00 per 1M | 1,000 |
|
||||
| gpt-5.1 | text | openai | ✅ | ✅ | $1.25/$10.00 per 1M | 1,000 |
|
||||
| runware:97@1 | image | runware | ✅ | ❌ | $0.012/image | 1 credit/image |
|
||||
| dall-e-3 | image | openai | ✅ | ✅ | $0.04/image | 5 credits/image |
|
||||
| google:4@2 | image | runware | ✅ | ❌ | $0.14/image | 15 credits/image |
|
||||
|
||||
---
|
||||
|
||||
### Current End-to-End Flow (Traced from Code)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ CURRENT ARCHITECTURE │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
1. CONFIGURATION LAYER
|
||||
┌────────────────────────────────────────┐
|
||||
│ GlobalIntegrationSettings (singleton) │ ← API keys stored here
|
||||
│ - openai_api_key │
|
||||
│ - runware_api_key │
|
||||
│ - anthropic_api_key (unused) │
|
||||
│ - bria_api_key (unused) │
|
||||
│ - openai_model: gpt-4o-mini ❌ │ ← Should be gpt-5.1
|
||||
│ - runware_model: bria:10@1 ❌ │ ← Model doesn't exist!
|
||||
│ - HARDCODED CHOICES duplicating DB │
|
||||
└────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────────────┐
|
||||
│ AIModelConfig (database) │ ← Source of truth for models
|
||||
│ - model_name, provider, costs │
|
||||
│ - is_active, is_default │
|
||||
└────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────────────┐
|
||||
│ constants.py │ ← DUPLICATE/LEGACY
|
||||
│ - MODEL_RATES (hardcoded) │
|
||||
│ - IMAGE_MODEL_RATES (hardcoded) │
|
||||
└────────────────────────────────────────┘
|
||||
|
||||
2. SETTINGS RESOLUTION LAYER
|
||||
┌────────────────────────────────────────┐
|
||||
│ settings.py │
|
||||
│ get_model_config(function, account) │
|
||||
│ - Gets model from GlobalIntegration │
|
||||
│ - Gets max_tokens from AIModelConfig │
|
||||
│ - Allows IntegrationSettings override │
|
||||
└────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────────────┐
|
||||
│ model_registry.py │
|
||||
│ ModelRegistry.get_model(model_id) │
|
||||
│ - Try DB (AIModelConfig) first │
|
||||
│ - Fallback to constants.py ❌ │ ← Should only use DB
|
||||
└────────────────────────────────────────┘
|
||||
|
||||
3. AI EXECUTION LAYER
|
||||
┌────────────────────────────────────────┐
|
||||
│ engine.py (AIEngine) │
|
||||
│ - Orchestrates all AI functions │
|
||||
│ - Progress tracking, cost tracking │
|
||||
└────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────────────┐
|
||||
│ ai_core.py (AICore) │
|
||||
│ - _load_account_settings() │ ← Gets API keys from Global
|
||||
│ - run_ai_request() for text │
|
||||
│ - generate_image() for images │
|
||||
│ - Uses IMAGE_MODEL_RATES fallback ❌ │
|
||||
└────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────────────┐
|
||||
│ ai/functions/ │
|
||||
│ - generate_images.py │
|
||||
│ - generate_content.py │
|
||||
│ - auto_cluster.py │
|
||||
│ - Each function uses AICore │
|
||||
└────────────────────────────────────────┘
|
||||
|
||||
4. CREDIT CALCULATION LAYER
|
||||
┌────────────────────────────────────────┐
|
||||
│ CreditCostConfig (database) │
|
||||
│ - operation_type │
|
||||
│ - tokens_per_credit │
|
||||
│ - min_credits │
|
||||
│ - price_per_credit_usd │
|
||||
│ - For images: 50 tokens/credit, min 5 │
|
||||
└────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────────────┐
|
||||
│ CreditService │
|
||||
│ - calculate_credits_from_tokens() │
|
||||
│ - deduct_credits_for_operation() │
|
||||
│ - Text: tokens → credits AFTER call │
|
||||
│ - Images: ??? (not token-based) │
|
||||
└────────────────────────────────────────┘
|
||||
|
||||
5. FRONTEND (Sites/Settings.tsx)
|
||||
┌────────────────────────────────────────┐
|
||||
│ HARDCODED model choices ❌ │
|
||||
│ - QUALITY_TO_CONFIG │
|
||||
│ - RUNWARE_MODEL_CHOICES │
|
||||
│ - DALLE_MODEL_CHOICES │
|
||||
│ - MODEL_LANDSCAPE_SIZES │
|
||||
└────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Problems Identified
|
||||
|
||||
| # | Problem | Location | Impact |
|
||||
|---|---------|----------|--------|
|
||||
| 1 | GlobalIntegrationSettings has hardcoded model choices | global_settings_models.py | Duplicates AIModelConfig |
|
||||
| 2 | runware_model = "bria:10@1" but model doesn't exist | GlobalIntegrationSettings | Broken fallback |
|
||||
| 3 | API keys mixed with model config | GlobalIntegrationSettings | No separation of concerns |
|
||||
| 4 | IMAGE_MODEL_RATES still used as fallback | ai_core.py, model_registry.py | Inconsistent pricing |
|
||||
| 5 | Frontend hardcodes model choices | Settings.tsx | Not dynamic |
|
||||
| 6 | Image credit calculation unclear | CreditService | Not based on cost_per_image |
|
||||
| 7 | constants.py duplicates DB data | constants.py | Maintenance burden |
|
||||
|
||||
---
|
||||
|
||||
### Target Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ TARGET ARCHITECTURE │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
1. NEW: IntegrationProvider Model (stores ALL 3rd party API keys)
|
||||
┌────────────────────────────────────────┐
|
||||
│ IntegrationProvider │ ← Future-proof: ALL integrations
|
||||
│ - provider_id: str (primary key) │
|
||||
│ Examples: openai, runware, google, │
|
||||
│ resend, stripe, etc. │
|
||||
│ - display_name: str │
|
||||
│ - provider_type: ai | email | payment │
|
||||
│ - api_key: encrypted str │
|
||||
│ - api_endpoint: URL (optional) │
|
||||
│ - is_active: bool │
|
||||
│ - config: JSON (rate limits, etc.) │
|
||||
└────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
2. CLEANED: AIModelConfig (references IntegrationProvider)
|
||||
┌────────────────────────────────────────┐
|
||||
│ AIModelConfig │
|
||||
│ - model_name: str │
|
||||
│ - display_name: str │
|
||||
│ - model_type: text | image │
|
||||
│ - provider: str → IntegrationProvider │
|
||||
│ - cost fields (unchanged) │
|
||||
│ + credits_per_image: int (NEW) │ ← For image models
|
||||
│ + tokens_per_credit: int (NEW) │ ← For text models
|
||||
│ + quality_tier: basic|quality|premium │ ← For UI display
|
||||
│ - is_default: bool │ ← Loads automatically
|
||||
└────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
3. SIMPLIFIED: GlobalIntegrationSettings
|
||||
┌────────────────────────────────────────┐
|
||||
│ GlobalIntegrationSettings │
|
||||
│ NO hardcoded model names │
|
||||
│ NO API keys (in IntegrationProvider) │
|
||||
│ Loads defaults from AIModelConfig │
|
||||
│ where is_default=True │
|
||||
│ - image_style: str │
|
||||
│ - max_in_article_images: int │
|
||||
│ - image_quality: str │
|
||||
└────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
4. UNIFIED: Model Resolution
|
||||
┌────────────────────────────────────────┐
|
||||
│ ModelRegistry │
|
||||
│ - get_default_model(type) → from DB │
|
||||
│ - get_model(model_id) → AIModelConfig │
|
||||
│ - get_provider(id) → IntegrationProv │
|
||||
│ - get_api_key(provider) → key │
|
||||
│ - NO fallback to constants │
|
||||
│ - NO hardcoded defaults │
|
||||
└────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
5. DYNAMIC: Frontend API
|
||||
┌────────────────────────────────────────┐
|
||||
│ GET /api/v1/system/ai-models/ │
|
||||
│ Returns models from DB with defaults │
|
||||
│ marked, no hardcoding needed │
|
||||
└────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Implementation Plan (Complete)
|
||||
|
||||
#### Phase 1: Database Schema Changes
|
||||
|
||||
**1.1 Create IntegrationProvider Model (Future-proof for ALL integrations)**
|
||||
|
||||
File: models.py
|
||||
|
||||
```python
|
||||
class IntegrationProvider(models.Model):
|
||||
"""
|
||||
Centralized 3rd party integration provider configuration.
|
||||
Single location for ALL external service API keys and configs.
|
||||
"""
|
||||
PROVIDER_TYPE_CHOICES = [
|
||||
('ai', 'AI Provider'),
|
||||
('email', 'Email Service'),
|
||||
('payment', 'Payment Gateway'),
|
||||
('storage', 'Storage Service'),
|
||||
('other', 'Other'),
|
||||
]
|
||||
|
||||
provider_id = models.CharField(max_length=50, unique=True, primary_key=True)
|
||||
# Examples: openai, runware, google, resend, stripe, aws_s3, etc.
|
||||
|
||||
display_name = models.CharField(max_length=100)
|
||||
provider_type = models.CharField(max_length=20, choices=PROVIDER_TYPE_CHOICES, default='ai')
|
||||
api_key = models.CharField(max_length=500, blank=True) # Should be encrypted
|
||||
api_secret = models.CharField(max_length=500, blank=True) # For services needing secret
|
||||
api_endpoint = models.URLField(blank=True) # Custom endpoint if needed
|
||||
webhook_secret = models.CharField(max_length=500, blank=True) # For Stripe etc.
|
||||
is_active = models.BooleanField(default=True)
|
||||
config = models.JSONField(default=dict, blank=True) # Rate limits, regions, etc.
|
||||
created_at = models.DateTimeField(auto_now_add=True)
|
||||
updated_at = models.DateTimeField(auto_now=True)
|
||||
|
||||
class Meta:
|
||||
db_table = 'igny8_integration_providers'
|
||||
verbose_name = 'Integration Provider'
|
||||
verbose_name_plural = 'Integration Providers'
|
||||
```
|
||||
|
||||
**1.2 Add fields to AIModelConfig**
|
||||
|
||||
```python
|
||||
# Add to AIModelConfig for IMAGE models
|
||||
credits_per_image = models.IntegerField(
|
||||
null=True, blank=True,
|
||||
help_text="Fixed credits per image generated. For image models only."
|
||||
)
|
||||
|
||||
# Add to AIModelConfig for TEXT models
|
||||
tokens_per_credit = models.IntegerField(
|
||||
null=True, blank=True,
|
||||
help_text="Number of tokens that equal 1 credit. For text models only."
|
||||
)
|
||||
|
||||
# Add quality tier for UI display (image models)
|
||||
quality_tier = models.CharField(
|
||||
max_length=20,
|
||||
choices=[('basic', 'Basic'), ('quality', 'Quality'), ('premium', 'Premium')],
|
||||
null=True, blank=True,
|
||||
help_text="Quality tier for frontend UI display"
|
||||
)
|
||||
```
|
||||
|
||||
**1.3 Migration to populate data**
|
||||
|
||||
Create IntegrationProvider records:
|
||||
```
|
||||
| provider_id | display_name | provider_type | Notes |
|
||||
|-------------|-------------------|---------------|--------------------------|
|
||||
| openai | OpenAI | ai | GPT models, DALL-E |
|
||||
| runware | Runware | ai | Image generation |
|
||||
| google | Google Cloud | ai | Future: Gemini, etc. |
|
||||
| resend | Resend | email | Transactional email |
|
||||
| stripe | Stripe | payment | Payment processing |
|
||||
```
|
||||
|
||||
Update AIModelConfig with credit/token values:
|
||||
```
|
||||
| model_name | type | tokens_per_credit | credits_per_image | quality_tier |
|
||||
|---------------|-------|-------------------|-------------------|--------------|
|
||||
| gpt-4o-mini | text | 10000 | - | - |
|
||||
| gpt-4o | text | 1000 | - | - |
|
||||
| gpt-5.1 | text | 1000 | - | - |
|
||||
| runware:97@1 | image | - | 1 | basic |
|
||||
| dall-e-3 | image | - | 5 | quality |
|
||||
| google:4@2 | image | - | 15 | premium |
|
||||
```
|
||||
|
||||
#### Phase 2: Backend Code Changes
|
||||
|
||||
**2.1 Remove hardcoded constants**
|
||||
|
||||
File: constants.py
|
||||
- Remove: MODEL_RATES, IMAGE_MODEL_RATES
|
||||
- Keep: JSON_MODE_MODELS (or move to AIModelConfig.supports_json_mode check)
|
||||
|
||||
**2.2 Update ModelRegistry**
|
||||
|
||||
File: model_registry.py
|
||||
- Remove fallback to constants.py
|
||||
- Add: `get_default_model(model_type) → AIModelConfig where is_default=True`
|
||||
- Add: `get_provider(provider_id) → IntegrationProvider`
|
||||
- Add: `get_api_key(provider_id) → str`
|
||||
- **NO hardcoded model names** - always query DB
|
||||
|
||||
**2.3 Update AICore**
|
||||
|
||||
File: ai_core.py
|
||||
- Change `_load_account_settings()` to use IntegrationProvider for API keys
|
||||
- Remove IMAGE_MODEL_RATES import and usage
|
||||
- Use `ModelRegistry.get_default_model('text')` instead of hardcoded model
|
||||
- Use `ModelRegistry.calculate_cost()` exclusively
|
||||
|
||||
**2.4 Update CreditService**
|
||||
|
||||
File: credit_service.py
|
||||
|
||||
For IMAGE models:
|
||||
```python
|
||||
def calculate_credits_for_image(model_name: str, num_images: int) -> int:
|
||||
"""Calculate credits for image generation from AIModelConfig"""
|
||||
model = AIModelConfig.objects.get(model_name=model_name, is_active=True)
|
||||
return model.credits_per_image * num_images
|
||||
```
|
||||
|
||||
For TEXT models:
|
||||
```python
|
||||
def calculate_credits_from_tokens(model_name: str, total_tokens: int) -> int:
|
||||
"""Calculate credits from token usage based on model's tokens_per_credit"""
|
||||
model = AIModelConfig.objects.get(model_name=model_name, is_active=True)
|
||||
tokens_per_credit = model.tokens_per_credit or 1000 # fallback
|
||||
return math.ceil(total_tokens / tokens_per_credit)
|
||||
```
|
||||
|
||||
**2.5 Simplify GlobalIntegrationSettings**
|
||||
|
||||
File: global_settings_models.py
|
||||
- Remove: All API key fields (moved to IntegrationProvider)
|
||||
- Remove: All hardcoded CHOICES
|
||||
- Remove: Model name fields (defaults loaded from AIModelConfig.is_default)
|
||||
- Keep: image_style, max_in_article_images, image_quality
|
||||
- Add: Helper methods to get defaults from AIModelConfig
|
||||
|
||||
#### Phase 3: API Endpoints
|
||||
|
||||
**3.1 New endpoint: GET /api/v1/system/ai-models/**
|
||||
|
||||
Returns all active models from database with defaults marked (no hardcoding):
|
||||
```json
|
||||
{
|
||||
"text_models": [
|
||||
{
|
||||
"model_name": "gpt-5.1",
|
||||
"display_name": "GPT-5.1 Premium",
|
||||
"is_default": true,
|
||||
"tokens_per_credit": 1000,
|
||||
"max_output_tokens": 8192
|
||||
},
|
||||
{
|
||||
"model_name": "gpt-4o-mini",
|
||||
"display_name": "GPT-4o Mini",
|
||||
"is_default": false,
|
||||
"tokens_per_credit": 10000,
|
||||
"max_output_tokens": 16000
|
||||
}
|
||||
],
|
||||
"image_models": [
|
||||
{
|
||||
"model_name": "runware:97@1",
|
||||
"display_name": "Basic",
|
||||
"quality_tier": "basic",
|
||||
"is_default": false,
|
||||
"credits_per_image": 1,
|
||||
"valid_sizes": ["1024x1024", "1280x768"]
|
||||
},
|
||||
{
|
||||
"model_name": "dall-e-3",
|
||||
"display_name": "Quality",
|
||||
"quality_tier": "quality",
|
||||
"is_default": true,
|
||||
"credits_per_image": 5,
|
||||
"valid_sizes": ["1024x1024", "1792x1024"]
|
||||
},
|
||||
{
|
||||
"model_name": "google:4@2",
|
||||
"display_name": "Premium",
|
||||
"quality_tier": "premium",
|
||||
"is_default": false,
|
||||
"credits_per_image": 15,
|
||||
"valid_sizes": ["1024x1024", "1376x768"]
|
||||
}
|
||||
],
|
||||
"image_settings": {
|
||||
"style": "photorealistic",
|
||||
"max_in_article_images": 4,
|
||||
"quality": "hd"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Phase 4: Frontend Changes
|
||||
|
||||
**4.1 Settings.tsx**
|
||||
|
||||
- Remove: QUALITY_TO_CONFIG, RUNWARE_MODEL_CHOICES, DALLE_MODEL_CHOICES hardcodes
|
||||
- Remove: MODEL_LANDSCAPE_SIZES hardcodes
|
||||
- Add: Fetch models from `/api/v1/system/ai-models/`
|
||||
- Load valid_sizes from API response per model
|
||||
- Display to user (no provider/model names visible):
|
||||
- **"Basic (1 credit/image)"**
|
||||
- **"Quality (5 credits/image)"**
|
||||
- **"Premium (15 credits/image)"**
|
||||
- Default selection: model where `is_default=true` from API
|
||||
|
||||
#### Phase 5: Cleanup
|
||||
|
||||
**5.1 Files to clean/remove**
|
||||
- Remove unused fields from GlobalIntegrationSettings: anthropic_*, bria_*, all API key fields, hardcoded model fields
|
||||
- Remove deprecated methods from AICore
|
||||
- Update all imports removing constants.py usage
|
||||
- Remove CreditCostConfig dependency for image operations (use AIModelConfig.credits_per_image directly)
|
||||
|
||||
---
|
||||
|
||||
### Credit Calculation Summary
|
||||
|
||||
**Text Models (token-based):**
|
||||
| Model | tokens_per_credit | Example: 5000 tokens |
|
||||
|-------|-------------------|----------------------|
|
||||
| gpt-5.1 | 1,000 | 5 credits |
|
||||
| gpt-4o | 1,000 | 5 credits |
|
||||
| gpt-4o-mini | 10,000 | 1 credit |
|
||||
|
||||
**Image Models (per-image):**
|
||||
| Model | credits_per_image | quality_tier | Display |
|
||||
|-------|-------------------|--------------|---------|
|
||||
| runware:97@1 | 1 | basic | "Basic (1 credit/image)" |
|
||||
| dall-e-3 | 5 | quality | "Quality (5 credits/image)" |
|
||||
| google:4@2 | 15 | premium | "Premium (15 credits/image)" |
|
||||
|
||||
---
|
||||
|
||||
### Migration Order
|
||||
|
||||
1. Create IntegrationProvider model + migration
|
||||
2. Add credits_per_image, tokens_per_credit, quality_tier to AIModelConfig + migration
|
||||
3. Data migration: populate IntegrationProvider, update AIModelConfig with credit values
|
||||
4. Update ModelRegistry (remove constants fallback, add get_default_model)
|
||||
5. Update AICore (use IntegrationProvider for keys)
|
||||
6. Update CreditService (model-based credit calculation)
|
||||
7. Create API endpoint /api/v1/system/ai-models/
|
||||
8. Update frontend (load from API, no hardcodes)
|
||||
9. Cleanup GlobalIntegrationSettings (remove API keys, hardcoded choices)
|
||||
10. Remove constants.py hardcoded rates
|
||||
|
||||
---
|
||||
|
||||
### Files Changed Summary
|
||||
|
||||
| File | Action |
|
||||
|------|--------|
|
||||
| models.py | Add IntegrationProvider, update AIModelConfig with credit fields |
|
||||
| model_registry.py | Remove constants fallback, add get_default_model(), get_provider() |
|
||||
| ai_core.py | Use IntegrationProvider for keys, ModelRegistry for defaults |
|
||||
| constants.py | Remove MODEL_RATES, IMAGE_MODEL_RATES |
|
||||
| credit_service.py | Model-based credit calculation for both text and images |
|
||||
| global_settings_models.py | Remove API keys, hardcoded choices, model fields |
|
||||
| backend/igny8_core/api/views/system.py | Add ai-models endpoint |
|
||||
| Settings.tsx | Load models from API, remove all hardcodes |
|
||||
|
||||
---
|
||||
|
||||
### Key Principles
|
||||
|
||||
1. **No hardcoded model names** - GlobalIntegrationSettings loads defaults from AIModelConfig where is_default=True
|
||||
2. **Single source of truth** - AIModelConfig is THE source for all model info including credit costs
|
||||
3. **Future-proof** - IntegrationProvider handles ALL 3rd party integrations (AI, email, payment, etc.)
|
||||
4. **Dynamic frontend** - All model choices loaded from API, not hardcoded
|
||||
5. **Configurable credits** - Change credits_per_image or tokens_per_credit in admin, no code changes needed
|
||||
@@ -1,939 +0,0 @@
|
||||
# Safe Migration & Testing Plan
|
||||
|
||||
## Overview
|
||||
|
||||
This document ensures the refactor is done safely with zero breakage. All existing functionality must work identically before and after migration.
|
||||
|
||||
---
|
||||
|
||||
## Current Working System Components
|
||||
|
||||
### 1. AI Functions (Core - Used by ALL paths)
|
||||
|
||||
All AI operations go through these functions in `backend/igny8_core/ai/functions/`:
|
||||
|
||||
| Function | File | Operation Type | Credits | Called Via |
|
||||
|----------|------|----------------|---------|------------|
|
||||
| Auto Cluster | `auto_cluster.py` | `clustering` | Token-based | AIEngine |
|
||||
| Generate Ideas | `generate_ideas.py` | `idea_generation` | Token-based | AIEngine |
|
||||
| Generate Content | `generate_content.py` | `content_generation` | Token-based | AIEngine |
|
||||
| Generate Image Prompts | `generate_image_prompts.py` | `image_prompt_extraction` | Token-based | AIEngine |
|
||||
| Generate Images | `generate_images.py` | `image_generation` | Per-image | AICore direct |
|
||||
| Optimize Content | `optimize_content.py` | `content_optimization` | Token-based | OptimizerService |
|
||||
|
||||
**Operation Type Mapping** (in `engine.py`):
|
||||
```python
|
||||
mapping = {
|
||||
'auto_cluster': 'clustering',
|
||||
'generate_ideas': 'idea_generation',
|
||||
'generate_content': 'content_generation',
|
||||
'generate_image_prompts': 'image_prompt_extraction',
|
||||
'generate_images': 'image_generation',
|
||||
'generate_site_structure': 'site_structure_generation',
|
||||
}
|
||||
```
|
||||
|
||||
### 1.1 Non-AIEngine Credit Deduction Points
|
||||
|
||||
| Service | Operation Type | Direct Credit Deduction |
|
||||
|---------|---------------|------------------------|
|
||||
| LinkerService | `internal_linking` | ✅ Uses CreditService directly |
|
||||
| OptimizerService | `content_optimization` | ✅ Uses CreditService directly |
|
||||
|
||||
### 2. Execution Paths (All use same AI functions)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ ALL EXECUTION PATHS │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
PATH 1: Manual Module Page Buttons (via run_ai_task Celery task)
|
||||
├── Planner Page → "Cluster Keywords" button → api/planner.py → run_ai_task.delay('auto_cluster')
|
||||
├── Planner Page → "Generate Ideas" button → api/planner.py → run_ai_task.delay('generate_ideas')
|
||||
├── Writer Page → "Generate Content" button → api/writer.py → run_ai_task.delay('generate_content')
|
||||
├── Writer Page → "Extract Prompts" button → api/writer.py → run_ai_task.delay('generate_image_prompts')
|
||||
├── Writer Page → "Generate Images" button → api/writer.py → process_image_generation_queue.delay()
|
||||
└── All go through: Celery task → AIEngine → AI Function → AICore → API → CreditService
|
||||
|
||||
PATH 2: Automation Page Manual Run
|
||||
├── Automation Page → "Run Now" button
|
||||
├── automation/tasks.py → run_automation_task
|
||||
├── AutomationService.run_automation()
|
||||
│ Stage 1: _run_clustering() → AIEngine.execute('auto_cluster')
|
||||
│ Stage 2: _run_idea_generation() → AIEngine.execute('generate_ideas')
|
||||
│ Stage 3: _run_task_creation() → Database only (no AI)
|
||||
│ Stage 4: _run_content_generation() → AIEngine.execute('generate_content')
|
||||
│ Stage 5: _run_image_prompt_extraction() → AIEngine.execute('generate_image_prompts')
|
||||
│ Stage 6: _run_image_generation() → process_image_generation_queue.delay()
|
||||
│ Stage 7: _run_publishing() → WordPress API (no AI)
|
||||
└── Same credit deduction path (stages 1,2,4,5 via AIEngine, stage 6 via AICore)
|
||||
|
||||
PATH 3: Scheduled Automation Run
|
||||
├── Celery Beat scheduler → check_scheduled_automations task (hourly)
|
||||
├── automation_tasks.py → run_automation_task.delay()
|
||||
├── AutomationService.run_automation()
|
||||
├── Calls same AI functions through AIEngine
|
||||
└── Same credit deduction path
|
||||
|
||||
PATH 4: Direct Service Operations (Bypass AIEngine)
|
||||
├── Linker Module → LinkerService → CreditService.deduct_credits_for_operation('internal_linking')
|
||||
├── Optimizer Module → OptimizerService → CreditService.deduct_credits_for_operation('content_optimization')
|
||||
└── These use direct credit deduction, not via AIEngine
|
||||
```
|
||||
|
||||
### 3. Credit Flow (Current)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ CURRENT CREDIT FLOW │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
|
||||
For TEXT operations (clustering, ideas, content) - VIA AIEngine:
|
||||
1. API endpoint receives request
|
||||
2. Celery task run_ai_task triggered
|
||||
3. AIEngine.execute(function_name, payload)
|
||||
a. Step 2.5: CreditService.check_credits() - PRE-CHECK
|
||||
b. Step 3: AICore.run_ai_request() - makes OpenAI API call
|
||||
c. Response includes token usage (input_tokens, output_tokens)
|
||||
d. Step 5.5: CreditService.deduct_credits_for_operation() - DEDUCTION
|
||||
4. CreditService.calculate_credits_from_tokens() uses CreditCostConfig
|
||||
5. Creates CreditTransaction (balance history)
|
||||
6. Creates CreditUsageLog (detailed tracking)
|
||||
7. Updates Account.credits
|
||||
|
||||
For IMAGE operations - VIA process_image_generation_queue:
|
||||
1. API endpoint receives request
|
||||
2. Celery task process_image_generation_queue triggered
|
||||
3. For each image: AICore.generate_image()
|
||||
a. Makes Runware/OpenAI API call
|
||||
b. Calculates cost via ModelRegistry.calculate_cost()
|
||||
c. Returns image URL
|
||||
4. ⚠️ Credit deduction handled AFTER all images generated
|
||||
5. Uses CreditCostConfig.image_generation entry (min_credits, tokens_per_credit)
|
||||
6. Creates CreditUsageLog
|
||||
|
||||
For DIRECT SERVICE OPERATIONS (Linker, Optimizer):
|
||||
1. Service method called (e.g., LinkerService.generate_links())
|
||||
2. AI call made directly to AICore or external service
|
||||
3. CreditService.deduct_credits_for_operation() called directly
|
||||
4. Uses operation-specific CreditCostConfig entry
|
||||
```
|
||||
|
||||
### 3.1 Celery Task Definitions
|
||||
|
||||
| Task File | Task Name | Entry Point For |
|
||||
|-----------|-----------|-----------------|
|
||||
| `ai/tasks.py` | `run_ai_task` | All manual AI buttons (universal task) |
|
||||
| `ai/tasks.py` | `process_image_generation_queue` | Image generation queue |
|
||||
| `automation/tasks.py` | `check_scheduled_automations` | Hourly scheduler check |
|
||||
| `automation/tasks.py` | `run_automation_task` | Full automation pipeline |
|
||||
| `automation/tasks.py` | `resume_paused_automation` | Resume after pause |
|
||||
|
||||
### 4. Key Files Currently Working
|
||||
|
||||
| File | Purpose | Must Keep Working |
|
||||
|------|---------|-------------------|
|
||||
| `ai/engine.py` | Orchestrates AI functions | ✅ |
|
||||
| `ai/ai_core.py` | API calls, key loading | ✅ |
|
||||
| `ai/model_registry.py` | Model config lookup | ✅ |
|
||||
| `ai/settings.py` | get_model_config() | ✅ |
|
||||
| `ai/tasks.py` | Celery tasks | ✅ |
|
||||
| `ai/functions/*.py` | All AI functions | ✅ |
|
||||
| `billing/services/credit_service.py` | Credit calculation | ✅ |
|
||||
| `billing/models.py` | CreditUsageLog, CreditCostConfig | ✅ |
|
||||
| `automation/services/automation_service.py` | Automation runner | ✅ |
|
||||
| `modules/system/global_settings_models.py` | API keys, defaults | ✅ |
|
||||
|
||||
---
|
||||
|
||||
## Potential Issues to Verify Before Migration
|
||||
|
||||
### ⚠️ Issue 1: Image Credit Deduction May Be Incomplete
|
||||
|
||||
The `process_image_generation_queue` task calculates cost via `ModelRegistry.calculate_cost()` but credit deduction needs verification:
|
||||
- Does it call `CreditService.deduct_credits_for_operation()`?
|
||||
- Or does it just log cost without deducting credits?
|
||||
|
||||
**Verification Query:**
|
||||
```bash
|
||||
docker exec igny8_backend python manage.py shell -c "
|
||||
from igny8_core.business.billing.models import CreditUsageLog
|
||||
logs = CreditUsageLog.objects.filter(operation_type='image_generation').order_by('-created_at')[:5]
|
||||
for log in logs:
|
||||
print(f'{log.operation_type}: {log.credits_used} credits, cost=${log.cost_usd}, model={log.model_used}')
|
||||
if not logs:
|
||||
print('⚠️ NO image_generation credit logs found - may not be tracking credits!')
|
||||
"
|
||||
```
|
||||
|
||||
### ⚠️ Issue 2: GlobalIntegrationSettings.runware_model Points to Non-Existent Model
|
||||
|
||||
Current value: `bria:10@1` - but this model doesn't exist in AIModelConfig.
|
||||
This may cause fallback issues.
|
||||
|
||||
### ⚠️ Issue 3: Multiple Credit Calculation Paths
|
||||
|
||||
Credits are calculated in different ways depending on the path:
|
||||
- Via AIEngine → uses `CreditCostConfig` token-based calculation
|
||||
- Via direct service → uses `CreditCostConfig` with operation-specific rates
|
||||
- Via image generation → may use different calculation
|
||||
|
||||
Need to unify after migration.
|
||||
|
||||
---
|
||||
|
||||
## Pre-Migration Baseline Capture
|
||||
|
||||
### Step 1: Database State Snapshot
|
||||
|
||||
Run this BEFORE any changes to establish baseline:
|
||||
|
||||
```bash
|
||||
docker exec igny8_backend python manage.py shell << 'EOF'
|
||||
import json
|
||||
from datetime import datetime
|
||||
|
||||
print("=" * 80)
|
||||
print(f"BASELINE SNAPSHOT - {datetime.now().isoformat()}")
|
||||
print("=" * 80)
|
||||
|
||||
# 1. AIModelConfig
|
||||
print("\n=== AIModelConfig ===")
|
||||
from igny8_core.business.billing.models import AIModelConfig
|
||||
for m in AIModelConfig.objects.all().order_by('model_type', 'sort_order'):
|
||||
print(json.dumps({
|
||||
'model_name': m.model_name,
|
||||
'model_type': m.model_type,
|
||||
'provider': m.provider,
|
||||
'is_active': m.is_active,
|
||||
'is_default': m.is_default,
|
||||
'cost_per_image': str(m.cost_per_image) if m.cost_per_image else None,
|
||||
'input_cost_per_1m': str(m.input_cost_per_1m) if m.input_cost_per_1m else None,
|
||||
'output_cost_per_1m': str(m.output_cost_per_1m) if m.output_cost_per_1m else None,
|
||||
}))
|
||||
|
||||
# 2. CreditCostConfig
|
||||
print("\n=== CreditCostConfig ===")
|
||||
from igny8_core.business.billing.models import CreditCostConfig
|
||||
for c in CreditCostConfig.objects.filter(is_active=True):
|
||||
print(json.dumps({
|
||||
'operation_type': c.operation_type,
|
||||
'tokens_per_credit': c.tokens_per_credit,
|
||||
'min_credits': c.min_credits,
|
||||
'price_per_credit_usd': str(c.price_per_credit_usd),
|
||||
}))
|
||||
|
||||
# 3. GlobalIntegrationSettings
|
||||
print("\n=== GlobalIntegrationSettings ===")
|
||||
from igny8_core.modules.system.global_settings_models import GlobalIntegrationSettings
|
||||
g = GlobalIntegrationSettings.get_instance()
|
||||
print(json.dumps({
|
||||
'openai_model': g.openai_model,
|
||||
'openai_temperature': g.openai_temperature,
|
||||
'openai_max_tokens': g.openai_max_tokens,
|
||||
'default_text_provider': g.default_text_provider,
|
||||
'dalle_model': g.dalle_model,
|
||||
'runware_model': g.runware_model,
|
||||
'default_image_service': g.default_image_service,
|
||||
'image_style': g.image_style,
|
||||
'max_in_article_images': g.max_in_article_images,
|
||||
'has_openai_key': bool(g.openai_api_key),
|
||||
'has_runware_key': bool(g.runware_api_key),
|
||||
}))
|
||||
|
||||
# 4. Recent Credit Usage (last 20)
|
||||
print("\n=== Recent CreditUsageLog (last 20) ===")
|
||||
from igny8_core.business.billing.models import CreditUsageLog
|
||||
for log in CreditUsageLog.objects.all().order_by('-created_at')[:20]:
|
||||
print(json.dumps({
|
||||
'operation_type': log.operation_type,
|
||||
'credits_used': log.credits_used,
|
||||
'model_used': log.model_used,
|
||||
'tokens_input': log.tokens_input,
|
||||
'tokens_output': log.tokens_output,
|
||||
'cost_usd': str(log.cost_usd) if log.cost_usd else None,
|
||||
}))
|
||||
|
||||
# 5. Account Credits
|
||||
print("\n=== Account Credits ===")
|
||||
from igny8_core.auth.models import Account
|
||||
for acc in Account.objects.all()[:5]:
|
||||
print(json.dumps({
|
||||
'account_id': acc.id,
|
||||
'name': acc.name,
|
||||
'credits': acc.credits,
|
||||
}))
|
||||
|
||||
print("\n" + "=" * 80)
|
||||
print("BASELINE CAPTURE COMPLETE")
|
||||
print("=" * 80)
|
||||
EOF
|
||||
```
|
||||
|
||||
Save this output to: `4th-jan-refactor/baseline-snapshot.json`
|
||||
|
||||
---
|
||||
|
||||
## Integration Tests (Must Pass Before AND After)
|
||||
|
||||
### Test File: `backend/igny8_core/tests/test_ai_system_integration.py`
|
||||
|
||||
```python
|
||||
"""
|
||||
AI System Integration Tests
|
||||
===========================
|
||||
These tests verify the entire AI pipeline works end-to-end.
|
||||
Run BEFORE migration to establish baseline.
|
||||
Run AFTER each phase to verify nothing broke.
|
||||
|
||||
Usage:
|
||||
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration
|
||||
"""
|
||||
from decimal import Decimal
|
||||
from django.test import TestCase, TransactionTestCase
|
||||
from django.db import transaction
|
||||
from unittest.mock import patch, MagicMock
|
||||
import json
|
||||
|
||||
|
||||
class AIModelConfigTests(TestCase):
|
||||
"""Test AIModelConfig loading and access"""
|
||||
|
||||
def test_text_models_exist(self):
|
||||
"""All required text models exist and are configured"""
|
||||
from igny8_core.business.billing.models import AIModelConfig
|
||||
|
||||
required_text_models = ['gpt-5.1', 'gpt-4o-mini']
|
||||
for model_name in required_text_models:
|
||||
model = AIModelConfig.objects.filter(model_name=model_name).first()
|
||||
self.assertIsNotNone(model, f"Text model {model_name} not found")
|
||||
self.assertEqual(model.model_type, 'text')
|
||||
|
||||
def test_image_models_exist(self):
|
||||
"""All required image models exist and are configured"""
|
||||
from igny8_core.business.billing.models import AIModelConfig
|
||||
|
||||
required_image_models = ['runware:97@1', 'dall-e-3', 'google:4@2']
|
||||
for model_name in required_image_models:
|
||||
model = AIModelConfig.objects.filter(model_name=model_name).first()
|
||||
self.assertIsNotNone(model, f"Image model {model_name} not found")
|
||||
self.assertEqual(model.model_type, 'image')
|
||||
self.assertIsNotNone(model.cost_per_image, f"{model_name} missing cost_per_image")
|
||||
|
||||
def test_default_text_model_exists(self):
|
||||
"""Exactly one default text model is set"""
|
||||
from igny8_core.business.billing.models import AIModelConfig
|
||||
|
||||
defaults = AIModelConfig.objects.filter(model_type='text', is_default=True, is_active=True)
|
||||
self.assertEqual(defaults.count(), 1, "Should have exactly 1 default text model")
|
||||
self.assertEqual(defaults.first().model_name, 'gpt-5.1')
|
||||
|
||||
def test_default_image_model_exists(self):
|
||||
"""Exactly one default image model is set"""
|
||||
from igny8_core.business.billing.models import AIModelConfig
|
||||
|
||||
defaults = AIModelConfig.objects.filter(model_type='image', is_default=True, is_active=True)
|
||||
self.assertEqual(defaults.count(), 1, "Should have exactly 1 default image model")
|
||||
|
||||
|
||||
class ModelRegistryTests(TestCase):
|
||||
"""Test ModelRegistry functionality"""
|
||||
|
||||
def test_get_model_from_db(self):
|
||||
"""ModelRegistry.get_model() returns model from database"""
|
||||
from igny8_core.ai.model_registry import ModelRegistry
|
||||
|
||||
model = ModelRegistry.get_model('gpt-5.1')
|
||||
self.assertIsNotNone(model)
|
||||
# Should be AIModelConfig instance or dict with model_name
|
||||
if hasattr(model, 'model_name'):
|
||||
self.assertEqual(model.model_name, 'gpt-5.1')
|
||||
else:
|
||||
self.assertEqual(model.get('model_name'), 'gpt-5.1')
|
||||
|
||||
def test_get_image_model(self):
|
||||
"""ModelRegistry returns image models correctly"""
|
||||
from igny8_core.ai.model_registry import ModelRegistry
|
||||
|
||||
for model_name in ['runware:97@1', 'dall-e-3', 'google:4@2']:
|
||||
model = ModelRegistry.get_model(model_name)
|
||||
self.assertIsNotNone(model, f"Model {model_name} not found")
|
||||
|
||||
def test_calculate_cost_text(self):
|
||||
"""Cost calculation for text models works"""
|
||||
from igny8_core.ai.model_registry import ModelRegistry
|
||||
|
||||
cost = ModelRegistry.calculate_cost('gpt-5.1', input_tokens=1000, output_tokens=500)
|
||||
self.assertIsInstance(cost, (int, float, Decimal))
|
||||
self.assertGreater(cost, 0)
|
||||
|
||||
def test_calculate_cost_image(self):
|
||||
"""Cost calculation for image models works"""
|
||||
from igny8_core.ai.model_registry import ModelRegistry
|
||||
|
||||
cost = ModelRegistry.calculate_cost('dall-e-3', num_images=1)
|
||||
self.assertIsInstance(cost, (int, float, Decimal))
|
||||
self.assertGreater(cost, 0)
|
||||
|
||||
|
||||
class APIKeyLoadingTests(TestCase):
|
||||
"""Test API key loading from GlobalIntegrationSettings"""
|
||||
|
||||
def test_openai_key_loads(self):
|
||||
"""OpenAI API key loads from GlobalIntegrationSettings"""
|
||||
from igny8_core.ai.ai_core import AICore
|
||||
|
||||
ai_core = AICore()
|
||||
key = ai_core.get_api_key('openai')
|
||||
self.assertIsNotNone(key, "OpenAI API key not configured")
|
||||
self.assertTrue(len(key) > 10, "OpenAI API key too short")
|
||||
|
||||
def test_runware_key_loads(self):
|
||||
"""Runware API key loads from GlobalIntegrationSettings"""
|
||||
from igny8_core.ai.ai_core import AICore
|
||||
|
||||
ai_core = AICore()
|
||||
key = ai_core.get_api_key('runware')
|
||||
self.assertIsNotNone(key, "Runware API key not configured")
|
||||
|
||||
|
||||
class CreditServiceTests(TestCase):
|
||||
"""Test credit calculation and deduction"""
|
||||
|
||||
def test_calculate_credits_text_operation(self):
|
||||
"""Credit calculation for text operations works"""
|
||||
from igny8_core.business.billing.services.credit_service import CreditService
|
||||
|
||||
# Test content generation
|
||||
credits = CreditService.calculate_credits_from_tokens(
|
||||
'content_generation',
|
||||
tokens_input=1000,
|
||||
tokens_output=2000
|
||||
)
|
||||
self.assertIsInstance(credits, int)
|
||||
self.assertGreater(credits, 0)
|
||||
|
||||
def test_calculate_credits_image_operation(self):
|
||||
"""Credit calculation for image operations works"""
|
||||
from igny8_core.business.billing.services.credit_service import CreditService
|
||||
|
||||
# Image generation uses min_credits from CreditCostConfig
|
||||
credits = CreditService.calculate_credits_from_tokens(
|
||||
'image_generation',
|
||||
tokens_input=0,
|
||||
tokens_output=0
|
||||
)
|
||||
self.assertIsInstance(credits, int)
|
||||
self.assertGreaterEqual(credits, 1)
|
||||
|
||||
def test_credit_cost_config_exists(self):
|
||||
"""All required CreditCostConfig entries exist"""
|
||||
from igny8_core.business.billing.models import CreditCostConfig
|
||||
|
||||
required_ops = ['clustering', 'idea_generation', 'content_generation',
|
||||
'image_generation', 'image_prompt_extraction']
|
||||
for op in required_ops:
|
||||
config = CreditCostConfig.objects.filter(operation_type=op, is_active=True).first()
|
||||
self.assertIsNotNone(config, f"CreditCostConfig for {op} not found")
|
||||
|
||||
|
||||
class AISettingsTests(TestCase):
|
||||
"""Test get_model_config() function"""
|
||||
|
||||
def test_get_model_config_returns_config(self):
|
||||
"""get_model_config() returns valid configuration"""
|
||||
from igny8_core.ai.settings import get_model_config
|
||||
from igny8_core.auth.models import Account
|
||||
|
||||
account = Account.objects.first()
|
||||
if account:
|
||||
config = get_model_config('content_generation', account)
|
||||
|
||||
self.assertIn('model', config)
|
||||
self.assertIn('max_tokens', config)
|
||||
self.assertIn('temperature', config)
|
||||
self.assertIsNotNone(config['model'])
|
||||
|
||||
|
||||
class AIFunctionValidationTests(TestCase):
|
||||
"""Test AI function classes load correctly"""
|
||||
|
||||
def test_auto_cluster_function_loads(self):
|
||||
"""AutoClusterFunction loads and validates"""
|
||||
from igny8_core.ai.functions.auto_cluster import AutoClusterFunction
|
||||
|
||||
fn = AutoClusterFunction()
|
||||
self.assertEqual(fn.get_name(), 'auto_cluster')
|
||||
self.assertIsNotNone(fn.get_metadata())
|
||||
|
||||
def test_generate_ideas_function_loads(self):
|
||||
"""GenerateIdeasFunction loads and validates"""
|
||||
from igny8_core.ai.functions.generate_ideas import GenerateIdeasFunction
|
||||
|
||||
fn = GenerateIdeasFunction()
|
||||
self.assertEqual(fn.get_name(), 'generate_ideas')
|
||||
|
||||
def test_generate_content_function_loads(self):
|
||||
"""GenerateContentFunction loads and validates"""
|
||||
from igny8_core.ai.functions.generate_content import GenerateContentFunction
|
||||
|
||||
fn = GenerateContentFunction()
|
||||
self.assertEqual(fn.get_name(), 'generate_content')
|
||||
|
||||
def test_generate_images_function_loads(self):
|
||||
"""GenerateImagesFunction loads and validates"""
|
||||
from igny8_core.ai.functions.generate_images import GenerateImagesFunction
|
||||
|
||||
fn = GenerateImagesFunction()
|
||||
self.assertEqual(fn.get_name(), 'generate_images')
|
||||
|
||||
|
||||
class GlobalIntegrationSettingsTests(TestCase):
|
||||
"""Test GlobalIntegrationSettings singleton"""
|
||||
|
||||
def test_singleton_loads(self):
|
||||
"""GlobalIntegrationSettings singleton loads"""
|
||||
from igny8_core.modules.system.global_settings_models import GlobalIntegrationSettings
|
||||
|
||||
settings = GlobalIntegrationSettings.get_instance()
|
||||
self.assertIsNotNone(settings)
|
||||
self.assertEqual(settings.pk, 1)
|
||||
|
||||
def test_image_settings_exist(self):
|
||||
"""Image settings are configured"""
|
||||
from igny8_core.modules.system.global_settings_models import GlobalIntegrationSettings
|
||||
|
||||
settings = GlobalIntegrationSettings.get_instance()
|
||||
self.assertIsNotNone(settings.image_style)
|
||||
self.assertIsNotNone(settings.max_in_article_images)
|
||||
self.assertGreater(settings.max_in_article_images, 0)
|
||||
|
||||
|
||||
class AutomationServiceTests(TestCase):
|
||||
"""Test AutomationService uses same AI functions"""
|
||||
|
||||
def test_automation_service_imports(self):
|
||||
"""AutomationService can import AI functions"""
|
||||
from igny8_core.business.automation.services.automation_service import AutomationService
|
||||
|
||||
# Just verify the import works and class exists
|
||||
self.assertIsNotNone(AutomationService)
|
||||
```
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Run all integration tests
|
||||
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration -v 2
|
||||
|
||||
# Run specific test class
|
||||
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration.AIModelConfigTests -v 2
|
||||
|
||||
# Run with coverage
|
||||
docker exec igny8_backend coverage run manage.py test igny8_core.tests.test_ai_system_integration
|
||||
docker exec igny8_backend coverage report
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phased Migration with Verification
|
||||
|
||||
### Phase 1: Add New Models (NO BREAKING CHANGES)
|
||||
|
||||
**Changes:**
|
||||
- Add `IntegrationProvider` model
|
||||
- Add new fields to `AIModelConfig`: `credits_per_image`, `tokens_per_credit`, `quality_tier`
|
||||
- Create migrations
|
||||
- Populate new data
|
||||
|
||||
**Verification:**
|
||||
```bash
|
||||
# 1. Run migrations
|
||||
docker exec igny8_backend python manage.py makemigrations
|
||||
docker exec igny8_backend python manage.py migrate
|
||||
|
||||
# 2. Run ALL tests - must pass
|
||||
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration -v 2
|
||||
|
||||
# 3. Manual verification - existing features still work
|
||||
# - Go to Writer page, generate content (check credits deducted)
|
||||
# - Go to Writer page, generate images (check credits deducted)
|
||||
# - Run automation manually (check it completes)
|
||||
```
|
||||
|
||||
**Rollback:** Delete new tables/fields (no existing data affected)
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Add Parallel Code Paths (OLD KEEPS WORKING)
|
||||
|
||||
**Changes:**
|
||||
- Add new methods to `ModelRegistry` (keep old ones)
|
||||
- Add new methods to `CreditService` (keep old ones)
|
||||
- Add comparison logging
|
||||
|
||||
**Pattern:**
|
||||
```python
|
||||
# model_registry.py
|
||||
@classmethod
|
||||
def get_default_model_NEW(cls, model_type: str):
|
||||
"""NEW: Get default model from AIModelConfig.is_default"""
|
||||
from igny8_core.business.billing.models import AIModelConfig
|
||||
return AIModelConfig.objects.filter(
|
||||
model_type=model_type,
|
||||
is_default=True,
|
||||
is_active=True
|
||||
).first()
|
||||
|
||||
@classmethod
|
||||
def get_model(cls, model_id: str):
|
||||
"""EXISTING: Keep working exactly as before"""
|
||||
# ... existing code unchanged ...
|
||||
```
|
||||
|
||||
```python
|
||||
# credit_service.py
|
||||
@staticmethod
|
||||
def calculate_credits_for_image_NEW(model_name: str, num_images: int) -> int:
|
||||
"""NEW: Calculate from AIModelConfig.credits_per_image"""
|
||||
from igny8_core.business.billing.models import AIModelConfig
|
||||
try:
|
||||
model = AIModelConfig.objects.get(model_name=model_name, is_active=True)
|
||||
if model.credits_per_image:
|
||||
return model.credits_per_image * num_images
|
||||
except AIModelConfig.DoesNotExist:
|
||||
pass
|
||||
return None # Signal to use old method
|
||||
|
||||
@staticmethod
|
||||
def calculate_credits_from_tokens(operation_type, tokens_input, tokens_output):
|
||||
"""EXISTING: Keep working, add comparison logging"""
|
||||
# Calculate old way
|
||||
old_result = cls._calculate_old(operation_type, tokens_input, tokens_output)
|
||||
|
||||
# Calculate new way (for comparison only)
|
||||
new_result = cls._calculate_new(operation_type, tokens_input, tokens_output)
|
||||
|
||||
# Log if different
|
||||
if old_result != new_result:
|
||||
logger.warning(f"[MIGRATION] Credit calc mismatch: {operation_type} old={old_result} new={new_result}")
|
||||
|
||||
# Return OLD (safe) for now
|
||||
return old_result
|
||||
```
|
||||
|
||||
**Verification:**
|
||||
```bash
|
||||
# 1. Run tests
|
||||
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration -v 2
|
||||
|
||||
# 2. Check logs for any mismatch warnings
|
||||
docker logs igny8_backend 2>&1 | grep "MIGRATION"
|
||||
|
||||
# 3. Manual test all paths
|
||||
```
|
||||
|
||||
**Rollback:** Remove new methods only
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Add IntegrationProvider for API Keys (OLD FALLBACK)
|
||||
|
||||
**Changes:**
|
||||
- `AICore._load_account_settings()` tries `IntegrationProvider` first, falls back to `GlobalIntegrationSettings`
|
||||
|
||||
**Pattern:**
|
||||
```python
|
||||
# ai_core.py
|
||||
def _load_account_settings(self):
|
||||
"""Load API keys - try new IntegrationProvider, fallback to old"""
|
||||
|
||||
# Try NEW way first
|
||||
try:
|
||||
from igny8_core.business.billing.models import IntegrationProvider
|
||||
openai_provider = IntegrationProvider.objects.filter(
|
||||
provider_id='openai', is_active=True
|
||||
).first()
|
||||
if openai_provider and openai_provider.api_key:
|
||||
self._openai_api_key = openai_provider.api_key
|
||||
logger.info("[MIGRATION] Loaded OpenAI key from IntegrationProvider")
|
||||
else:
|
||||
raise Exception("Fallback to old method")
|
||||
except Exception:
|
||||
# FALLBACK to old GlobalIntegrationSettings
|
||||
from igny8_core.modules.system.global_settings_models import GlobalIntegrationSettings
|
||||
global_settings = GlobalIntegrationSettings.get_instance()
|
||||
self._openai_api_key = global_settings.openai_api_key
|
||||
logger.info("[MIGRATION] Loaded OpenAI key from GlobalIntegrationSettings (fallback)")
|
||||
|
||||
# Same pattern for runware...
|
||||
```
|
||||
|
||||
**Verification:**
|
||||
```bash
|
||||
# 1. Run tests
|
||||
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration -v 2
|
||||
|
||||
# 2. Verify API calls work
|
||||
# - Generate content (uses OpenAI)
|
||||
# - Generate images with Runware model
|
||||
# - Generate images with DALL-E model
|
||||
|
||||
# 3. Check logs
|
||||
docker logs igny8_backend 2>&1 | grep "MIGRATION"
|
||||
```
|
||||
|
||||
**Rollback:** Revert `_load_account_settings()` to old version
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Switch to New Credit Calculation
|
||||
|
||||
**Changes:**
|
||||
- Use `AIModelConfig.credits_per_image` for image operations
|
||||
- Use `AIModelConfig.tokens_per_credit` for text operations
|
||||
- Keep `CreditCostConfig` as fallback
|
||||
|
||||
**Verification:**
|
||||
```bash
|
||||
# 1. Before switching, compare calculations for 1 week
|
||||
# Log both old and new results, verify they match
|
||||
|
||||
# 2. Run tests
|
||||
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration -v 2
|
||||
|
||||
# 3. Verify credit logs show correct values
|
||||
docker exec igny8_backend python manage.py shell -c "
|
||||
from igny8_core.business.billing.models import CreditUsageLog
|
||||
for log in CreditUsageLog.objects.order_by('-created_at')[:10]:
|
||||
print(f'{log.operation_type}: {log.credits_used} credits, model={log.model_used}')
|
||||
"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: Create API Endpoint
|
||||
|
||||
**Changes:**
|
||||
- Add `/api/v1/system/ai-models/` endpoint
|
||||
- Returns models from database
|
||||
|
||||
**Verification:**
|
||||
```bash
|
||||
# 1. Test endpoint
|
||||
curl -H "Authorization: Bearer $TOKEN" http://localhost:8000/api/v1/system/ai-models/
|
||||
|
||||
# 2. Verify response structure
|
||||
# Should have text_models, image_models, image_settings
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 6: Update Frontend
|
||||
|
||||
**Changes:**
|
||||
- Remove hardcoded model choices from `Settings.tsx`
|
||||
- Load from API
|
||||
|
||||
**Verification:**
|
||||
```bash
|
||||
# 1. Frontend tests
|
||||
cd frontend && npm run test
|
||||
|
||||
# 2. Manual verification
|
||||
# - Open Sites > Settings > Image Settings
|
||||
# - Verify dropdown shows models from API
|
||||
# - Change model, save, verify it persists
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 7: Cleanup (After 1 Week Stable)
|
||||
|
||||
**Remove from GlobalIntegrationSettings:**
|
||||
```python
|
||||
# API key fields (moved to IntegrationProvider)
|
||||
- openai_api_key
|
||||
- anthropic_api_key # unused
|
||||
- bria_api_key # unused
|
||||
- runware_api_key
|
||||
|
||||
# Model selection fields (now from AIModelConfig.is_default)
|
||||
- openai_model
|
||||
- dalle_model
|
||||
- runware_model
|
||||
- default_text_provider
|
||||
- default_image_service
|
||||
|
||||
# Hardcoded CHOICES
|
||||
- OPENAI_MODEL_CHOICES
|
||||
- DALLE_MODEL_CHOICES
|
||||
- RUNWARE_MODEL_CHOICES
|
||||
- TEXT_PROVIDER_CHOICES
|
||||
- IMAGE_SERVICE_CHOICES
|
||||
```
|
||||
|
||||
**Remove from constants.py:**
|
||||
```python
|
||||
- MODEL_RATES = {...}
|
||||
- IMAGE_MODEL_RATES = {...}
|
||||
```
|
||||
|
||||
**Remove from model_registry.py:**
|
||||
```python
|
||||
- Fallback to constants.py (all lookups must use DB)
|
||||
- Hardcoded default model names
|
||||
```
|
||||
|
||||
**Remove from ai_core.py:**
|
||||
```python
|
||||
- IMAGE_MODEL_RATES import and usage
|
||||
- Direct GlobalIntegrationSettings key access (use IntegrationProvider)
|
||||
```
|
||||
|
||||
**Remove CreditCostConfig entries (optional - use AIModelConfig):**
|
||||
```python
|
||||
- image_generation entry (use AIModelConfig.credits_per_image instead)
|
||||
```
|
||||
|
||||
**Remove from frontend Settings.tsx:**
|
||||
```python
|
||||
- QUALITY_TO_CONFIG hardcoded mapping
|
||||
- RUNWARE_MODEL_CHOICES hardcoded array
|
||||
- DALLE_MODEL_CHOICES hardcoded array
|
||||
- MODEL_LANDSCAPE_SIZES hardcoded mapping
|
||||
- Any hardcoded model names
|
||||
```
|
||||
|
||||
**Verification:**
|
||||
```bash
|
||||
# 1. Run full test suite
|
||||
docker exec igny8_backend python manage.py test
|
||||
|
||||
# 2. Run integration tests
|
||||
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration -v 2
|
||||
|
||||
# 3. Full manual test of all paths
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Complete Test Checklist
|
||||
|
||||
### Manual Test Checklist (Run After Each Phase)
|
||||
|
||||
#### Path 1: Manual Module Page Actions
|
||||
|
||||
| Test | Page | Button/Action | Expected Result | Verify Credits | Pass? |
|
||||
|------|------|---------------|-----------------|----------------|-------|
|
||||
| Cluster Keywords | Planner | "Cluster" button | Creates clusters | Check CreditUsageLog for `clustering` | ☐ |
|
||||
| Generate Ideas | Planner | "Generate Ideas" button | Creates ideas | Check CreditUsageLog for `idea_generation` | ☐ |
|
||||
| Generate Content | Writer | "Generate" button | Creates content | Check CreditUsageLog for `content_generation` | ☐ |
|
||||
| Generate Product Content | Writer | "Generate Product" button | Creates product content | Check CreditUsageLog for `content_generation` | ☐ |
|
||||
| Generate Service Content | Writer | "Generate Service" button | Creates service content | Check CreditUsageLog for `content_generation` | ☐ |
|
||||
| Extract Image Prompts | Writer | "Extract Prompts" button | Creates prompts | Check CreditUsageLog for `image_prompt_extraction` | ☐ |
|
||||
| Generate Images (Basic) | Writer | Image gen w/ Runware basic | Creates images | Check CreditUsageLog: 1 credit/image | ☐ |
|
||||
| Generate Images (Quality) | Writer | Image gen w/ DALL-E | Creates images | Check CreditUsageLog: 5 credits/image | ☐ |
|
||||
| Generate Images (Premium) | Writer | Image gen w/ Google | Creates images | Check CreditUsageLog: 15 credits/image | ☐ |
|
||||
| Internal Linking | Linker | "Generate Links" button | Creates links | Check CreditUsageLog for `internal_linking` | ☐ |
|
||||
| Optimize Content | Optimizer | "Optimize" button | Optimizes content | Check CreditUsageLog for `content_optimization` | ☐ |
|
||||
|
||||
#### Path 2: Automation Manual Run
|
||||
|
||||
| Test | Page | Action | Expected Result | Verify Credits | Pass? |
|
||||
|------|------|--------|-----------------|----------------|-------|
|
||||
| Run Full Automation | Automation | "Run Now" button | All stages complete | Credits for each stage | ☐ |
|
||||
| Run Stage 1 Only | Automation | Run clustering stage | Clusters created | Check `clustering` credits | ☐ |
|
||||
| Run Stage 2 Only | Automation | Run idea stage | Ideas created | Check `idea_generation` credits | ☐ |
|
||||
| Run Stage 4 Only | Automation | Run content stage | Content created | Check `content_generation` credits | ☐ |
|
||||
| Run Stage 5 Only | Automation | Run prompts stage | Prompts created | Check `image_prompt_extraction` credits | ☐ |
|
||||
| Run Stage 6 Only | Automation | Run image stage | Images created | Check `image_generation` credits | ☐ |
|
||||
| Pause/Resume | Automation | Pause then resume | Continues correctly | No double-charging | ☐ |
|
||||
|
||||
#### Path 3: Scheduled Automation
|
||||
|
||||
| Test | Setup | Expected | Verify | Pass? |
|
||||
|------|-------|----------|--------|-------|
|
||||
| Schedule Triggers | Set schedule for automation | Runs on schedule | Check logs at scheduled time | ☐ |
|
||||
| Credits Deducted | After scheduled run | Credits reduced | Check account balance | ☐ |
|
||||
| Multiple Automations | Multiple scheduled | All run | Each deducts credits | ☐ |
|
||||
|
||||
#### Credit Verification Queries
|
||||
|
||||
```bash
|
||||
# Check recent credit usage logs
|
||||
docker exec igny8_backend python manage.py shell -c "
|
||||
from igny8_core.business.billing.models import CreditUsageLog
|
||||
for log in CreditUsageLog.objects.order_by('-created_at')[:20]:
|
||||
print(f'{log.created_at}: {log.operation_type} | {log.credits_used} credits | model={log.model_used} | tokens={log.tokens_input}+{log.tokens_output}')
|
||||
"
|
||||
|
||||
# Check account balance
|
||||
docker exec igny8_backend python manage.py shell -c "
|
||||
from igny8_core.auth.models import Account
|
||||
for acc in Account.objects.all()[:5]:
|
||||
print(f'{acc.name}: {acc.credits} credits')
|
||||
"
|
||||
|
||||
# Check credit transactions
|
||||
docker exec igny8_backend python manage.py shell -c "
|
||||
from igny8_core.business.billing.models import CreditTransaction
|
||||
for txn in CreditTransaction.objects.order_by('-created_at')[:10]:
|
||||
print(f'{txn.created_at}: {txn.transaction_type} | {txn.credits_amount} | balance={txn.balance_after}')
|
||||
"
|
||||
```
|
||||
|
||||
#### Frontend Verification
|
||||
|
||||
| Test | Page | Action | Expected | Pass? |
|
||||
|------|------|--------|----------|-------|
|
||||
| Settings Load | Sites > Settings | Open page | Image settings dropdown populated | ☐ |
|
||||
| Model Selection | Sites > Settings | Change image model | Shows Basic/Quality/Premium | ☐ |
|
||||
| Model Persists | Sites > Settings | Save and reload | Selected model persists | ☐ |
|
||||
| Credits Display | Any page | Header/sidebar | Shows correct credit balance | ☐ |
|
||||
| Usage Analytics | Analytics | View usage | Shows correct breakdown | ☐ |
|
||||
|
||||
---
|
||||
|
||||
## Rollback Procedures
|
||||
|
||||
### If Phase 1 Fails
|
||||
```bash
|
||||
# Remove new migrations
|
||||
docker exec igny8_backend python manage.py migrate billing <previous_migration>
|
||||
docker exec igny8_backend python manage.py migrate system <previous_migration>
|
||||
```
|
||||
|
||||
### If Phase 2-6 Fails
|
||||
```bash
|
||||
# Revert code changes via git
|
||||
git checkout -- backend/igny8_core/ai/
|
||||
git checkout -- backend/igny8_core/business/billing/
|
||||
|
||||
# Restart containers
|
||||
docker-compose restart backend
|
||||
```
|
||||
|
||||
### If Phase 7 Fails (Cleanup)
|
||||
```bash
|
||||
# This is why we wait 1 week before cleanup
|
||||
# Restore from backup or revert git
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
Migration is COMPLETE when:
|
||||
|
||||
1. ✅ All integration tests pass
|
||||
2. ✅ All manual tests pass
|
||||
3. ✅ No errors in logs for 1 week
|
||||
4. ✅ Credit calculations match expected values
|
||||
5. ✅ All execution paths work (manual, automation, scheduled)
|
||||
6. ✅ Frontend loads models from API
|
||||
7. ✅ No legacy code remains:
|
||||
- No `MODEL_RATES` / `IMAGE_MODEL_RATES` in constants.py
|
||||
- No API keys in `GlobalIntegrationSettings`
|
||||
- No hardcoded models in frontend
|
||||
- No fallback to constants in `ModelRegistry`
|
||||
@@ -1,91 +0,0 @@
|
||||
My Recommendation: Simplify to 3 Core Concepts
|
||||
Based on analyzing your plans and the current system, here's the simplest path forward:
|
||||
Current Pain Points (Summarized)
|
||||
Area Problem
|
||||
API Keys Scattered in GlobalIntegrationSettings with unused fields (anthropic, bria)
|
||||
Models Duplicated in DB (AIModelConfig), constants.py, GlobalIntegrationSettings CHOICES, frontend
|
||||
Credits Split between CreditCostConfig (token-based) and image models (per-image) - confusing
|
||||
Frontend Hardcoded model lists, not dynamic
|
||||
Proposed Simplified Architecture
|
||||
Instead of 7-phase migration, do this:
|
||||
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ SIMPLIFIED TARGET STATE │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
|
||||
1. AIModelConfig (ENHANCED) - SINGLE SOURCE OF TRUTH
|
||||
├── All models (text + image)
|
||||
├── is_default flag (one per type)
|
||||
├── tokens_per_credit (text models)
|
||||
├── credits_per_image (image models)
|
||||
├── quality_tier (basic/quality/premium for user display)
|
||||
└── provider field links to API key source
|
||||
|
||||
2. IntegrationProvider (NEW) - API KEYS ONLY
|
||||
├── openai → api_key
|
||||
├── runware → api_key
|
||||
└── (future: resend, stripe, etc.)
|
||||
|
||||
3. GlobalIntegrationSettings (SIMPLIFIED) - NON-MODEL SETTINGS ONLY
|
||||
├── image_style: "photorealistic"
|
||||
├── max_in_article_images: 4
|
||||
└── temperature: 0.7
|
||||
(NO model names, NO API keys, NO hardcoded choices)
|
||||
User-Facing Simplification
|
||||
For Images (what user sees):
|
||||
|
||||
Quality Selection:
|
||||
○ Basic (1 credit/image) ← maps to runware:97@1
|
||||
○ Quality (5 credits/image) ← maps to dall-e-3
|
||||
○ Premium (15 credits/image) ← maps to google:4@2
|
||||
|
||||
Image Count: [1-8 dropdown]
|
||||
User picks quality tier + count. Backend handles:
|
||||
Model selection based on quality_tier
|
||||
Credit calculation: credits_per_image × count
|
||||
API routing to correct provider
|
||||
For Text (hidden from user):
|
||||
Admin controls default model in AIModelConfig.is_default
|
||||
User has NO options - uses system default
|
||||
Credits calculated: ceil(tokens / tokens_per_credit)
|
||||
Simplest Migration Path
|
||||
Phase 1: Add fields to AIModelConfig (no breaking changes)
|
||||
|
||||
# Add to existing AIModelConfig
|
||||
credits_per_image = IntegerField(null=True) # For image models
|
||||
tokens_per_credit = IntegerField(null=True) # For text models
|
||||
quality_tier = CharField(choices=['basic','quality','premium'])
|
||||
Phase 2: Create IntegrationProvider (move API keys)
|
||||
|
||||
# New simple model
|
||||
class IntegrationProvider(Model):
|
||||
provider_id = CharField(primary_key=True) # openai, runware
|
||||
api_key = CharField()
|
||||
is_active = BooleanField()
|
||||
Phase 3: Create single API endpoint
|
||||
|
||||
GET /api/v1/system/ai-config/
|
||||
Returns:
|
||||
{
|
||||
"image_qualities": [
|
||||
{"tier": "basic", "credits_per_image": 1, "label": "Basic"},
|
||||
{"tier": "quality", "credits_per_image": 5, "label": "Quality"},
|
||||
{"tier": "premium", "credits_per_image": 15, "label": "Premium"}
|
||||
],
|
||||
"text_model": "gpt-5.1", // Admin-controlled, user can't change
|
||||
"image_style": "photorealistic",
|
||||
"max_images": 4
|
||||
}
|
||||
Phase 4: Update frontend - load from API, remove hardcodes Phase 5: Cleanup - remove legacy fields from GlobalIntegrationSettings
|
||||
Key Simplifications
|
||||
Before After
|
||||
User picks model name User picks quality tier
|
||||
Multiple credit calculation paths One path: AIModelConfig
|
||||
API keys in GlobalIntegrationSettings API keys in IntegrationProvider
|
||||
Hardcoded constants.py rates All from database
|
||||
6+ admin pages to configure 2 pages: Models + Providers
|
||||
Do you want me to:
|
||||
Start implementing this simplified approach? - I can create the migration files and update the models
|
||||
Create a more detailed breakdown of each phase with specific file changes?
|
||||
Focus on a specific area first (e.g., just fix image credits, or just move API keys)?
|
||||
The existing plans in your docs are comprehensive but may be over-engineered. The core problem is simpler: consolidate to AIModelConfig as single source of truth, give users quality tiers instead of model names, and move API keys to a dedicated model.
|
||||
Reference in New Issue
Block a user