Django admin cleanup
This commit is contained in:
IGNY8 VPS (Salman)
214
docs/plans/4th-jan-refactor/REFACTOR-OVERVIEW.md
Normal file
214
docs/plans/4th-jan-refactor/REFACTOR-OVERVIEW.md
Normal file
@@ -0,0 +1,214 @@
|
||||
# AI Configuration Refactor - Master Overview
|
||||
|
||||
**Created:** January 4, 2026
|
||||
**Goal:** Clean, unified AI configuration with single source of truth
|
||||
|
||||
---
|
||||
|
||||
## Documentation Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| [REFACTOR-OVERVIEW.md](REFACTOR-OVERVIEW.md) | This file - master overview |
|
||||
| [implementation-plan-for-ai-models-and-cost.md](implementation-plan-for-ai-models-and-cost.md) | Detailed implementation plan with schema changes |
|
||||
| [safe-migration-and-testing-plan.md](safe-migration-and-testing-plan.md) | Testing strategy and verification steps |
|
||||
|
||||
---
|
||||
|
||||
## What We're Fixing
|
||||
|
||||
### Current Problems
|
||||
|
||||
1. **Duplicate configuration sources**
|
||||
- `AIModelConfig` (database) has model definitions
|
||||
- `GlobalIntegrationSettings` (database) has hardcoded CHOICES duplicating the same info
|
||||
- `constants.py` has `MODEL_RATES` and `IMAGE_MODEL_RATES` hardcoded
|
||||
- Frontend `Settings.tsx` has hardcoded model choices
|
||||
|
||||
2. **Broken/inconsistent data**
|
||||
- `GlobalIntegrationSettings.runware_model = "bria:10@1"` but this model doesn't exist
|
||||
- `GlobalIntegrationSettings.openai_model = "gpt-4o-mini"` but default should be `gpt-5.1`
|
||||
|
||||
3. **API keys mixed with model config**
|
||||
- `GlobalIntegrationSettings` stores both API keys and model preferences
|
||||
- No separation of concerns
|
||||
- Can't easily extend to other integrations (email, payment, etc.)
|
||||
|
||||
4. **Credit calculation scattered**
|
||||
- `CreditCostConfig` for token-based operations
|
||||
- `ModelRegistry.calculate_cost()` for image costs
|
||||
- No clear per-model credit rate
|
||||
|
||||
---
|
||||
|
||||
## Target State
|
||||
|
||||
### Single Source of Truth
|
||||
|
||||
```
|
||||
IntegrationProvider (NEW)
|
||||
├── openai (api_key, active)
|
||||
├── runware (api_key, active)
|
||||
├── resend (api_key, active) ← future
|
||||
├── stripe (api_key, webhook_secret, active) ← future
|
||||
└── ...
|
||||
|
||||
AIModelConfig (ENHANCED)
|
||||
├── Text Models
|
||||
│ ├── gpt-5.1 (is_default=true, tokens_per_credit=1000)
|
||||
│ ├── gpt-4o (tokens_per_credit=1000)
|
||||
│ └── gpt-4o-mini (tokens_per_credit=10000)
|
||||
└── Image Models
|
||||
├── runware:97@1 (credits_per_image=1, quality_tier=basic)
|
||||
├── dall-e-3 (is_default=true, credits_per_image=5, quality_tier=quality)
|
||||
└── google:4@2 (credits_per_image=15, quality_tier=premium)
|
||||
```
|
||||
|
||||
### No More Hardcoding
|
||||
|
||||
- **Backend**: Remove `MODEL_RATES`, `IMAGE_MODEL_RATES` from constants.py
|
||||
- **Backend**: Remove hardcoded CHOICES from `GlobalIntegrationSettings`
|
||||
- **Backend**: `ModelRegistry` always uses database, no fallbacks
|
||||
- **Frontend**: Load model choices from API endpoint
|
||||
|
||||
### Clear Credit Calculation
|
||||
|
||||
| Model Type | How Credits Calculated |
|
||||
|------------|----------------------|
|
||||
| Text | `ceil(total_tokens / tokens_per_credit)` from AIModelConfig |
|
||||
| Image | `credits_per_image * num_images` from AIModelConfig |
|
||||
|
||||
---
|
||||
|
||||
## Migration Phases Summary
|
||||
|
||||
| Phase | What Changes | Risk | Rollback |
|
||||
|-------|--------------|------|----------|
|
||||
| 1 | Add new models/fields | None - additive only | Drop new tables |
|
||||
| 2 | Add parallel code paths | Low - old code untouched | Remove new methods |
|
||||
| 3 | IntegrationProvider for keys | Medium - API key loading | Revert key loading |
|
||||
| 4 | Switch to new credit calc | Medium - financial impact | Revert credit service |
|
||||
| 5 | Create API endpoint | None - new endpoint | Remove endpoint |
|
||||
| 6 | Update frontend | Low - UI only | Revert frontend |
|
||||
| 7 | Cleanup legacy code | High if bugs found | Restore from backup |
|
||||
|
||||
**Rule:** Run full test suite after EACH phase before proceeding.
|
||||
|
||||
---
|
||||
|
||||
## Files Being Modified
|
||||
|
||||
### Backend Files
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `billing/models.py` | Add IntegrationProvider, add fields to AIModelConfig |
|
||||
| `ai/model_registry.py` | Add `get_default_model()`, remove constants fallback |
|
||||
| `ai/ai_core.py` | Use IntegrationProvider for API keys |
|
||||
| `ai/constants.py` | Remove MODEL_RATES, IMAGE_MODEL_RATES |
|
||||
| `billing/services/credit_service.py` | Model-based credit calculation |
|
||||
| `modules/system/global_settings_models.py` | Remove API keys, hardcoded choices |
|
||||
| `api/views/system.py` | Add `/api/v1/system/ai-models/` endpoint |
|
||||
|
||||
### Frontend Files
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `src/pages/Sites/Settings.tsx` | Load models from API, remove hardcodes |
|
||||
|
||||
### New Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `tests/test_ai_system_integration.py` | Integration tests for migration |
|
||||
|
||||
---
|
||||
|
||||
## Credit Values (Final)
|
||||
|
||||
### Text Models (per 1 credit)
|
||||
|
||||
| Model | Tokens per Credit | Example: 5000 tokens |
|
||||
|-------|-------------------|---------------------|
|
||||
| gpt-5.1 | 1,000 | 5 credits |
|
||||
| gpt-4o | 1,000 | 5 credits |
|
||||
| gpt-4o-mini | 10,000 | 1 credit |
|
||||
|
||||
### Image Models (per image)
|
||||
|
||||
| Model | Credits per Image | UI Display |
|
||||
|-------|-------------------|------------|
|
||||
| runware:97@1 | 1 | "Basic (1 credit/image)" |
|
||||
| dall-e-3 | 5 | "Quality (5 credits/image)" |
|
||||
| google:4@2 | 15 | "Premium (15 credits/image)" |
|
||||
|
||||
---
|
||||
|
||||
## Execution Paths That Must Keep Working
|
||||
|
||||
All these paths use the same AI functions and must work identically after migration:
|
||||
|
||||
1. **Manual Buttons** (Planner/Writer pages)
|
||||
- Cluster Keywords → `auto_cluster`
|
||||
- Generate Ideas → `generate_ideas`
|
||||
- Generate Content → `generate_content`
|
||||
- Extract Prompts → `generate_image_prompts`
|
||||
- Generate Images → `generate_images`
|
||||
|
||||
2. **Automation Manual Run** (Automation page → Run Now)
|
||||
- Same AI functions called via AutomationService
|
||||
|
||||
3. **Scheduled Automation** (Celery Beat scheduler)
|
||||
- Same AI functions called via scheduled tasks
|
||||
|
||||
4. **Direct Services** (Linker, Optimizer modules)
|
||||
- Use CreditService directly for credit deduction
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
Migration is **COMPLETE** when:
|
||||
|
||||
- [ ] All integration tests pass
|
||||
- [ ] All manual test checklist items pass
|
||||
- [ ] No errors in production logs for 1 week
|
||||
- [ ] Credit calculations match expected values
|
||||
- [ ] All execution paths work (manual, automation, scheduled)
|
||||
- [ ] Frontend loads models dynamically from API
|
||||
- [ ] No legacy code remains:
|
||||
- [ ] No `MODEL_RATES` / `IMAGE_MODEL_RATES` in constants.py
|
||||
- [ ] No API keys in `GlobalIntegrationSettings`
|
||||
- [ ] No hardcoded models in frontend
|
||||
- [ ] No fallback to constants in `ModelRegistry`
|
||||
- [ ] No unused fields in `GlobalIntegrationSettings`
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference Commands
|
||||
|
||||
```bash
|
||||
# Run integration tests
|
||||
docker exec igny8_backend python manage.py test igny8_core.tests.test_ai_system_integration -v 2
|
||||
|
||||
# Check AIModelConfig
|
||||
docker exec igny8_backend python manage.py shell -c "
|
||||
from igny8_core.business.billing.models import AIModelConfig
|
||||
for m in AIModelConfig.objects.all().order_by('model_type'):
|
||||
print(f'{m.model_name}: {m.model_type}, default={m.is_default}')
|
||||
"
|
||||
|
||||
# Check recent credit usage
|
||||
docker exec igny8_backend python manage.py shell -c "
|
||||
from igny8_core.business.billing.models import CreditUsageLog
|
||||
for log in CreditUsageLog.objects.order_by('-created_at')[:10]:
|
||||
print(f'{log.operation_type}: {log.credits_used} credits')
|
||||
"
|
||||
|
||||
# Check account credits
|
||||
docker exec igny8_backend python manage.py shell -c "
|
||||
from igny8_core.auth.models import Account
|
||||
for acc in Account.objects.all():
|
||||
print(f'{acc.name}: {acc.credits} credits')
|
||||
"
|
||||
```
|
||||
Reference in New Issue
Block a user